[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:58:55.457335   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:44605
I20260709 12:58:55.459156   431 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:58:55.460337   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:55.478876   437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:55.481076   438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:55.479408   440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:56.715854   439 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1235 milliseconds
I20260709 12:58:56.716056   431 server_base.cc:1056] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260709 12:58:56.717481   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:56.717770   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:56.717998   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601936717972 us; error 0 us; skew 500 ppm
I20260709 12:58:56.718843   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:56.726078   431 webserver.cc:533] Webserver started at http://127.0.107.254:46395/ using document root <none> and password file <none>
I20260709 12:58:56.727064   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:56.727336   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:56.727820   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:56.734041   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/master-0-root/instance:
uuid: "9376f5f363e14159b00adddbd0d53fb5"
format_stamp: "Formatted at 2026-07-09 12:58:56 on dist-test-slave-jk05"
I20260709 12:58:56.742148   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:58:56.747853   446 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:56.749118   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 12:58:56.749562   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/master-0-root
uuid: "9376f5f363e14159b00adddbd0d53fb5"
format_stamp: "Formatted at 2026-07-09 12:58:56 on dist-test-slave-jk05"
I20260709 12:58:56.749941   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:56.803269   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:56.805327   431 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:58:56.805915   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:56.891304   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:44605
I20260709 12:58:56.891383   497 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:44605 every 8 connection(s)
I20260709 12:58:56.900328   498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:56.923341   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5: Bootstrap starting.
I20260709 12:58:56.929873   498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:56.932248   498 log.cc:826] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5: Log is configured to *not* fsync() on all Append() calls
I20260709 12:58:56.937070   498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5: No bootstrap required, opened a new log
I20260709 12:58:56.958524   498 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9376f5f363e14159b00adddbd0d53fb5" member_type: VOTER }
I20260709 12:58:56.959101   498 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:56.959396   498 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9376f5f363e14159b00adddbd0d53fb5, State: Initialized, Role: FOLLOWER
I20260709 12:58:56.960417   498 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9376f5f363e14159b00adddbd0d53fb5" member_type: VOTER }
I20260709 12:58:56.960918   498 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:58:56.961212   498 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:58:56.961593   498 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:56.965744   498 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9376f5f363e14159b00adddbd0d53fb5" member_type: VOTER }
I20260709 12:58:56.966538   498 leader_election.cc:304] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9376f5f363e14159b00adddbd0d53fb5; no voters: 
I20260709 12:58:56.968643   498 leader_election.cc:290] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:58:56.969024   501 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:56.971916   501 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [term 1 LEADER]: Becoming Leader. State: Replica: 9376f5f363e14159b00adddbd0d53fb5, State: Running, Role: LEADER
I20260709 12:58:56.973067   501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9376f5f363e14159b00adddbd0d53fb5" member_type: VOTER }
I20260709 12:58:56.973834   498 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:58:56.986222   502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9376f5f363e14159b00adddbd0d53fb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9376f5f363e14159b00adddbd0d53fb5" member_type: VOTER } }
I20260709 12:58:56.985973   503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9376f5f363e14159b00adddbd0d53fb5. Latest consensus state: current_term: 1 leader_uuid: "9376f5f363e14159b00adddbd0d53fb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9376f5f363e14159b00adddbd0d53fb5" member_type: VOTER } }
I20260709 12:58:56.986945   502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:58:56.986948   503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [sys.catalog]: This master's current role is: LEADER
I20260709 12:58:56.991671   511 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:58:57.007181   511 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:58:57.009948   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:58:57.027179   511 catalog_manager.cc:1383] Generated new cluster ID: b8f51d74c6294f4d9cd532cd2fbf0a12
I20260709 12:58:57.027498   511 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:58:57.054787   511 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:58:57.056603   511 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:58:57.079663   511 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5: Generated new TSK 0
I20260709 12:58:57.080796   511 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:58:57.141080   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:57.151152   520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:57.154346   521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:57.158344   523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:57.159780   431 server_base.cc:1061] running on GCE node
I20260709 12:58:57.160743   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:57.160996   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:57.161173   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601937161142 us; error 0 us; skew 500 ppm
I20260709 12:58:57.161837   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:57.166467   431 webserver.cc:533] Webserver started at http://127.0.107.193:37543/ using document root <none> and password file <none>
I20260709 12:58:57.167073   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:57.167310   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:57.167618   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:57.168807   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-0-root/instance:
uuid: "3bdacb3268db46cba845b6218fb33877"
format_stamp: "Formatted at 2026-07-09 12:58:57 on dist-test-slave-jk05"
I20260709 12:58:57.175457   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.001s	sys 0.007s
I20260709 12:58:57.180667   528 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.181589   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:58:57.181959   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-0-root
uuid: "3bdacb3268db46cba845b6218fb33877"
format_stamp: "Formatted at 2026-07-09 12:58:57 on dist-test-slave-jk05"
I20260709 12:58:57.182345   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:57.237605   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:57.239013   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:57.241904   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:58:57.247401   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:58:57.247692   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.247977   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:58:57.248180   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.367479   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:46807
I20260709 12:58:57.367561   590 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:46807 every 8 connection(s)
I20260709 12:58:57.375422   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:57.392642   595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:57.396319   596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:57.400336   598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:57.405463   431 server_base.cc:1061] running on GCE node
I20260709 12:58:57.406538   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:58:57.406845   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:57.407065   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601937407049 us; error 0 us; skew 500 ppm
I20260709 12:58:57.407859   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:57.411193   431 webserver.cc:533] Webserver started at http://127.0.107.194:43625/ using document root <none> and password file <none>
I20260709 12:58:57.411882   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:57.412156   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:57.412497   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:57.414053   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-1-root/instance:
uuid: "5e31c4f4178e4b799d0276d453376995"
format_stamp: "Formatted at 2026-07-09 12:58:57 on dist-test-slave-jk05"
I20260709 12:58:57.415493   591 heartbeater.cc:344] Connected to a master server at 127.0.107.254:44605
I20260709 12:58:57.415968   591 heartbeater.cc:461] Registering TS with master...
I20260709 12:58:57.416962   591 heartbeater.cc:507] Master 127.0.107.254:44605 requested a full tablet report, sending...
I20260709 12:58:57.420244   463 ts_manager.cc:194] Registered new tserver with Master: 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807)
I20260709 12:58:57.421859   431 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:58:57.423197   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57994
I20260709 12:58:57.427258   603 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.428198   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:58:57.428542   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-1-root
uuid: "5e31c4f4178e4b799d0276d453376995"
format_stamp: "Formatted at 2026-07-09 12:58:57 on dist-test-slave-jk05"
I20260709 12:58:57.428894   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:57.451293   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:57.453045   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:57.456028   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:58:57.459820   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:58:57.460108   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.460400   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:58:57.460605   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.563691   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:34357
I20260709 12:58:57.563846   665 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:34357 every 8 connection(s)
I20260709 12:58:57.573232   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:58:57.581344   669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:57.582867   670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:58:57.588578   673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:58:57.589162   431 server_base.cc:1061] running on GCE node
I20260709 12:58:57.589720   666 heartbeater.cc:344] Connected to a master server at 127.0.107.254:44605
I20260709 12:58:57.590152   666 heartbeater.cc:461] Registering TS with master...
I20260709 12:58:57.590616   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:58:57.590886   666 heartbeater.cc:507] Master 127.0.107.254:44605 requested a full tablet report, sending...
W20260709 12:58:57.590888   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:58:57.591907   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601937591886 us; error 0 us; skew 500 ppm
I20260709 12:58:57.592537   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:58:57.594099   463 ts_manager.cc:194] Registered new tserver with Master: 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:58:57.595527   431 webserver.cc:533] Webserver started at http://127.0.107.195:35537/ using document root <none> and password file <none>
I20260709 12:58:57.595623   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58004
I20260709 12:58:57.596176   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:58:57.596436   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:58:57.596763   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:58:57.598162   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-2-root/instance:
uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
format_stamp: "Formatted at 2026-07-09 12:58:57 on dist-test-slave-jk05"
I20260709 12:58:57.604451   431 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.001s	sys 0.005s
I20260709 12:58:57.608757   677 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.609716   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:58:57.610062   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-2-root
uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
format_stamp: "Formatted at 2026-07-09 12:58:57 on dist-test-slave-jk05"
I20260709 12:58:57.610445   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783601935286233-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:58:57.632934   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:58:57.634616   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:58:57.637674   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:58:57.641378   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:58:57.641701   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:58:57.641994   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:58:57.642205   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:58:57.757931   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:44803
I20260709 12:58:57.758401   739 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:44803 every 8 connection(s)
I20260709 12:58:57.776145   740 heartbeater.cc:344] Connected to a master server at 127.0.107.254:44605
I20260709 12:58:57.776546   740 heartbeater.cc:461] Registering TS with master...
I20260709 12:58:57.777379   740 heartbeater.cc:507] Master 127.0.107.254:44605 requested a full tablet report, sending...
I20260709 12:58:57.779829   463 ts_manager.cc:194] Registered new tserver with Master: 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:57.780545   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017094302s
I20260709 12:58:57.781075   463 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58016
I20260709 12:58:57.838274   463 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58030:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:58:57.841888   463 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:58:57.944893   631 tablet_service.cc:1511] Processing CreateTablet for tablet b053f7eed9c24e07a7aa1dd5f8f4ca4c (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:58:57.946947   630 tablet_service.cc:1511] Processing CreateTablet for tablet e62ff30fc2d84ec3aa02b4b15df20059 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:58:57.948824   630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e62ff30fc2d84ec3aa02b4b15df20059. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:57.950994   631 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b053f7eed9c24e07a7aa1dd5f8f4ca4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:57.951399   629 tablet_service.cc:1511] Processing CreateTablet for tablet 830161be8ab340f095080bafe7c11544 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:58:57.954344   629 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 830161be8ab340f095080bafe7c11544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:57.959883   628 tablet_service.cc:1511] Processing CreateTablet for tablet b76685879b204835ad5a1b543e151a61 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:58:57.961861   628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b76685879b204835ad5a1b543e151a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:57.989825   627 tablet_service.cc:1511] Processing CreateTablet for tablet 7c373e47ac8f4c409285379c51efb58c (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:58:57.994431   627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c373e47ac8f4c409285379c51efb58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.006789   553 tablet_service.cc:1511] Processing CreateTablet for tablet b76685879b204835ad5a1b543e151a61 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:58:58.004838   555 tablet_service.cc:1511] Processing CreateTablet for tablet e62ff30fc2d84ec3aa02b4b15df20059 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
W20260709 12:58:58.004832   510 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:58:58.006222   556 tablet_service.cc:1511] Processing CreateTablet for tablet b053f7eed9c24e07a7aa1dd5f8f4ca4c (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:58:58.022555   626 tablet_service.cc:1511] Processing CreateTablet for tablet 56505abb1d1d4125aa901433f539577d (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:58:58.024983   626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56505abb1d1d4125aa901433f539577d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.028162   553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b76685879b204835ad5a1b543e151a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.018877   552 tablet_service.cc:1511] Processing CreateTablet for tablet 7c373e47ac8f4c409285379c51efb58c (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:58:58.018494   554 tablet_service.cc:1511] Processing CreateTablet for tablet 830161be8ab340f095080bafe7c11544 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:58:58.033613   556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b053f7eed9c24e07a7aa1dd5f8f4ca4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.034142   704 tablet_service.cc:1511] Processing CreateTablet for tablet e62ff30fc2d84ec3aa02b4b15df20059 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:58:58.036216   551 tablet_service.cc:1511] Processing CreateTablet for tablet 56505abb1d1d4125aa901433f539577d (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:58:58.039134   703 tablet_service.cc:1511] Processing CreateTablet for tablet 830161be8ab340f095080bafe7c11544 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:58:58.032662   705 tablet_service.cc:1511] Processing CreateTablet for tablet b053f7eed9c24e07a7aa1dd5f8f4ca4c (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:58:58.041452   702 tablet_service.cc:1511] Processing CreateTablet for tablet b76685879b204835ad5a1b543e151a61 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:58:58.050101   701 tablet_service.cc:1511] Processing CreateTablet for tablet 7c373e47ac8f4c409285379c51efb58c (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:58:58.051313   704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e62ff30fc2d84ec3aa02b4b15df20059. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.050345   555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e62ff30fc2d84ec3aa02b4b15df20059. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.054401   551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56505abb1d1d4125aa901433f539577d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.065824   700 tablet_service.cc:1511] Processing CreateTablet for tablet 56505abb1d1d4125aa901433f539577d (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:58:58.061601   550 tablet_service.cc:1511] Processing CreateTablet for tablet 02c9b5f41d8c4aa987a40e3f070c7831 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:58:58.067595   705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b053f7eed9c24e07a7aa1dd5f8f4ca4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.074141   699 tablet_service.cc:1511] Processing CreateTablet for tablet 02c9b5f41d8c4aa987a40e3f070c7831 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:58:58.061412   625 tablet_service.cc:1511] Processing CreateTablet for tablet 02c9b5f41d8c4aa987a40e3f070c7831 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:58:58.079924   625 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02c9b5f41d8c4aa987a40e3f070c7831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.076653   552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c373e47ac8f4c409285379c51efb58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.085526   703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 830161be8ab340f095080bafe7c11544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.089012   700 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56505abb1d1d4125aa901433f539577d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.093281   550 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02c9b5f41d8c4aa987a40e3f070c7831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.099350   699 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02c9b5f41d8c4aa987a40e3f070c7831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.097257   758 tablet_bootstrap.cc:492] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995: Bootstrap starting.
I20260709 12:58:58.097886   549 tablet_service.cc:1511] Processing CreateTablet for tablet 5c1fa50662e24eff94fc63c2d5895884 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:58:58.106740   624 tablet_service.cc:1511] Processing CreateTablet for tablet 5c1fa50662e24eff94fc63c2d5895884 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:58:58.117121   554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 830161be8ab340f095080bafe7c11544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.116449   701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c373e47ac8f4c409285379c51efb58c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.120290   549 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c1fa50662e24eff94fc63c2d5895884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.126134   624 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c1fa50662e24eff94fc63c2d5895884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.136250   698 tablet_service.cc:1511] Processing CreateTablet for tablet 5c1fa50662e24eff94fc63c2d5895884 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:58:58.134786   702 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b76685879b204835ad5a1b543e151a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.153816   548 tablet_service.cc:1511] Processing CreateTablet for tablet d91fc29f3c5b4ce391f618db929bd3d6 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:58:58.148689   698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c1fa50662e24eff94fc63c2d5895884. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.159144   548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d91fc29f3c5b4ce391f618db929bd3d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.168170   630 tablet_service.cc:1511] Processing CreateTablet for tablet d91fc29f3c5b4ce391f618db929bd3d6 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:58:58.180152   697 tablet_service.cc:1511] Processing CreateTablet for tablet d91fc29f3c5b4ce391f618db929bd3d6 (DEFAULT_TABLE table=test-workload [id=66f7b2185470477390f0d2cf0c2f84f8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:58:58.188179   697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d91fc29f3c5b4ce391f618db929bd3d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.186650   630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d91fc29f3c5b4ce391f618db929bd3d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:58:58.272506   758 tablet_bootstrap.cc:654] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.289599   759 tablet_bootstrap.cc:492] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877: Bootstrap starting.
I20260709 12:58:58.340070   758 tablet_bootstrap.cc:492] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995: No bootstrap required, opened a new log
I20260709 12:58:58.340260   760 tablet_bootstrap.cc:492] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0: Bootstrap starting.
I20260709 12:58:58.340641   758 ts_tablet_manager.cc:1403] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995: Time spent bootstrapping tablet: real 0.248s	user 0.005s	sys 0.018s
I20260709 12:58:58.342748   759 tablet_bootstrap.cc:654] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.350443   760 tablet_bootstrap.cc:654] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.350517   758 raft_consensus.cc:359] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.351586   758 raft_consensus.cc:385] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.352432   758 raft_consensus.cc:740] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.353329   758 consensus_queue.cc:260] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.358757   759 tablet_bootstrap.cc:492] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877: No bootstrap required, opened a new log
I20260709 12:58:58.359264   759 ts_tablet_manager.cc:1403] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877: Time spent bootstrapping tablet: real 0.114s	user 0.008s	sys 0.013s
I20260709 12:58:58.362474   758 ts_tablet_manager.cc:1434] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.004s
I20260709 12:58:58.363095   760 tablet_bootstrap.cc:492] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0: No bootstrap required, opened a new log
I20260709 12:58:58.362758   759 raft_consensus.cc:359] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.363418   666 heartbeater.cc:499] Master 127.0.107.254:44605 was elected leader, sending a full tablet report...
I20260709 12:58:58.363551   760 ts_tablet_manager.cc:1403] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent bootstrapping tablet: real 0.029s	user 0.019s	sys 0.000s
I20260709 12:58:58.363518   759 raft_consensus.cc:385] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.363973   759 raft_consensus.cc:740] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.364710   759 consensus_queue.cc:260] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.367405   760 raft_consensus.cc:359] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.368180   760 raft_consensus.cc:385] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.368472   760 raft_consensus.cc:740] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.369410   760 consensus_queue.cc:260] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.373931   591 heartbeater.cc:499] Master 127.0.107.254:44605 was elected leader, sending a full tablet report...
W20260709 12:58:58.375301   592 tablet.cc:2406] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:58:58.376368   759 ts_tablet_manager.cc:1434] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877: Time spent starting tablet: real 0.017s	user 0.012s	sys 0.004s
I20260709 12:58:58.377547   759 tablet_bootstrap.cc:492] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877: Bootstrap starting.
I20260709 12:58:58.379719   760 ts_tablet_manager.cc:1434] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 12:58:58.380424   740 heartbeater.cc:499] Master 127.0.107.254:44605 was elected leader, sending a full tablet report...
I20260709 12:58:58.380752   760 tablet_bootstrap.cc:492] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0: Bootstrap starting.
I20260709 12:58:58.383138   758 tablet_bootstrap.cc:492] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995: Bootstrap starting.
I20260709 12:58:58.392624   759 tablet_bootstrap.cc:654] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.392624   758 tablet_bootstrap.cc:654] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.399348   760 tablet_bootstrap.cc:654] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.409096   758 tablet_bootstrap.cc:492] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995: No bootstrap required, opened a new log
I20260709 12:58:58.409709   758 ts_tablet_manager.cc:1403] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.007s
I20260709 12:58:58.413261   759 tablet_bootstrap.cc:492] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877: No bootstrap required, opened a new log
I20260709 12:58:58.414753   759 ts_tablet_manager.cc:1403] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.000s
I20260709 12:58:58.417752   759 raft_consensus.cc:359] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.413805   758 raft_consensus.cc:359] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.418689   758 raft_consensus.cc:385] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.419065   758 raft_consensus.cc:740] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.419863   758 consensus_queue.cc:260] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.422099   758 ts_tablet_manager.cc:1434] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 12:58:58.423475   758 tablet_bootstrap.cc:492] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995: Bootstrap starting.
I20260709 12:58:58.429992   758 tablet_bootstrap.cc:654] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.422437   759 raft_consensus.cc:385] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.430433   759 raft_consensus.cc:740] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.431630   759 consensus_queue.cc:260] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.434152   759 ts_tablet_manager.cc:1434] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:58:58.434952   764 raft_consensus.cc:493] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.435745   759 tablet_bootstrap.cc:492] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877: Bootstrap starting.
I20260709 12:58:58.435658   764 raft_consensus.cc:515] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.443310   759 tablet_bootstrap.cc:654] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.445175   764 leader_election.cc:290] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:58.470966   760 tablet_bootstrap.cc:492] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0: No bootstrap required, opened a new log
I20260709 12:58:58.471531   760 ts_tablet_manager.cc:1403] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent bootstrapping tablet: real 0.091s	user 0.019s	sys 0.056s
I20260709 12:58:58.472680   758 tablet_bootstrap.cc:492] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995: No bootstrap required, opened a new log
I20260709 12:58:58.473217   758 ts_tablet_manager.cc:1403] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.004s
I20260709 12:58:58.475158   760 raft_consensus.cc:359] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.475966   760 raft_consensus.cc:385] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.476521   760 raft_consensus.cc:740] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.476785   758 raft_consensus.cc:359] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.477694   758 raft_consensus.cc:385] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.478024   758 raft_consensus.cc:740] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.480820   765 raft_consensus.cc:493] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.481410   760 consensus_queue.cc:260] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.481364   765 raft_consensus.cc:515] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.483912   759 tablet_bootstrap.cc:492] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877: No bootstrap required, opened a new log
I20260709 12:58:58.487473   765 leader_election.cc:290] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:58.489804   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76685879b204835ad5a1b543e151a61" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877" is_pre_election: true
I20260709 12:58:58.491035   760 ts_tablet_manager.cc:1434] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.004s
I20260709 12:58:58.487521   759 ts_tablet_manager.cc:1403] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877: Time spent bootstrapping tablet: real 0.052s	user 0.007s	sys 0.014s
W20260709 12:58:58.501777   607 leader_election.cc:343] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.502532   760 tablet_bootstrap.cc:492] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0: Bootstrap starting.
I20260709 12:58:58.485821   758 consensus_queue.cc:260] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.518105   759 raft_consensus.cc:359] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.519088   759 raft_consensus.cc:385] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.519713   759 raft_consensus.cc:740] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.519742   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76685879b204835ad5a1b543e151a61" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" is_pre_election: true
I20260709 12:58:58.520480   759 consensus_queue.cc:260] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
W20260709 12:58:58.528612   606 leader_election.cc:343] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.529099   606 leader_election.cc:304] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e31c4f4178e4b799d0276d453376995; no voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0
I20260709 12:58:58.530097   764 raft_consensus.cc:2749] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:58.531546   758 ts_tablet_manager.cc:1434] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995: Time spent starting tablet: real 0.058s	user 0.001s	sys 0.008s
I20260709 12:58:58.540493   760 tablet_bootstrap.cc:654] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.532574   758 tablet_bootstrap.cc:492] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995: Bootstrap starting.
I20260709 12:58:58.543076   764 raft_consensus.cc:493] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.543951   759 ts_tablet_manager.cc:1434] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877: Time spent starting tablet: real 0.044s	user 0.005s	sys 0.003s
I20260709 12:58:58.543681   764 raft_consensus.cc:515] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.544919   759 tablet_bootstrap.cc:492] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877: Bootstrap starting.
I20260709 12:58:58.546324   764 leader_election.cc:290] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:58.547459   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830161be8ab340f095080bafe7c11544" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877" is_pre_election: true
I20260709 12:58:58.547730   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830161be8ab340f095080bafe7c11544" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" is_pre_election: true
W20260709 12:58:58.549281   607 leader_election.cc:343] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:58:58.549882   606 leader_election.cc:343] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.550321   606 leader_election.cc:304] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e31c4f4178e4b799d0276d453376995; no voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0
I20260709 12:58:58.551113   764 raft_consensus.cc:2749] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:58.562219   758 tablet_bootstrap.cc:654] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.567847   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b053f7eed9c24e07a7aa1dd5f8f4ca4c" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" is_pre_election: true
W20260709 12:58:58.569229   531 leader_election.cc:343] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.572162   760 tablet_bootstrap.cc:492] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0: No bootstrap required, opened a new log
I20260709 12:58:58.572641   760 ts_tablet_manager.cc:1403] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent bootstrapping tablet: real 0.070s	user 0.009s	sys 0.013s
I20260709 12:58:58.576279   760 raft_consensus.cc:359] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.577049   760 raft_consensus.cc:385] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.577343   760 raft_consensus.cc:740] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.577965   759 tablet_bootstrap.cc:654] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.578140   760 consensus_queue.cc:260] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.583052   760 ts_tablet_manager.cc:1434] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:58:58.585315   760 tablet_bootstrap.cc:492] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0: Bootstrap starting.
I20260709 12:58:58.586118   758 tablet_bootstrap.cc:492] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995: No bootstrap required, opened a new log
I20260709 12:58:58.587030   758 ts_tablet_manager.cc:1403] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995: Time spent bootstrapping tablet: real 0.055s	user 0.013s	sys 0.010s
I20260709 12:58:58.593029   758 raft_consensus.cc:359] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.594216   766 raft_consensus.cc:493] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.595688   758 raft_consensus.cc:385] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.596010   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b053f7eed9c24e07a7aa1dd5f8f4ca4c" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995" is_pre_election: true
I20260709 12:58:58.596537   766 raft_consensus.cc:515] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.597313   760 tablet_bootstrap.cc:654] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.598057   641 raft_consensus.cc:2468] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3bdacb3268db46cba845b6218fb33877 in term 0.
I20260709 12:58:58.597500   758 raft_consensus.cc:740] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.599824   758 consensus_queue.cc:260] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.600940   530 leader_election.cc:304] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5e31c4f4178e4b799d0276d453376995; no voters: 5d26397b4ee543e8b8920dc10bbc54c0
I20260709 12:58:58.603717   758 ts_tablet_manager.cc:1434] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260709 12:58:58.603271   765 raft_consensus.cc:2804] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:58.604204   765 raft_consensus.cc:493] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:58.604537   765 raft_consensus.cc:3060] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:58.605284   758 tablet_bootstrap.cc:492] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995: Bootstrap starting.
I20260709 12:58:58.617987   766 leader_election.cc:290] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:58:58.620637   758 tablet_bootstrap.cc:654] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.634332   766 raft_consensus.cc:493] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.635788   766 raft_consensus.cc:515] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.633703   765 raft_consensus.cc:515] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.639767   766 leader_election.cc:290] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:58:58.643749   759 tablet_bootstrap.cc:492] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877: No bootstrap required, opened a new log
I20260709 12:58:58.650975   759 ts_tablet_manager.cc:1403] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877: Time spent bootstrapping tablet: real 0.106s	user 0.017s	sys 0.013s
I20260709 12:58:58.655738   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b053f7eed9c24e07a7aa1dd5f8f4ca4c" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:58:58.656325   641 raft_consensus.cc:3060] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:58.661523   759 raft_consensus.cc:359] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.663841   758 tablet_bootstrap.cc:492] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995: No bootstrap required, opened a new log
I20260709 12:58:58.664152   641 raft_consensus.cc:2468] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bdacb3268db46cba845b6218fb33877 in term 1.
I20260709 12:58:58.664603   758 ts_tablet_manager.cc:1403] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.018s
I20260709 12:58:58.665453   530 leader_election.cc:304] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:58:58.666177   765 leader_election.cc:290] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 election: Requested vote from peers 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:58.664183   759 raft_consensus.cc:385] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.668145   759 raft_consensus.cc:740] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.666741   765 raft_consensus.cc:493] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.674645   765 raft_consensus.cc:515] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.674507   759 consensus_queue.cc:260] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.679797   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b053f7eed9c24e07a7aa1dd5f8f4ca4c" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:58:58.681233   786 raft_consensus.cc:2804] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:58.683044   786 raft_consensus.cc:697] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 1 LEADER]: Becoming Leader. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Running, Role: LEADER
I20260709 12:58:58.683113   765 leader_election.cc:290] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:58.683952   786 consensus_queue.cc:237] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.686661   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d91fc29f3c5b4ce391f618db929bd3d6" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995" is_pre_election: true
I20260709 12:58:58.687539   759 ts_tablet_manager.cc:1434] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.003s
I20260709 12:58:58.689862   758 raft_consensus.cc:359] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.693807   760 tablet_bootstrap.cc:492] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0: No bootstrap required, opened a new log
W20260709 12:58:58.693902   531 leader_election.cc:343] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.694564   760 ts_tablet_manager.cc:1403] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent bootstrapping tablet: real 0.110s	user 0.019s	sys 0.023s
W20260709 12:58:58.691511   530 leader_election.cc:343] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.692698   758 raft_consensus.cc:385] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.694859   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d91fc29f3c5b4ce391f618db929bd3d6" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" is_pre_election: true
I20260709 12:58:58.696285   758 raft_consensus.cc:740] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.697147   758 consensus_queue.cc:260] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.699021   759 tablet_bootstrap.cc:492] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877: Bootstrap starting.
I20260709 12:58:58.698726   760 raft_consensus.cc:359] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.699465   760 raft_consensus.cc:385] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.699805   758 ts_tablet_manager.cc:1434] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.004s
I20260709 12:58:58.699759   760 raft_consensus.cc:740] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Initialized, Role: FOLLOWER
W20260709 12:58:58.698019   531 leader_election.cc:343] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.700975   531 leader_election.cc:304] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877; no voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995
I20260709 12:58:58.702292   765 raft_consensus.cc:2749] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:58.705025   758 tablet_bootstrap.cc:492] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995: Bootstrap starting.
I20260709 12:58:58.710458   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877" is_pre_election: true
I20260709 12:58:58.711094   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877" is_pre_election: true
W20260709 12:58:58.718824   681 leader_election.cc:343] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.718961   566 raft_consensus.cc:2468] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 0.
I20260709 12:58:58.720142   681 leader_election.cc:304] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 
I20260709 12:58:58.721215   766 raft_consensus.cc:2804] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:58.721480   759 tablet_bootstrap.cc:654] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.721650   766 raft_consensus.cc:493] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:58.722012   766 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:58.712862   758 tablet_bootstrap.cc:654] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.713840   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995" is_pre_election: true
I20260709 12:58:58.714968   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995" is_pre_election: true
I20260709 12:58:58.726083   641 raft_consensus.cc:2468] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 0.
W20260709 12:58:58.727641   679 leader_election.cc:343] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.728616   679 leader_election.cc:304] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 3bdacb3268db46cba845b6218fb33877, 5e31c4f4178e4b799d0276d453376995
I20260709 12:58:58.704532   760 consensus_queue.cc:260] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.728662   458 catalog_manager.cc:5697] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3bdacb3268db46cba845b6218fb33877 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "3bdacb3268db46cba845b6218fb33877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:58.733932   789 raft_consensus.cc:2749] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:58.731662   766 raft_consensus.cc:515] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.737910   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:58:58.738637   566 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:58.739132   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:58:58.739722   641 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:58.744340   759 tablet_bootstrap.cc:492] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877: No bootstrap required, opened a new log
I20260709 12:58:58.744867   759 ts_tablet_manager.cc:1403] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877: Time spent bootstrapping tablet: real 0.051s	user 0.009s	sys 0.013s
I20260709 12:58:58.746457   566 raft_consensus.cc:2468] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 1.
I20260709 12:58:58.746748   641 raft_consensus.cc:2468] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 1.
I20260709 12:58:58.747210   766 leader_election.cc:290] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:58:58.749837   759 raft_consensus.cc:359] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.750618   759 raft_consensus.cc:385] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.750926   759 raft_consensus.cc:740] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.751103   679 leader_election.cc:304] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:58:58.757782   766 raft_consensus.cc:2804] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:58.758526   766 raft_consensus.cc:697] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Becoming Leader. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Running, Role: LEADER
I20260709 12:58:58.759265   760 ts_tablet_manager.cc:1434] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent starting tablet: real 0.064s	user 0.011s	sys 0.002s
I20260709 12:58:58.760341   760 tablet_bootstrap.cc:492] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0: Bootstrap starting.
I20260709 12:58:58.762902   758 tablet_bootstrap.cc:492] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995: No bootstrap required, opened a new log
I20260709 12:58:58.763360   758 ts_tablet_manager.cc:1403] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.007s
I20260709 12:58:58.759841   766 consensus_queue.cc:237] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.766964   758 raft_consensus.cc:359] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.769606   759 consensus_queue.cc:260] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.770581   758 raft_consensus.cc:385] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.774309   758 raft_consensus.cc:740] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.773514   759 ts_tablet_manager.cc:1434] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.006s
I20260709 12:58:58.776263   759 tablet_bootstrap.cc:492] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877: Bootstrap starting.
I20260709 12:58:58.776018   758 consensus_queue.cc:260] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.773356   760 tablet_bootstrap.cc:654] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.784150   758 ts_tablet_manager.cc:1434] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.003s
I20260709 12:58:58.785239   758 tablet_bootstrap.cc:492] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995: Bootstrap starting.
I20260709 12:58:58.792505   758 tablet_bootstrap.cc:654] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.804986   759 tablet_bootstrap.cc:654] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.809984   758 tablet_bootstrap.cc:492] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995: No bootstrap required, opened a new log
I20260709 12:58:58.810681   758 ts_tablet_manager.cc:1403] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.012s
I20260709 12:58:58.814492   760 tablet_bootstrap.cc:492] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0: No bootstrap required, opened a new log
I20260709 12:58:58.815217   760 ts_tablet_manager.cc:1403] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.017s
I20260709 12:58:58.817678   758 raft_consensus.cc:359] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.819489   758 raft_consensus.cc:385] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.819829   758 raft_consensus.cc:740] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.820626   758 consensus_queue.cc:260] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.820580   760 raft_consensus.cc:359] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.823122   759 tablet_bootstrap.cc:492] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877: No bootstrap required, opened a new log
I20260709 12:58:58.823412   760 raft_consensus.cc:385] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.823796   759 ts_tablet_manager.cc:1403] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.007s
I20260709 12:58:58.823825   760 raft_consensus.cc:740] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.824908   758 ts_tablet_manager.cc:1434] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 12:58:58.825011   760 consensus_queue.cc:260] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.826215   758 tablet_bootstrap.cc:492] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995: Bootstrap starting.
I20260709 12:58:58.831964   760 ts_tablet_manager.cc:1434] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent starting tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 12:58:58.831905   462 catalog_manager.cc:5697] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:58.836896   760 tablet_bootstrap.cc:492] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0: Bootstrap starting.
I20260709 12:58:58.841054   764 raft_consensus.cc:493] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.841590   764 raft_consensus.cc:515] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.843777   764 leader_election.cc:290] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:58.844590   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56505abb1d1d4125aa901433f539577d" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877" is_pre_election: true
I20260709 12:58:58.846540   760 tablet_bootstrap.cc:654] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.849318   758 tablet_bootstrap.cc:654] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.850700   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56505abb1d1d4125aa901433f539577d" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" is_pre_election: true
I20260709 12:58:58.851359   715 raft_consensus.cc:2468] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 0.
I20260709 12:58:58.853744   606 leader_election.cc:304] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:58:58.854516   764 raft_consensus.cc:2804] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:58.854859   764 raft_consensus.cc:493] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:58.855196   764 raft_consensus.cc:3060] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:58:58.855458   607 leader_election.cc:343] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.857923   759 raft_consensus.cc:359] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.858719   759 raft_consensus.cc:385] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.859052   759 raft_consensus.cc:740] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.859658   760 tablet_bootstrap.cc:492] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0: No bootstrap required, opened a new log
I20260709 12:58:58.860181   760 ts_tablet_manager.cc:1403] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent bootstrapping tablet: real 0.024s	user 0.020s	sys 0.000s
I20260709 12:58:58.859814   759 consensus_queue.cc:260] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.861990   764 raft_consensus.cc:515] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.862682   759 ts_tablet_manager.cc:1434] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.004s
I20260709 12:58:58.863678   759 tablet_bootstrap.cc:492] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877: Bootstrap starting.
I20260709 12:58:58.864006   764 leader_election.cc:290] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:58.863720   760 raft_consensus.cc:359] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.864614   760 raft_consensus.cc:385] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.864954   760 raft_consensus.cc:740] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.865751   760 consensus_queue.cc:260] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.868916   760 ts_tablet_manager.cc:1434] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.001s
I20260709 12:58:58.869974   760 tablet_bootstrap.cc:492] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0: Bootstrap starting.
I20260709 12:58:58.871620   759 tablet_bootstrap.cc:654] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.872635   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56505abb1d1d4125aa901433f539577d" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877"
W20260709 12:58:58.874137   607 leader_election.cc:343] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.875855   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56505abb1d1d4125aa901433f539577d" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:58:58.876470   715 raft_consensus.cc:3060] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:58.882670   760 tablet_bootstrap.cc:654] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.884733   759 tablet_bootstrap.cc:492] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877: No bootstrap required, opened a new log
I20260709 12:58:58.885313   759 ts_tablet_manager.cc:1403] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.007s
I20260709 12:58:58.889953   759 raft_consensus.cc:359] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.891079   759 raft_consensus.cc:385] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.891532   759 raft_consensus.cc:740] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.892671   759 consensus_queue.cc:260] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.896095   759 ts_tablet_manager.cc:1434] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:58:58.898618   759 tablet_bootstrap.cc:492] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877: Bootstrap starting.
I20260709 12:58:58.903761   760 tablet_bootstrap.cc:492] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0: No bootstrap required, opened a new log
I20260709 12:58:58.904310   760 ts_tablet_manager.cc:1403] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.012s
I20260709 12:58:58.906832   759 tablet_bootstrap.cc:654] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.908011   760 raft_consensus.cc:359] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.908797   760 raft_consensus.cc:385] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.909113   760 raft_consensus.cc:740] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.909224   766 raft_consensus.cc:493] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.909752   766 raft_consensus.cc:515] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.910597   715 raft_consensus.cc:2468] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 1.
I20260709 12:58:58.909987   760 consensus_queue.cc:260] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.911921   766 leader_election.cc:290] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:58:58.911886   606 leader_election.cc:304] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995; no voters: 3bdacb3268db46cba845b6218fb33877
I20260709 12:58:58.912796   764 raft_consensus.cc:2804] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:58.913479   760 ts_tablet_manager.cc:1434] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.007s
I20260709 12:58:58.914636   760 tablet_bootstrap.cc:492] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0: Bootstrap starting.
I20260709 12:58:58.915107   790 raft_consensus.cc:493] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.915058   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76685879b204835ad5a1b543e151a61" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877" is_pre_election: true
I20260709 12:58:58.915752   566 raft_consensus.cc:2468] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 0.
I20260709 12:58:58.915628   790 raft_consensus.cc:515] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.917017   681 leader_election.cc:304] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 
I20260709 12:58:58.917856   790 leader_election.cc:290] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:58:58.918390   790 raft_consensus.cc:2804] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:58.918747   790 raft_consensus.cc:493] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:58.918893   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d91fc29f3c5b4ce391f618db929bd3d6" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877" is_pre_election: true
I20260709 12:58:58.919287   790 raft_consensus.cc:3060] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:58.919586   566 raft_consensus.cc:2468] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 0.
I20260709 12:58:58.920872   759 tablet_bootstrap.cc:492] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877: No bootstrap required, opened a new log
I20260709 12:58:58.920751   681 leader_election.cc:304] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 
I20260709 12:58:58.921464   759 ts_tablet_manager.cc:1403] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.005s
I20260709 12:58:58.921646   766 raft_consensus.cc:2804] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:58.921909   760 tablet_bootstrap.cc:654] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.922068   766 raft_consensus.cc:493] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:58.922465   766 raft_consensus.cc:3060] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:58.922987   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76685879b204835ad5a1b543e151a61" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995" is_pre_election: true
I20260709 12:58:58.923636   641 raft_consensus.cc:2468] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 0.
I20260709 12:58:58.924733   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d91fc29f3c5b4ce391f618db929bd3d6" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995" is_pre_election: true
I20260709 12:58:58.925607   759 raft_consensus.cc:359] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
W20260709 12:58:58.926744   679 leader_election.cc:343] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:58.927057   759 raft_consensus.cc:385] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:58.927559   764 raft_consensus.cc:697] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Becoming Leader. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:58:58.927651   759 raft_consensus.cc:740] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.928498   764 consensus_queue.cc:237] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.928925   759 consensus_queue.cc:260] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.932948   759 ts_tablet_manager.cc:1434] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:58:58.934962   759 tablet_bootstrap.cc:492] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877: Bootstrap starting.
I20260709 12:58:58.938905   790 raft_consensus.cc:515] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.939281   804 raft_consensus.cc:493] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.940053   804 raft_consensus.cc:515] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.941972   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76685879b204835ad5a1b543e151a61" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:58:58.942024   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76685879b204835ad5a1b543e151a61" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:58:58.942754   640 raft_consensus.cc:3060] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:58.944044   804 leader_election.cc:290] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:58.944953   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877" is_pre_election: true
I20260709 12:58:58.945665   565 raft_consensus.cc:2468] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 0.
I20260709 12:58:58.947824   607 leader_election.cc:304] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:58:58.949716   804 raft_consensus.cc:2804] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:58.950129   566 raft_consensus.cc:3060] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:58.950209   804 raft_consensus.cc:493] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:58.950780   804 raft_consensus.cc:3060] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:58.953260   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" is_pre_election: true
I20260709 12:58:58.954762   715 raft_consensus.cc:2468] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 0.
I20260709 12:58:58.964505   640 raft_consensus.cc:2468] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 1.
I20260709 12:58:58.965992   679 leader_election.cc:304] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:58:58.966910   789 raft_consensus.cc:2804] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:58.967023   459 catalog_manager.cc:5697] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e31c4f4178e4b799d0276d453376995 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5e31c4f4178e4b799d0276d453376995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:58.981603   789 raft_consensus.cc:697] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Becoming Leader. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Running, Role: LEADER
I20260709 12:58:58.973541   790 leader_election.cc:290] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:58:58.994495   759 tablet_bootstrap.cc:654] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:58.995240   786 raft_consensus.cc:493] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:58.995736   786 raft_consensus.cc:515] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:58.996618   760 tablet_bootstrap.cc:492] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0: No bootstrap required, opened a new log
I20260709 12:58:58.996678   566 raft_consensus.cc:2468] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 1.
I20260709 12:58:58.997146   760 ts_tablet_manager.cc:1403] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent bootstrapping tablet: real 0.083s	user 0.011s	sys 0.046s
I20260709 12:58:59.000037   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56505abb1d1d4125aa901433f539577d" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995" is_pre_election: true
I20260709 12:58:59.000765   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56505abb1d1d4125aa901433f539577d" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" is_pre_election: true
I20260709 12:58:59.001780   760 raft_consensus.cc:359] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.002254   760 raft_consensus.cc:385] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.002485   760 raft_consensus.cc:740] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Initialized, Role: FOLLOWER
I20260709 12:58:58.999078   786 leader_election.cc:290] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:59.003397   715 raft_consensus.cc:2393] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bdacb3268db46cba845b6218fb33877 in current term 1: Already voted for candidate 5e31c4f4178e4b799d0276d453376995 in this term.
I20260709 12:58:59.003588   760 consensus_queue.cc:260] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.005312   808 raft_consensus.cc:493] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.005023   766 raft_consensus.cc:515] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.005791   808 raft_consensus.cc:515] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.006078   531 leader_election.cc:304] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877; no voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995
I20260709 12:58:59.001266   804 raft_consensus.cc:515] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.008311   786 raft_consensus.cc:3060] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.009770   760 ts_tablet_manager.cc:1434] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:58:59.009179   566 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d91fc29f3c5b4ce391f618db929bd3d6" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:58:59.010859   760 tablet_bootstrap.cc:492] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0: Bootstrap starting.
I20260709 12:58:59.012928   566 raft_consensus.cc:3060] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.010138   766 leader_election.cc:290] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:58:59.015926   764 raft_consensus.cc:493] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.003818   789 consensus_queue.cc:237] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.017578   764 raft_consensus.cc:515] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
W20260709 12:58:59.019110   510 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:58:59.021328   764 leader_election.cc:290] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:59.022114   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1fa50662e24eff94fc63c2d5895884" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877" is_pre_election: true
I20260709 12:58:59.023500   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830161be8ab340f095080bafe7c11544" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877" is_pre_election: true
I20260709 12:58:59.023684   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1fa50662e24eff94fc63c2d5895884" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" is_pre_election: true
I20260709 12:58:59.024132   564 raft_consensus.cc:2468] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 0.
I20260709 12:58:59.024282   715 raft_consensus.cc:2468] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 0.
I20260709 12:58:59.025166   808 leader_election.cc:290] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:58:59.026319   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830161be8ab340f095080bafe7c11544" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995" is_pre_election: true
I20260709 12:58:59.026921   641 raft_consensus.cc:2468] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 0.
I20260709 12:58:59.027156   681 leader_election.cc:304] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 
I20260709 12:58:59.027853   606 leader_election.cc:304] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995; no voters: 
W20260709 12:58:59.028481   607 leader_election.cc:343] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:59.028626   804 leader_election.cc:290] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:59.029265   808 raft_consensus.cc:2804] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:59.029663   808 raft_consensus.cc:493] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:59.030012   808 raft_consensus.cc:3060] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.030865   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:58:59.031432   715 raft_consensus.cc:3060] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.032346   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:58:59.033329   760 tablet_bootstrap.cc:654] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.033655   804 raft_consensus.cc:2804] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:59.034335   804 raft_consensus.cc:493] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:59.033027   564 raft_consensus.cc:3060] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.034677   804 raft_consensus.cc:3060] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.038211   808 raft_consensus.cc:515] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.041115   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830161be8ab340f095080bafe7c11544" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:58:59.041749   565 raft_consensus.cc:3060] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.018517   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d91fc29f3c5b4ce391f618db929bd3d6" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:58:59.046844   566 raft_consensus.cc:2468] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 1.
I20260709 12:58:59.048153   681 leader_election.cc:304] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 
W20260709 12:58:59.049789   679 leader_election.cc:343] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:59.051013   766 raft_consensus.cc:2804] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.054368   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830161be8ab340f095080bafe7c11544" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:58:59.054659   766 raft_consensus.cc:697] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Becoming Leader. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Running, Role: LEADER
I20260709 12:58:59.055009   640 raft_consensus.cc:3060] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.055392   758 tablet_bootstrap.cc:492] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995: No bootstrap required, opened a new log
I20260709 12:58:59.057301   758 ts_tablet_manager.cc:1403] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995: Time spent bootstrapping tablet: real 0.231s	user 0.017s	sys 0.033s
I20260709 12:58:59.055123   565 raft_consensus.cc:2468] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 1.
I20260709 12:58:59.058670   681 leader_election.cc:304] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 
I20260709 12:58:59.059453   790 raft_consensus.cc:2804] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.063484   759 tablet_bootstrap.cc:492] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877: No bootstrap required, opened a new log
I20260709 12:58:59.064078   759 ts_tablet_manager.cc:1403] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877: Time spent bootstrapping tablet: real 0.130s	user 0.014s	sys 0.015s
I20260709 12:58:59.061815   808 leader_election.cc:290] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:58:59.068882   760 tablet_bootstrap.cc:492] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0: No bootstrap required, opened a new log
I20260709 12:58:59.069470   760 ts_tablet_manager.cc:1403] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.002s
I20260709 12:58:59.055938   766 consensus_queue.cc:237] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.071512   640 raft_consensus.cc:2468] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 1.
I20260709 12:58:59.052383   786 raft_consensus.cc:2749] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:59.053889   715 raft_consensus.cc:2468] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 1.
I20260709 12:58:59.074043   564 raft_consensus.cc:2468] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 1.
I20260709 12:58:59.074282   606 leader_election.cc:304] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:58:59.077491   764 raft_consensus.cc:2804] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.077548   790 raft_consensus.cc:697] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Becoming Leader. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Running, Role: LEADER
I20260709 12:58:59.078059   764 raft_consensus.cc:697] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Becoming Leader. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:58:59.078331   790 consensus_queue.cc:237] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.078863   764 consensus_queue.cc:237] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.081821   812 raft_consensus.cc:493] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.082499   812 raft_consensus.cc:515] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.086561   758 raft_consensus.cc:359] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.087251   758 raft_consensus.cc:385] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.087199   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b053f7eed9c24e07a7aa1dd5f8f4ca4c" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995" is_pre_election: true
I20260709 12:58:59.087626   758 raft_consensus.cc:740] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.076215   804 raft_consensus.cc:515] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.088328   758 consensus_queue.cc:260] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.088153   640 raft_consensus.cc:2393] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5d26397b4ee543e8b8920dc10bbc54c0 in current term 1: Already voted for candidate 3bdacb3268db46cba845b6218fb33877 in this term.
I20260709 12:58:59.091462   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1fa50662e24eff94fc63c2d5895884" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877"
W20260709 12:58:59.094254   607 leader_election.cc:343] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:58:59.095319   459 catalog_manager.cc:5697] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:59.096697   804 leader_election.cc:290] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:59.097775   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1fa50662e24eff94fc63c2d5895884" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:58:59.098342   715 raft_consensus.cc:3060] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.107359   715 raft_consensus.cc:2468] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 1.
I20260709 12:58:59.122879   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b053f7eed9c24e07a7aa1dd5f8f4ca4c" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877" is_pre_election: true
I20260709 12:58:59.124902   758 ts_tablet_manager.cc:1434] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995: Time spent starting tablet: real 0.067s	user 0.004s	sys 0.004s
I20260709 12:58:59.124549   760 raft_consensus.cc:359] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.125756   760 raft_consensus.cc:385] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.125957   758 tablet_bootstrap.cc:492] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995: Bootstrap starting.
I20260709 12:58:59.126034   760 raft_consensus.cc:740] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.127017   681 leader_election.cc:304] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 3bdacb3268db46cba845b6218fb33877, 5e31c4f4178e4b799d0276d453376995
I20260709 12:58:59.126824   760 consensus_queue.cc:260] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.124780   606 leader_election.cc:304] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995; no voters: 3bdacb3268db46cba845b6218fb33877
I20260709 12:58:59.128403   714 raft_consensus.cc:3060] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.123212   759 raft_consensus.cc:359] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.129050   759 raft_consensus.cc:385] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.129247   760 ts_tablet_manager.cc:1434] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0: Time spent starting tablet: real 0.059s	user 0.005s	sys 0.002s
I20260709 12:58:59.129329   759 raft_consensus.cc:740] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.129689   764 raft_consensus.cc:2804] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.130149   759 consensus_queue.cc:260] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.124571   812 leader_election.cc:290] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357), 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807)
I20260709 12:58:59.137391   759 ts_tablet_manager.cc:1434] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877: Time spent starting tablet: real 0.073s	user 0.005s	sys 0.003s
I20260709 12:58:59.137027   459 catalog_manager.cc:5697] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:59.139256   459 catalog_manager.cc:5697] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: HEALTHY } } }
I20260709 12:58:59.141928   758 tablet_bootstrap.cc:654] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995: Neither blocks nor log segments found. Creating new log.
I20260709 12:58:59.143677   462 catalog_manager.cc:5697] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e31c4f4178e4b799d0276d453376995 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5e31c4f4178e4b799d0276d453376995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.137528   764 raft_consensus.cc:697] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Becoming Leader. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:58:59.147265   764 consensus_queue.cc:237] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.161796   790 raft_consensus.cc:2749] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:59.165330   758 tablet_bootstrap.cc:492] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995: No bootstrap required, opened a new log
I20260709 12:58:59.165908   758 ts_tablet_manager.cc:1403] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.001s
I20260709 12:58:59.166453   786 consensus_queue.cc:1048] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:59.168586   758 raft_consensus.cc:359] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.169252   758 raft_consensus.cc:385] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:58:59.169556   758 raft_consensus.cc:740] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Initialized, Role: FOLLOWER
I20260709 12:58:59.171017   758 consensus_queue.cc:260] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.173990   758 ts_tablet_manager.cc:1434] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:58:59.185766   461 catalog_manager.cc:5697] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e31c4f4178e4b799d0276d453376995 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5e31c4f4178e4b799d0276d453376995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.207700   765 consensus_queue.cc:1048] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.279053   764 raft_consensus.cc:493] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.279610   764 raft_consensus.cc:515] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.281740   764 leader_election.cc:290] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:59.282735   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c373e47ac8f4c409285379c51efb58c" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877" is_pre_election: true
I20260709 12:58:59.282774   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c373e47ac8f4c409285379c51efb58c" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" is_pre_election: true
I20260709 12:58:59.283818   564 raft_consensus.cc:2468] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 0.
I20260709 12:58:59.284047   714 raft_consensus.cc:2468] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 0.
I20260709 12:58:59.285768   607 leader_election.cc:304] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:58:59.286633   764 raft_consensus.cc:2804] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:58:59.287120   764 raft_consensus.cc:493] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:58:59.287621   764 raft_consensus.cc:3060] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.294888   764 raft_consensus.cc:515] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.297444   764 leader_election.cc:290] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:59.298161   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c373e47ac8f4c409285379c51efb58c" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:58:59.298672   564 raft_consensus.cc:3060] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.302788   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c373e47ac8f4c409285379c51efb58c" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:58:59.303438   714 raft_consensus.cc:3060] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.304049   564 raft_consensus.cc:2468] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 1.
I20260709 12:58:59.305234   607 leader_election.cc:304] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:58:59.306268   764 raft_consensus.cc:2804] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:58:59.306744   764 raft_consensus.cc:697] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Becoming Leader. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:58:59.307528   764 consensus_queue.cc:237] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.310607   714 raft_consensus.cc:2468] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 1.
I20260709 12:58:59.316126   459 catalog_manager.cc:5697] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e31c4f4178e4b799d0276d453376995 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "5e31c4f4178e4b799d0276d453376995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.327286   790 consensus_queue.cc:1048] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.338204   766 consensus_queue.cc:1048] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.363730   764 consensus_queue.cc:1048] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.378170   804 consensus_queue.cc:1048] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.405498   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059"
dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
mode: GRACEFUL
new_leader_uuid: "3bdacb3268db46cba845b6218fb33877"
 from {username='slave'} at 127.0.0.1:36400
I20260709 12:58:59.406242   714 raft_consensus.cc:606] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Received request to transfer leadership to TS 3bdacb3268db46cba845b6218fb33877
I20260709 12:58:59.412395   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "830161be8ab340f095080bafe7c11544"
dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
mode: GRACEFUL
new_leader_uuid: "3bdacb3268db46cba845b6218fb33877"
 from {username='slave'} at 127.0.0.1:36400
I20260709 12:58:59.413128   714 raft_consensus.cc:606] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Received request to transfer leadership to TS 3bdacb3268db46cba845b6218fb33877
I20260709 12:58:59.418288   714 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b76685879b204835ad5a1b543e151a61"
dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
mode: GRACEFUL
new_leader_uuid: "3bdacb3268db46cba845b6218fb33877"
 from {username='slave'} at 127.0.0.1:36400
I20260709 12:58:59.418815   714 raft_consensus.cc:606] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Received request to transfer leadership to TS 3bdacb3268db46cba845b6218fb33877
I20260709 12:58:59.511660   804 consensus_queue.cc:1048] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.518723   765 raft_consensus.cc:493] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.519222   765 raft_consensus.cc:515] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.521332   765 leader_election.cc:290] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:59.523229   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1fa50662e24eff94fc63c2d5895884" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" is_pre_election: true
I20260709 12:58:59.524035   715 raft_consensus.cc:2393] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3bdacb3268db46cba845b6218fb33877 in current term 1: Already voted for candidate 5e31c4f4178e4b799d0276d453376995 in this term.
I20260709 12:58:59.524788   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1fa50662e24eff94fc63c2d5895884" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e31c4f4178e4b799d0276d453376995" is_pre_election: true
I20260709 12:58:59.526347   807 consensus_queue.cc:1048] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.526373   530 leader_election.cc:304] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877; no voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995
I20260709 12:58:59.528857   784 raft_consensus.cc:3060] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.537240   784 raft_consensus.cc:2749] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:59.578177   790 consensus_queue.cc:1048] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.586269   766 raft_consensus.cc:993] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0: : Instructing follower 3bdacb3268db46cba845b6218fb33877 to start an election
I20260709 12:58:59.586762   766 raft_consensus.cc:1081] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Signalling peer 3bdacb3268db46cba845b6218fb33877 to start an election
I20260709 12:58:59.588524   790 consensus_queue.cc:1048] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:59.591621   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b76685879b204835ad5a1b543e151a61"
dest_uuid: "3bdacb3268db46cba845b6218fb33877"
 from {username='slave'} at 127.0.0.1:44528
I20260709 12:58:59.592279   565 raft_consensus.cc:493] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:59.592665   565 raft_consensus.cc:3060] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.598322   807 raft_consensus.cc:493] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:58:59.598951   807 raft_consensus.cc:515] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.600469   565 raft_consensus.cc:515] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.604142   641 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76685879b204835ad5a1b543e151a61" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:58:59.605376   565 leader_election.cc:290] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 2 election: Requested vote from peers 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:59.606287   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76685879b204835ad5a1b543e151a61" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:58:59.607288   715 raft_consensus.cc:3055] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:59.607610   715 raft_consensus.cc:740] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Running, Role: LEADER
I20260709 12:58:59.607729   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d91fc29f3c5b4ce391f618db929bd3d6" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" is_pre_election: true
I20260709 12:58:59.608495   715 consensus_queue.cc:260] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.609611   807 leader_election.cc:290] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:59.610428   715 raft_consensus.cc:3060] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.610498   564 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d91fc29f3c5b4ce391f618db929bd3d6" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bdacb3268db46cba845b6218fb33877" is_pre_election: true
I20260709 12:58:59.611301   564 raft_consensus.cc:2393] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e31c4f4178e4b799d0276d453376995 in current term 1: Already voted for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in this term.
I20260709 12:58:59.612886   607 leader_election.cc:304] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e31c4f4178e4b799d0276d453376995; no voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0
I20260709 12:58:59.620064   715 raft_consensus.cc:2468] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bdacb3268db46cba845b6218fb33877 in term 2.
I20260709 12:58:59.621234   531 leader_election.cc:304] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 5e31c4f4178e4b799d0276d453376995
I20260709 12:58:59.622375   784 raft_consensus.cc:2804] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:59.622850   784 raft_consensus.cc:697] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Becoming Leader. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Running, Role: LEADER
I20260709 12:58:59.623091   807 raft_consensus.cc:3060] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:58:59.624717   784 consensus_queue.cc:237] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.631342   790 consensus_queue.cc:1048] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.632351   807 raft_consensus.cc:2749] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:58:59.634629   462 catalog_manager.cc:5697] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 reported cstate change: term changed from 1 to 2, leader changed from 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195) to 3bdacb3268db46cba845b6218fb33877 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3bdacb3268db46cba845b6218fb33877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.645212   766 consensus_queue.cc:1048] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.659857   807 consensus_queue.cc:1048] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.673516   808 consensus_queue.cc:1048] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.679697   764 consensus_queue.cc:1048] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.691129   766 consensus_queue.cc:1048] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:59.712702   766 raft_consensus.cc:993] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0: : Instructing follower 3bdacb3268db46cba845b6218fb33877 to start an election
I20260709 12:58:59.713160   766 raft_consensus.cc:1081] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Signalling peer 3bdacb3268db46cba845b6218fb33877 to start an election
I20260709 12:58:59.717705   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "830161be8ab340f095080bafe7c11544"
dest_uuid: "3bdacb3268db46cba845b6218fb33877"
 from {username='slave'} at 127.0.0.1:44528
I20260709 12:58:59.719228   565 raft_consensus.cc:493] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:59.719731   565 raft_consensus.cc:3060] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.732439   565 raft_consensus.cc:515] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.734870   807 consensus_queue.cc:1048] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:58:59.739219   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830161be8ab340f095080bafe7c11544" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:58:59.739944   640 raft_consensus.cc:3060] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.751856   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830161be8ab340f095080bafe7c11544" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:58:59.752753   714 raft_consensus.cc:3055] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:59.753185   714 raft_consensus.cc:740] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Running, Role: LEADER
I20260709 12:58:59.754321   714 consensus_queue.cc:260] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.755836   714 raft_consensus.cc:3060] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.765558   565 leader_election.cc:290] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 2 election: Requested vote from peers 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:59.767957   640 raft_consensus.cc:2468] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bdacb3268db46cba845b6218fb33877 in term 2.
I20260709 12:58:59.768921   714 raft_consensus.cc:2468] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bdacb3268db46cba845b6218fb33877 in term 2.
I20260709 12:58:59.769625   530 leader_election.cc:304] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:58:59.771454   765 raft_consensus.cc:2804] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:59.771986   765 raft_consensus.cc:697] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Becoming Leader. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Running, Role: LEADER
I20260709 12:58:59.773304   765 consensus_queue.cc:237] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.787405   804 consensus_queue.cc:1048] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:58:59.792953   462 catalog_manager.cc:5697] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 reported cstate change: term changed from 1 to 2, leader changed from 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195) to 3bdacb3268db46cba845b6218fb33877 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3bdacb3268db46cba845b6218fb33877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:58:59.916010   812 raft_consensus.cc:993] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0: : Instructing follower 3bdacb3268db46cba845b6218fb33877 to start an election
I20260709 12:58:59.916543   789 raft_consensus.cc:1081] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Signalling peer 3bdacb3268db46cba845b6218fb33877 to start an election
I20260709 12:58:59.918236   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059"
dest_uuid: "3bdacb3268db46cba845b6218fb33877"
 from {username='slave'} at 127.0.0.1:44528
I20260709 12:58:59.918844   565 raft_consensus.cc:493] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:58:59.919255   565 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.926401   565 raft_consensus.cc:515] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.928468   565 leader_election.cc:290] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 2 election: Requested vote from peers 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:58:59.929546   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:58:59.929409   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:58:59.930202   715 raft_consensus.cc:3055] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:58:59.930287   640 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.930637   715 raft_consensus.cc:740] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Running, Role: LEADER
I20260709 12:58:59.931497   715 consensus_queue.cc:260] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.932708   715 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:58:59.937865   640 raft_consensus.cc:2468] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bdacb3268db46cba845b6218fb33877 in term 2.
I20260709 12:58:59.939152   530 leader_election.cc:304] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:58:59.939821   715 raft_consensus.cc:2468] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bdacb3268db46cba845b6218fb33877 in term 2.
I20260709 12:58:59.940032   765 raft_consensus.cc:2804] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:58:59.940577   765 raft_consensus.cc:697] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Becoming Leader. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Running, Role: LEADER
I20260709 12:58:59.941457   765 consensus_queue.cc:237] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:58:59.948890   462 catalog_manager.cc:5697] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 reported cstate change: term changed from 1 to 2, leader changed from 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195) to 3bdacb3268db46cba845b6218fb33877 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3bdacb3268db46cba845b6218fb33877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:00.202224   715 raft_consensus.cc:1275] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Refusing update from remote peer 3bdacb3268db46cba845b6218fb33877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.203537   765 consensus_queue.cc:1048] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:00.212324   640 raft_consensus.cc:3060] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:00.220504   640 raft_consensus.cc:1275] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Refusing update from remote peer 3bdacb3268db46cba845b6218fb33877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.222563   842 consensus_queue.cc:1048] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:00.295113   715 raft_consensus.cc:1275] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Refusing update from remote peer 3bdacb3268db46cba845b6218fb33877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.296458   842 consensus_queue.cc:1048] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:00.304039   640 raft_consensus.cc:1275] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Refusing update from remote peer 3bdacb3268db46cba845b6218fb33877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.305750   786 consensus_queue.cc:1048] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:00.394270   715 raft_consensus.cc:1275] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Refusing update from remote peer 3bdacb3268db46cba845b6218fb33877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.395746   786 consensus_queue.cc:1048] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:00.404614   640 raft_consensus.cc:1275] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Refusing update from remote peer 3bdacb3268db46cba845b6218fb33877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:00.406294   765 consensus_queue.cc:1048] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:02.463265   431 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 12:59:02.464146   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:02.486449   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831"
dest_uuid: "5e31c4f4178e4b799d0276d453376995"
mode: GRACEFUL
new_leader_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
 from {username='slave'} at 127.0.0.1:53720
I20260709 12:59:02.487154   640 raft_consensus.cc:606] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Received request to transfer leadership to TS 5d26397b4ee543e8b8920dc10bbc54c0
I20260709 12:59:02.498070   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059"
dest_uuid: "3bdacb3268db46cba845b6218fb33877"
mode: GRACEFUL
new_leader_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
 from {username='slave'} at 127.0.0.1:44544
I20260709 12:59:02.498600   565 raft_consensus.cc:606] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Received request to transfer leadership to TS 5d26397b4ee543e8b8920dc10bbc54c0
I20260709 12:59:02.499874   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:59:02.500414   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:02.502492   804 raft_consensus.cc:993] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995: : Instructing follower 5d26397b4ee543e8b8920dc10bbc54c0 to start an election
I20260709 12:59:02.502895   849 raft_consensus.cc:1081] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Signalling peer 5d26397b4ee543e8b8920dc10bbc54c0 to start an election
I20260709 12:59:02.504381   715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831"
dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
 from {username='slave'} at 127.0.0.1:36388
I20260709 12:59:02.504875   715 raft_consensus.cc:493] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:02.505168   715 raft_consensus.cc:3060] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:02.509603   715 raft_consensus.cc:515] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:02.511345   715 leader_election.cc:290] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 2 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:59:02.512285   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:59:02.512516   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:59:02.512898   565 raft_consensus.cc:3060] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:02.513090   640 raft_consensus.cc:3055] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:02.513473   640 raft_consensus.cc:740] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:59:02.514110   640 consensus_queue.cc:260] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:02.515002   640 raft_consensus.cc:3060] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:02.518086   565 raft_consensus.cc:2468] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 2.
I20260709 12:59:02.519268   681 leader_election.cc:304] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 
I20260709 12:59:02.519500   640 raft_consensus.cc:2468] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 2.
I20260709 12:59:02.520059   789 raft_consensus.cc:2804] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:02.521665   789 raft_consensus.cc:697] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 LEADER]: Becoming Leader. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Running, Role: LEADER
I20260709 12:59:02.522620   789 consensus_queue.cc:237] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:02.532029   459 catalog_manager.cc:5697] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 reported cstate change: term changed from 1 to 2, leader changed from 5e31c4f4178e4b799d0276d453376995 (127.0.107.194) to 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:02.895481   786 raft_consensus.cc:993] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877: : Instructing follower 5d26397b4ee543e8b8920dc10bbc54c0 to start an election
I20260709 12:59:02.895890   856 raft_consensus.cc:1081] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Signalling peer 5d26397b4ee543e8b8920dc10bbc54c0 to start an election
I20260709 12:59:02.897655   715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059"
dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
 from {username='slave'} at 127.0.0.1:36394
I20260709 12:59:02.898176   715 raft_consensus.cc:493] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:02.898494   715 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:02.902866   715 raft_consensus.cc:515] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:02.904697   715 leader_election.cc:290] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 3 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:59:02.905871   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:59:02.905958   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:59:02.906404   640 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:02.906549   565 raft_consensus.cc:3055] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:02.906926   565 raft_consensus.cc:740] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Running, Role: LEADER
I20260709 12:59:02.907691   565 consensus_queue.cc:260] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:02.908608   565 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:02.911528   640 raft_consensus.cc:2468] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 3.
I20260709 12:59:02.912660   679 leader_election.cc:304] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:59:02.913520   789 raft_consensus.cc:2804] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:02.913872   565 raft_consensus.cc:2468] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 3.
I20260709 12:59:02.913929   789 raft_consensus.cc:697] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 LEADER]: Becoming Leader. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Running, Role: LEADER
I20260709 12:59:02.914939   789 consensus_queue.cc:237] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:02.923853   459 catalog_manager.cc:5697] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 reported cstate change: term changed from 2 to 3, leader changed from 3bdacb3268db46cba845b6218fb33877 (127.0.107.193) to 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:02.957298   640 raft_consensus.cc:1275] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Refusing update from remote peer 5d26397b4ee543e8b8920dc10bbc54c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:02.958663   789 consensus_queue.cc:1048] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:02.968777   565 raft_consensus.cc:1275] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Refusing update from remote peer 5d26397b4ee543e8b8920dc10bbc54c0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:02.970178   850 consensus_queue.cc:1048] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:03.355741   565 raft_consensus.cc:1275] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 3 FOLLOWER]: Refusing update from remote peer 5d26397b4ee543e8b8920dc10bbc54c0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:03.356997   860 consensus_queue.cc:1048] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:03.365509   640 raft_consensus.cc:1275] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 3 FOLLOWER]: Refusing update from remote peer 5d26397b4ee543e8b8920dc10bbc54c0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:03.367012   860 consensus_queue.cc:1048] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:03.501262   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:03.519995   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b053f7eed9c24e07a7aa1dd5f8f4ca4c"
dest_uuid: "3bdacb3268db46cba845b6218fb33877"
mode: GRACEFUL
new_leader_uuid: "5e31c4f4178e4b799d0276d453376995"
 from {username='slave'} at 127.0.0.1:58846
I20260709 12:59:03.520574   565 raft_consensus.cc:606] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 1 LEADER]: Received request to transfer leadership to TS 5e31c4f4178e4b799d0276d453376995
I20260709 12:59:03.526211   715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059"
dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
mode: GRACEFUL
new_leader_uuid: "5e31c4f4178e4b799d0276d453376995"
 from {username='slave'} at 127.0.0.1:36400
I20260709 12:59:03.526743   715 raft_consensus.cc:606] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 LEADER]: Received request to transfer leadership to TS 5e31c4f4178e4b799d0276d453376995
I20260709 12:59:03.531956   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "830161be8ab340f095080bafe7c11544"
dest_uuid: "3bdacb3268db46cba845b6218fb33877"
mode: GRACEFUL
new_leader_uuid: "5e31c4f4178e4b799d0276d453376995"
 from {username='slave'} at 127.0.0.1:58846
I20260709 12:59:03.532503   565 raft_consensus.cc:606] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Received request to transfer leadership to TS 5e31c4f4178e4b799d0276d453376995
I20260709 12:59:03.537462   565 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b76685879b204835ad5a1b543e151a61"
dest_uuid: "3bdacb3268db46cba845b6218fb33877"
mode: GRACEFUL
new_leader_uuid: "5e31c4f4178e4b799d0276d453376995"
 from {username='slave'} at 127.0.0.1:58846
I20260709 12:59:03.538113   565 raft_consensus.cc:606] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Received request to transfer leadership to TS 5e31c4f4178e4b799d0276d453376995
I20260709 12:59:03.548434   715 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831"
dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
mode: GRACEFUL
new_leader_uuid: "5e31c4f4178e4b799d0276d453376995"
 from {username='slave'} at 127.0.0.1:36400
I20260709 12:59:03.548895   715 raft_consensus.cc:606] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 LEADER]: Received request to transfer leadership to TS 5e31c4f4178e4b799d0276d453376995
I20260709 12:59:03.565646   869 raft_consensus.cc:993] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877: : Instructing follower 5e31c4f4178e4b799d0276d453376995 to start an election
I20260709 12:59:03.566066   856 raft_consensus.cc:1081] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 1 LEADER]: Signalling peer 5e31c4f4178e4b799d0276d453376995 to start an election
I20260709 12:59:03.567488   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b053f7eed9c24e07a7aa1dd5f8f4ca4c"
dest_uuid: "5e31c4f4178e4b799d0276d453376995"
 from {username='slave'} at 127.0.0.1:53706
I20260709 12:59:03.567977   640 raft_consensus.cc:493] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:03.568288   640 raft_consensus.cc:3060] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:03.574359   640 raft_consensus.cc:515] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.576195   640 leader_election.cc:290] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 2 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:59:03.577306   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b053f7eed9c24e07a7aa1dd5f8f4ca4c" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:59:03.576943   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b053f7eed9c24e07a7aa1dd5f8f4ca4c" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:59:03.577877   715 raft_consensus.cc:3060] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:03.578058   565 raft_consensus.cc:3055] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:03.578387   565 raft_consensus.cc:740] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Running, Role: LEADER
I20260709 12:59:03.578999   565 consensus_queue.cc:260] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.579862   565 raft_consensus.cc:3060] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:03.582171   715 raft_consensus.cc:2468] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 2.
I20260709 12:59:03.583307   606 leader_election.cc:304] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:59:03.584187   849 raft_consensus.cc:2804] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:03.584610   849 raft_consensus.cc:697] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 2 LEADER]: Becoming Leader. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:59:03.585295   565 raft_consensus.cc:2468] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 2.
I20260709 12:59:03.585328   849 consensus_queue.cc:237] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.593263   461 catalog_manager.cc:5697] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 reported cstate change: term changed from 1 to 2, leader changed from 3bdacb3268db46cba845b6218fb33877 (127.0.107.193) to 5e31c4f4178e4b799d0276d453376995 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "5e31c4f4178e4b799d0276d453376995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:03.714619   869 raft_consensus.cc:993] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877: : Instructing follower 5e31c4f4178e4b799d0276d453376995 to start an election
I20260709 12:59:03.715049   869 raft_consensus.cc:1081] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Signalling peer 5e31c4f4178e4b799d0276d453376995 to start an election
I20260709 12:59:03.716811   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b76685879b204835ad5a1b543e151a61"
dest_uuid: "5e31c4f4178e4b799d0276d453376995"
 from {username='slave'} at 127.0.0.1:53706
I20260709 12:59:03.717342   640 raft_consensus.cc:493] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:03.717711   640 raft_consensus.cc:3060] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.721673   640 raft_consensus.cc:515] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.723165   640 leader_election.cc:290] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 3 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:59:03.723980   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76685879b204835ad5a1b543e151a61" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:59:03.724205   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b76685879b204835ad5a1b543e151a61" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:59:03.724496   565 raft_consensus.cc:3055] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:03.724715   715 raft_consensus.cc:3060] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.724829   565 raft_consensus.cc:740] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Running, Role: LEADER
I20260709 12:59:03.725657   565 consensus_queue.cc:260] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.726558   565 raft_consensus.cc:3060] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.729066   715 raft_consensus.cc:2468] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 3.
I20260709 12:59:03.730202   606 leader_election.cc:304] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:59:03.730995   565 raft_consensus.cc:2468] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 3.
I20260709 12:59:03.731051   849 raft_consensus.cc:2804] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:03.731782   849 raft_consensus.cc:697] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 3 LEADER]: Becoming Leader. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:59:03.732570   849 consensus_queue.cc:237] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.739428   461 catalog_manager.cc:5697] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 reported cstate change: term changed from 2 to 3, leader changed from 3bdacb3268db46cba845b6218fb33877 (127.0.107.193) to 5e31c4f4178e4b799d0276d453376995 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "5e31c4f4178e4b799d0276d453376995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:03.753546   856 raft_consensus.cc:993] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877: : Instructing follower 5e31c4f4178e4b799d0276d453376995 to start an election
I20260709 12:59:03.753978   869 raft_consensus.cc:1081] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Signalling peer 5e31c4f4178e4b799d0276d453376995 to start an election
I20260709 12:59:03.755405   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "830161be8ab340f095080bafe7c11544"
dest_uuid: "5e31c4f4178e4b799d0276d453376995"
 from {username='slave'} at 127.0.0.1:53706
I20260709 12:59:03.755846   640 raft_consensus.cc:493] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:03.756074   640 raft_consensus.cc:3060] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.759935   640 raft_consensus.cc:515] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.761243   640 leader_election.cc:290] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 3 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:59:03.762171   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830161be8ab340f095080bafe7c11544" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:59:03.762291   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830161be8ab340f095080bafe7c11544" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:59:03.762670   565 raft_consensus.cc:3055] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:03.762796   715 raft_consensus.cc:3060] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.762957   565 raft_consensus.cc:740] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Running, Role: LEADER
I20260709 12:59:03.763681   565 consensus_queue.cc:260] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.764477   565 raft_consensus.cc:3060] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.766870   715 raft_consensus.cc:2468] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 3.
I20260709 12:59:03.767792   606 leader_election.cc:304] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5d26397b4ee543e8b8920dc10bbc54c0, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:59:03.768481   849 raft_consensus.cc:2804] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:03.768520   565 raft_consensus.cc:2468] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 3.
I20260709 12:59:03.769081   849 raft_consensus.cc:697] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 3 LEADER]: Becoming Leader. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:59:03.769881   849 consensus_queue.cc:237] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.777140   461 catalog_manager.cc:5697] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 reported cstate change: term changed from 2 to 3, leader changed from 3bdacb3268db46cba845b6218fb33877 (127.0.107.193) to 5e31c4f4178e4b799d0276d453376995 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "5e31c4f4178e4b799d0276d453376995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:03.878485   876 raft_consensus.cc:993] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0: : Instructing follower 5e31c4f4178e4b799d0276d453376995 to start an election
I20260709 12:59:03.878935   860 raft_consensus.cc:1081] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 LEADER]: Signalling peer 5e31c4f4178e4b799d0276d453376995 to start an election
I20260709 12:59:03.880275   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831"
dest_uuid: "5e31c4f4178e4b799d0276d453376995"
 from {username='slave'} at 127.0.0.1:53708
I20260709 12:59:03.880669   640 raft_consensus.cc:493] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:03.880892   640 raft_consensus.cc:3060] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.884795   640 raft_consensus.cc:515] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.886276   640 leader_election.cc:290] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 3 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:59:03.887050   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:59:03.887295   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:59:03.887621   565 raft_consensus.cc:3060] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.887827   715 raft_consensus.cc:3055] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:03.888150   715 raft_consensus.cc:740] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Running, Role: LEADER
I20260709 12:59:03.888840   715 consensus_queue.cc:260] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.889936   715 raft_consensus.cc:3060] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:03.893805   565 raft_consensus.cc:2468] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 3.
I20260709 12:59:03.894919   607 leader_election.cc:304] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:59:03.895555   715 raft_consensus.cc:2468] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 3.
I20260709 12:59:03.895668   849 raft_consensus.cc:2804] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:03.896183   849 raft_consensus.cc:697] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 3 LEADER]: Becoming Leader. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:59:03.896976   849 consensus_queue.cc:237] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.902542   461 catalog_manager.cc:5697] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 reported cstate change: term changed from 2 to 3, leader changed from 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195) to 5e31c4f4178e4b799d0276d453376995 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "5e31c4f4178e4b799d0276d453376995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:03.916154   877 raft_consensus.cc:993] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0: : Instructing follower 5e31c4f4178e4b799d0276d453376995 to start an election
I20260709 12:59:03.916643   860 raft_consensus.cc:1081] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 LEADER]: Signalling peer 5e31c4f4178e4b799d0276d453376995 to start an election
I20260709 12:59:03.918102   640 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059"
dest_uuid: "5e31c4f4178e4b799d0276d453376995"
 from {username='slave'} at 127.0.0.1:53708
I20260709 12:59:03.918579   640 raft_consensus.cc:493] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:03.918929   640 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:03.922838   640 raft_consensus.cc:515] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.924347   640 leader_election.cc:290] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 4 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:59:03.925144   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:59:03.925370   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059" candidate_uuid: "5e31c4f4178e4b799d0276d453376995" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:59:03.925752   565 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:03.925926   715 raft_consensus.cc:3055] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:03.926260   715 raft_consensus.cc:740] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Running, Role: LEADER
I20260709 12:59:03.926823   715 consensus_queue.cc:260] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.927610   715 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:03.930029   565 raft_consensus.cc:2468] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 4.
I20260709 12:59:03.930917   607 leader_election.cc:304] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5e31c4f4178e4b799d0276d453376995; no voters: 
I20260709 12:59:03.931674   849 raft_consensus.cc:2804] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:03.931768   715 raft_consensus.cc:2468] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e31c4f4178e4b799d0276d453376995 in term 4.
I20260709 12:59:03.932185   849 raft_consensus.cc:697] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 4 LEADER]: Becoming Leader. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:59:03.932964   849 consensus_queue.cc:237] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:03.938585   461 catalog_manager.cc:5697] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 reported cstate change: term changed from 3 to 4, leader changed from 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195) to 5e31c4f4178e4b799d0276d453376995 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "5e31c4f4178e4b799d0276d453376995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:04.121084   565 raft_consensus.cc:1275] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Refusing update from remote peer 5e31c4f4178e4b799d0276d453376995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:04.122282   849 consensus_queue.cc:1048] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:04.130896   715 raft_consensus.cc:1275] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Refusing update from remote peer 5e31c4f4178e4b799d0276d453376995: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:04.132813   849 consensus_queue.cc:1048] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:04.146114   565 raft_consensus.cc:1275] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 3 FOLLOWER]: Refusing update from remote peer 5e31c4f4178e4b799d0276d453376995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.147389   849 consensus_queue.cc:1048] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:04.158784   715 raft_consensus.cc:1275] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Refusing update from remote peer 5e31c4f4178e4b799d0276d453376995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.160200   880 consensus_queue.cc:1048] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:04.197813   565 raft_consensus.cc:1275] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 3 FOLLOWER]: Refusing update from remote peer 5e31c4f4178e4b799d0276d453376995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.199117   880 consensus_queue.cc:1048] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:04.207868   715 raft_consensus.cc:1275] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Refusing update from remote peer 5e31c4f4178e4b799d0276d453376995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.209735   887 consensus_queue.cc:1048] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:04.329027   565 raft_consensus.cc:1275] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 4 FOLLOWER]: Refusing update from remote peer 5e31c4f4178e4b799d0276d453376995: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:04.330214   887 consensus_queue.cc:1048] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:04.339918   715 raft_consensus.cc:1275] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 4 FOLLOWER]: Refusing update from remote peer 5e31c4f4178e4b799d0276d453376995: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:04.341650   880 consensus_queue.cc:1048] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:04.350075   715 raft_consensus.cc:1275] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Refusing update from remote peer 5e31c4f4178e4b799d0276d453376995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.351271   880 consensus_queue.cc:1048] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:04.359179   565 raft_consensus.cc:1275] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 3 FOLLOWER]: Refusing update from remote peer 5e31c4f4178e4b799d0276d453376995: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:04.361166   864 consensus_queue.cc:1048] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:06.556247   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:06.564865   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831"
dest_uuid: "5e31c4f4178e4b799d0276d453376995"
mode: GRACEFUL
new_leader_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
 from {username='slave'} at 127.0.0.1:53720
I20260709 12:59:06.565572   640 raft_consensus.cc:606] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 3 LEADER]: Received request to transfer leadership to TS 5d26397b4ee543e8b8920dc10bbc54c0
I20260709 12:59:06.568384   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56505abb1d1d4125aa901433f539577d"
dest_uuid: "5e31c4f4178e4b799d0276d453376995"
mode: GRACEFUL
new_leader_uuid: "3bdacb3268db46cba845b6218fb33877"
 from {username='slave'} at 127.0.0.1:53720
I20260709 12:59:06.568795   640 raft_consensus.cc:606] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Received request to transfer leadership to TS 3bdacb3268db46cba845b6218fb33877
I20260709 12:59:06.571233   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5c1fa50662e24eff94fc63c2d5895884"
dest_uuid: "5e31c4f4178e4b799d0276d453376995"
mode: GRACEFUL
new_leader_uuid: "3bdacb3268db46cba845b6218fb33877"
 from {username='slave'} at 127.0.0.1:53720
I20260709 12:59:06.571625   640 raft_consensus.cc:606] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Received request to transfer leadership to TS 3bdacb3268db46cba845b6218fb33877
I20260709 12:59:06.574424   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "830161be8ab340f095080bafe7c11544"
dest_uuid: "5e31c4f4178e4b799d0276d453376995"
mode: GRACEFUL
new_leader_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
 from {username='slave'} at 127.0.0.1:53720
I20260709 12:59:06.574837   640 raft_consensus.cc:606] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 3 LEADER]: Received request to transfer leadership to TS 5d26397b4ee543e8b8920dc10bbc54c0
I20260709 12:59:06.577205   640 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059"
dest_uuid: "5e31c4f4178e4b799d0276d453376995"
mode: GRACEFUL
new_leader_uuid: "3bdacb3268db46cba845b6218fb33877"
 from {username='slave'} at 127.0.0.1:53720
I20260709 12:59:06.577656   640 raft_consensus.cc:606] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 4 LEADER]: Received request to transfer leadership to TS 3bdacb3268db46cba845b6218fb33877
I20260709 12:59:06.578735   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:59:06.579269   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:06.718698   892 raft_consensus.cc:993] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995: : Instructing follower 3bdacb3268db46cba845b6218fb33877 to start an election
I20260709 12:59:06.719229   880 raft_consensus.cc:1081] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Signalling peer 3bdacb3268db46cba845b6218fb33877 to start an election
I20260709 12:59:06.720842   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56505abb1d1d4125aa901433f539577d"
dest_uuid: "3bdacb3268db46cba845b6218fb33877"
 from {username='slave'} at 127.0.0.1:44520
I20260709 12:59:06.721354   565 raft_consensus.cc:493] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:06.721724   565 raft_consensus.cc:3060] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:06.726603   565 raft_consensus.cc:515] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:06.728267   565 leader_election.cc:290] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 2 election: Requested vote from peers 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:59:06.729708   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56505abb1d1d4125aa901433f539577d" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:59:06.729842   714 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56505abb1d1d4125aa901433f539577d" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:59:06.730419   640 raft_consensus.cc:3055] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:06.730468   714 raft_consensus.cc:3060] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:06.730825   640 raft_consensus.cc:740] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:59:06.731650   640 consensus_queue.cc:260] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:06.732592   640 raft_consensus.cc:3060] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:06.736936   714 raft_consensus.cc:2468] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bdacb3268db46cba845b6218fb33877 in term 2.
I20260709 12:59:06.737782   640 raft_consensus.cc:2468] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bdacb3268db46cba845b6218fb33877 in term 2.
I20260709 12:59:06.738485   531 leader_election.cc:304] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 
I20260709 12:59:06.739521   892 raft_consensus.cc:993] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995: : Instructing follower 5d26397b4ee543e8b8920dc10bbc54c0 to start an election
I20260709 12:59:06.740131   880 raft_consensus.cc:1081] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 3 LEADER]: Signalling peer 5d26397b4ee543e8b8920dc10bbc54c0 to start an election
I20260709 12:59:06.740365   893 raft_consensus.cc:2804] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:06.741559   893 raft_consensus.cc:697] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Becoming Leader. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Running, Role: LEADER
I20260709 12:59:06.742398   715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "830161be8ab340f095080bafe7c11544"
dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
 from {username='slave'} at 127.0.0.1:36388
I20260709 12:59:06.742883   715 raft_consensus.cc:493] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:06.742617   893 consensus_queue.cc:237] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:06.743180   715 raft_consensus.cc:3060] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:06.749547   715 raft_consensus.cc:515] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:06.751204   715 leader_election.cc:290] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 4 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:59:06.750979   459 catalog_manager.cc:5697] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 reported cstate change: term changed from 1 to 2, leader changed from 5e31c4f4178e4b799d0276d453376995 (127.0.107.194) to 3bdacb3268db46cba845b6218fb33877 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3bdacb3268db46cba845b6218fb33877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:06.752169   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830161be8ab340f095080bafe7c11544" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:59:06.752341   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "830161be8ab340f095080bafe7c11544" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:59:06.752765   565 raft_consensus.cc:3060] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:06.752931   640 raft_consensus.cc:3055] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:06.753357   640 raft_consensus.cc:740] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:59:06.754194   640 consensus_queue.cc:260] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:06.755630   640 raft_consensus.cc:3060] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:06.759958   565 raft_consensus.cc:2468] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 4.
I20260709 12:59:06.762012   681 leader_election.cc:304] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 
I20260709 12:59:06.762979   640 raft_consensus.cc:2468] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 4.
I20260709 12:59:06.763217   860 raft_consensus.cc:2804] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:06.767098   860 raft_consensus.cc:697] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 4 LEADER]: Becoming Leader. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Running, Role: LEADER
I20260709 12:59:06.768064   860 consensus_queue.cc:237] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:06.772753   892 raft_consensus.cc:993] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995: : Instructing follower 3bdacb3268db46cba845b6218fb33877 to start an election
I20260709 12:59:06.773293   880 raft_consensus.cc:1081] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Signalling peer 3bdacb3268db46cba845b6218fb33877 to start an election
I20260709 12:59:06.775228   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5c1fa50662e24eff94fc63c2d5895884"
dest_uuid: "3bdacb3268db46cba845b6218fb33877"
 from {username='slave'} at 127.0.0.1:44520
I20260709 12:59:06.775782   565 raft_consensus.cc:493] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:06.776067   565 raft_consensus.cc:3060] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:06.778738   459 catalog_manager.cc:5697] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 reported cstate change: term changed from 3 to 4, leader changed from 5e31c4f4178e4b799d0276d453376995 (127.0.107.194) to 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:06.780632   565 raft_consensus.cc:515] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:06.782975   565 leader_election.cc:290] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 2 election: Requested vote from peers 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:59:06.783347   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1fa50662e24eff94fc63c2d5895884" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:59:06.783546   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5c1fa50662e24eff94fc63c2d5895884" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:59:06.783902   640 raft_consensus.cc:3055] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:06.783999   715 raft_consensus.cc:3060] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:06.784236   640 raft_consensus.cc:740] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:59:06.785032   640 consensus_queue.cc:260] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:06.786283   640 raft_consensus.cc:3060] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:06.790715   715 raft_consensus.cc:2468] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bdacb3268db46cba845b6218fb33877 in term 2.
I20260709 12:59:06.791934   531 leader_election.cc:304] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 
I20260709 12:59:06.792491   640 raft_consensus.cc:2468] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bdacb3268db46cba845b6218fb33877 in term 2.
I20260709 12:59:06.792819   893 raft_consensus.cc:2804] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:06.793292   893 raft_consensus.cc:697] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Becoming Leader. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Running, Role: LEADER
I20260709 12:59:06.794118   893 consensus_queue.cc:237] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:06.801642   459 catalog_manager.cc:5697] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 reported cstate change: term changed from 1 to 2, leader changed from 5e31c4f4178e4b799d0276d453376995 (127.0.107.194) to 3bdacb3268db46cba845b6218fb33877 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "3bdacb3268db46cba845b6218fb33877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:07.030463   892 raft_consensus.cc:993] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995: : Instructing follower 5d26397b4ee543e8b8920dc10bbc54c0 to start an election
I20260709 12:59:07.030936   880 raft_consensus.cc:1081] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 3 LEADER]: Signalling peer 5d26397b4ee543e8b8920dc10bbc54c0 to start an election
I20260709 12:59:07.032378   715 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831"
dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
 from {username='slave'} at 127.0.0.1:36388
I20260709 12:59:07.032783   715 raft_consensus.cc:493] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:07.033010   715 raft_consensus.cc:3060] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:07.036796   715 raft_consensus.cc:515] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:07.038364   715 leader_election.cc:290] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 4 election: Requested vote from peers 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807), 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357)
I20260709 12:59:07.039242   565 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3bdacb3268db46cba845b6218fb33877"
I20260709 12:59:07.039397   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "02c9b5f41d8c4aa987a40e3f070c7831" candidate_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:59:07.039829   565 raft_consensus.cc:3060] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:07.039937   640 raft_consensus.cc:3055] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:07.040297   640 raft_consensus.cc:740] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:59:07.040927   640 consensus_queue.cc:260] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:07.041778   640 raft_consensus.cc:3060] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:07.044514   565 raft_consensus.cc:2468] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 4.
I20260709 12:59:07.045670   640 raft_consensus.cc:2468] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5d26397b4ee543e8b8920dc10bbc54c0 in term 4.
I20260709 12:59:07.045689   681 leader_election.cc:304] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 
I20260709 12:59:07.046667   860 raft_consensus.cc:2804] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:07.047204   860 raft_consensus.cc:697] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 4 LEADER]: Becoming Leader. State: Replica: 5d26397b4ee543e8b8920dc10bbc54c0, State: Running, Role: LEADER
I20260709 12:59:07.047950   860 consensus_queue.cc:237] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:07.054912   459 catalog_manager.cc:5697] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 reported cstate change: term changed from 3 to 4, leader changed from 5e31c4f4178e4b799d0276d453376995 (127.0.107.194) to 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195). New cstate: current_term: 4 leader_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:07.088203   892 raft_consensus.cc:993] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995: : Instructing follower 3bdacb3268db46cba845b6218fb33877 to start an election
I20260709 12:59:07.088661   880 raft_consensus.cc:1081] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 4 LEADER]: Signalling peer 3bdacb3268db46cba845b6218fb33877 to start an election
I20260709 12:59:07.090404   565 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059"
dest_uuid: "3bdacb3268db46cba845b6218fb33877"
 from {username='slave'} at 127.0.0.1:44520
I20260709 12:59:07.090924   565 raft_consensus.cc:493] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:07.091259   565 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:59:07.095542   565 raft_consensus.cc:515] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:07.097252   565 leader_election.cc:290] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 5 election: Requested vote from peers 5e31c4f4178e4b799d0276d453376995 (127.0.107.194:34357), 5d26397b4ee543e8b8920dc10bbc54c0 (127.0.107.195:44803)
I20260709 12:59:07.098320   640 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "5e31c4f4178e4b799d0276d453376995"
I20260709 12:59:07.098320   715 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e62ff30fc2d84ec3aa02b4b15df20059" candidate_uuid: "3bdacb3268db46cba845b6218fb33877" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "5d26397b4ee543e8b8920dc10bbc54c0"
I20260709 12:59:07.099102   715 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:59:07.099102   640 raft_consensus.cc:3055] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:59:07.099534   640 raft_consensus.cc:740] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5e31c4f4178e4b799d0276d453376995, State: Running, Role: LEADER
I20260709 12:59:07.100275   640 consensus_queue.cc:260] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:07.101330   640 raft_consensus.cc:3060] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:59:07.105386   715 raft_consensus.cc:2468] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bdacb3268db46cba845b6218fb33877 in term 5.
I20260709 12:59:07.106741   531 leader_election.cc:304] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bdacb3268db46cba845b6218fb33877, 5d26397b4ee543e8b8920dc10bbc54c0; no voters: 
I20260709 12:59:07.107338   640 raft_consensus.cc:2468] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3bdacb3268db46cba845b6218fb33877 in term 5.
I20260709 12:59:07.107643   893 raft_consensus.cc:2804] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:59:07.108150   893 raft_consensus.cc:697] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 5 LEADER]: Becoming Leader. State: Replica: 3bdacb3268db46cba845b6218fb33877, State: Running, Role: LEADER
I20260709 12:59:07.109704   893 consensus_queue.cc:237] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } }
I20260709 12:59:07.116323   459 catalog_manager.cc:5697] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 reported cstate change: term changed from 4 to 5, leader changed from 5e31c4f4178e4b799d0276d453376995 (127.0.107.194) to 3bdacb3268db46cba845b6218fb33877 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "3bdacb3268db46cba845b6218fb33877" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:07.167126   715 raft_consensus.cc:1275] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Refusing update from remote peer 3bdacb3268db46cba845b6218fb33877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:07.168340   893 consensus_queue.cc:1048] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:07.179147   640 raft_consensus.cc:1275] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Refusing update from remote peer 3bdacb3268db46cba845b6218fb33877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:07.181788   894 consensus_queue.cc:1048] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:07.189667   641 raft_consensus.cc:1275] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Refusing update from remote peer 3bdacb3268db46cba845b6218fb33877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:07.191181   894 consensus_queue.cc:1048] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:07.200875   715 raft_consensus.cc:1275] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Refusing update from remote peer 3bdacb3268db46cba845b6218fb33877: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:07.202442   893 consensus_queue.cc:1048] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:07.244047   640 raft_consensus.cc:1275] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 4 FOLLOWER]: Refusing update from remote peer 5d26397b4ee543e8b8920dc10bbc54c0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:07.245404   860 consensus_queue.cc:1048] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:07.254887   565 raft_consensus.cc:1275] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 4 FOLLOWER]: Refusing update from remote peer 5d26397b4ee543e8b8920dc10bbc54c0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:07.256469   899 consensus_queue.cc:1048] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:07.445223   565 raft_consensus.cc:1275] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 4 FOLLOWER]: Refusing update from remote peer 5d26397b4ee543e8b8920dc10bbc54c0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:07.446357   899 consensus_queue.cc:1048] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bdacb3268db46cba845b6218fb33877" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:07.455490   640 raft_consensus.cc:1275] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 4 FOLLOWER]: Refusing update from remote peer 5d26397b4ee543e8b8920dc10bbc54c0: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:07.457093   860 consensus_queue.cc:1048] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:07.540490   640 raft_consensus.cc:1275] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 5 FOLLOWER]: Refusing update from remote peer 3bdacb3268db46cba845b6218fb33877: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:59:07.541823   893 consensus_queue.cc:1048] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e31c4f4178e4b799d0276d453376995" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 34357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:07.550498   715 raft_consensus.cc:1275] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 5 FOLLOWER]: Refusing update from remote peer 3bdacb3268db46cba845b6218fb33877: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:59:07.552362   894 consensus_queue.cc:1048] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d26397b4ee543e8b8920dc10bbc54c0" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:07.580174   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:07.586431   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 12:59:07.610296   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:07.611111   431 tablet_replica.cc:333] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877: stopping tablet replica
I20260709 12:59:07.612227   431 raft_consensus.cc:2243] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:59:07.613370   431 raft_consensus.cc:2272] T 5c1fa50662e24eff94fc63c2d5895884 P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.616518   431 tablet_replica.cc:333] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877: stopping tablet replica
I20260709 12:59:07.617019   431 raft_consensus.cc:2243] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.617393   431 raft_consensus.cc:2272] T 7c373e47ac8f4c409285379c51efb58c P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.619266   431 tablet_replica.cc:333] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877: stopping tablet replica
I20260709 12:59:07.619750   431 raft_consensus.cc:2243] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:59:07.620479   431 raft_consensus.cc:2272] T 56505abb1d1d4125aa901433f539577d P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.622820   431 tablet_replica.cc:333] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877: stopping tablet replica
I20260709 12:59:07.623458   431 raft_consensus.cc:2243] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.623911   431 raft_consensus.cc:2272] T 830161be8ab340f095080bafe7c11544 P 3bdacb3268db46cba845b6218fb33877 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.626001   431 tablet_replica.cc:333] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877: stopping tablet replica
I20260709 12:59:07.626542   431 raft_consensus.cc:2243] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.627013   431 raft_consensus.cc:2272] T b76685879b204835ad5a1b543e151a61 P 3bdacb3268db46cba845b6218fb33877 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.628821   431 tablet_replica.cc:333] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877: stopping tablet replica
I20260709 12:59:07.629335   431 raft_consensus.cc:2243] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.629846   431 raft_consensus.cc:2272] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 3bdacb3268db46cba845b6218fb33877 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.631783   431 tablet_replica.cc:333] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877: stopping tablet replica
I20260709 12:59:07.632302   431 raft_consensus.cc:2243] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.632736   431 raft_consensus.cc:2272] T d91fc29f3c5b4ce391f618db929bd3d6 P 3bdacb3268db46cba845b6218fb33877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.634471   431 tablet_replica.cc:333] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877: stopping tablet replica
I20260709 12:59:07.635010   431 raft_consensus.cc:2243] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.635442   431 raft_consensus.cc:2272] T 02c9b5f41d8c4aa987a40e3f070c7831 P 3bdacb3268db46cba845b6218fb33877 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.637722   431 tablet_replica.cc:333] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877: stopping tablet replica
I20260709 12:59:07.638216   431 raft_consensus.cc:2243] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:59:07.638976   431 raft_consensus.cc:2272] T e62ff30fc2d84ec3aa02b4b15df20059 P 3bdacb3268db46cba845b6218fb33877 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:07.741648   607 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46807: connect: Connection refused (error 111)
I20260709 12:59:07.743459   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 12:59:07.746284   607 consensus_peers.cc:597] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 -> Peer 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807): Couldn't send request to peer 3bdacb3268db46cba845b6218fb33877. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:07.762689   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 12:59:07.771883   681 consensus_peers.cc:597] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 -> Peer 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807): Couldn't send request to peer 3bdacb3268db46cba845b6218fb33877. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:07.792394   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:07.793175   431 tablet_replica.cc:333] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995: stopping tablet replica
I20260709 12:59:07.793776   431 raft_consensus.cc:2243] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:07.794500   431 raft_consensus.cc:2272] T 7c373e47ac8f4c409285379c51efb58c P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.796212   431 tablet_replica.cc:333] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995: stopping tablet replica
I20260709 12:59:07.796684   431 raft_consensus.cc:2243] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.797051   431 raft_consensus.cc:2272] T 5c1fa50662e24eff94fc63c2d5895884 P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.799868   431 tablet_replica.cc:333] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995: stopping tablet replica
I20260709 12:59:07.800347   431 raft_consensus.cc:2243] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.800698   431 raft_consensus.cc:2272] T 830161be8ab340f095080bafe7c11544 P 5e31c4f4178e4b799d0276d453376995 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.802706   431 tablet_replica.cc:333] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995: stopping tablet replica
I20260709 12:59:07.803169   431 raft_consensus.cc:2243] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.803512   431 raft_consensus.cc:2272] T 56505abb1d1d4125aa901433f539577d P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.805270   431 tablet_replica.cc:333] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995: stopping tablet replica
I20260709 12:59:07.805804   431 raft_consensus.cc:2243] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.806159   431 raft_consensus.cc:2272] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5e31c4f4178e4b799d0276d453376995 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.808010   431 tablet_replica.cc:333] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995: stopping tablet replica
I20260709 12:59:07.808460   431 raft_consensus.cc:2243] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.808794   431 raft_consensus.cc:2272] T e62ff30fc2d84ec3aa02b4b15df20059 P 5e31c4f4178e4b799d0276d453376995 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.810734   431 tablet_replica.cc:333] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995: stopping tablet replica
I20260709 12:59:07.811160   431 raft_consensus.cc:2243] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.811506   431 raft_consensus.cc:2272] T d91fc29f3c5b4ce391f618db929bd3d6 P 5e31c4f4178e4b799d0276d453376995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.813014   431 tablet_replica.cc:333] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995: stopping tablet replica
I20260709 12:59:07.813575   431 raft_consensus.cc:2243] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:59:07.814337   431 raft_consensus.cc:2272] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5e31c4f4178e4b799d0276d453376995 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.816407   431 tablet_replica.cc:333] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995: stopping tablet replica
I20260709 12:59:07.816983   431 raft_consensus.cc:2243] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:59:07.817976   431 raft_consensus.cc:2272] T b76685879b204835ad5a1b543e151a61 P 5e31c4f4178e4b799d0276d453376995 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:07.820092   681 consensus_peers.cc:597] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 -> Peer 3bdacb3268db46cba845b6218fb33877 (127.0.107.193:46807): Couldn't send request to peer 3bdacb3268db46cba845b6218fb33877. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:07.846339   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 12:59:07.862349   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 12:59:07.890282   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:07.890892   431 tablet_replica.cc:333] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0: stopping tablet replica
I20260709 12:59:07.891409   431 raft_consensus.cc:2243] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.891839   431 raft_consensus.cc:2272] T b053f7eed9c24e07a7aa1dd5f8f4ca4c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.893743   431 tablet_replica.cc:333] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0: stopping tablet replica
I20260709 12:59:07.894222   431 raft_consensus.cc:2243] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:07.894878   431 raft_consensus.cc:2272] T d91fc29f3c5b4ce391f618db929bd3d6 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.896366   431 tablet_replica.cc:333] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0: stopping tablet replica
I20260709 12:59:07.896893   431 raft_consensus.cc:2243] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.897327   431 raft_consensus.cc:2272] T b76685879b204835ad5a1b543e151a61 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.898928   431 tablet_replica.cc:333] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0: stopping tablet replica
I20260709 12:59:07.899363   431 raft_consensus.cc:2243] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.899685   431 raft_consensus.cc:2272] T 7c373e47ac8f4c409285379c51efb58c P 5d26397b4ee543e8b8920dc10bbc54c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.901094   431 tablet_replica.cc:333] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0: stopping tablet replica
I20260709 12:59:07.901589   431 raft_consensus.cc:2243] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.901919   431 raft_consensus.cc:2272] T 5c1fa50662e24eff94fc63c2d5895884 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.903884   431 tablet_replica.cc:333] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0: stopping tablet replica
I20260709 12:59:07.904321   431 raft_consensus.cc:2243] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.904666   431 raft_consensus.cc:2272] T 56505abb1d1d4125aa901433f539577d P 5d26397b4ee543e8b8920dc10bbc54c0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.906679   431 tablet_replica.cc:333] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0: stopping tablet replica
I20260709 12:59:07.907106   431 raft_consensus.cc:2243] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:59:07.907753   431 raft_consensus.cc:2272] T 830161be8ab340f095080bafe7c11544 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.909621   431 tablet_replica.cc:333] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0: stopping tablet replica
I20260709 12:59:07.910018   431 raft_consensus.cc:2243] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:07.910370   431 raft_consensus.cc:2272] T e62ff30fc2d84ec3aa02b4b15df20059 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.912307   431 tablet_replica.cc:333] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0: stopping tablet replica
I20260709 12:59:07.912698   431 raft_consensus.cc:2243] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:59:07.913334   431 raft_consensus.cc:2272] T 02c9b5f41d8c4aa987a40e3f070c7831 P 5d26397b4ee543e8b8920dc10bbc54c0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:07.939738   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 12:59:07.955456   431 master.cc:562] Master@127.0.107.254:44605 shutting down...
I20260709 12:59:08.101802   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:08.102380   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:08.102728   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9376f5f363e14159b00adddbd0d53fb5: stopping tablet replica
I20260709 12:59:08.123256   431 master.cc:584] Master@127.0.107.254:44605 shutdown complete.
3bdacb3268db46cba845b6218fb33877  4
5d26397b4ee543e8b8920dc10bbc54c0  1
5e31c4f4178e4b799d0276d453376995  4
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12711 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:59:08.166136   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:36925
I20260709 12:59:08.167352   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:08.173137   924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:08.173161   925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:08.174620   927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:08.175740   431 server_base.cc:1061] running on GCE node
I20260709 12:59:08.176667   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:08.176844   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:08.176959   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601948176947 us; error 0 us; skew 500 ppm
I20260709 12:59:08.177410   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:08.179947   431 webserver.cc:533] Webserver started at http://127.0.107.254:35749/ using document root <none> and password file <none>
I20260709 12:59:08.180397   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:08.180558   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:08.180771   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:08.181890   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/master-0-root/instance:
uuid: "0195911216294f3f843d11618de0e904"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-jk05"
I20260709 12:59:08.186308   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:59:08.189996   932 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.190774   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:59:08.191118   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/master-0-root
uuid: "0195911216294f3f843d11618de0e904"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-jk05"
I20260709 12:59:08.191418   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:08.213609   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:08.215267   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:08.252697   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:36925
I20260709 12:59:08.252787   983 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:36925 every 8 connection(s)
I20260709 12:59:08.256596   984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.265991   984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904: Bootstrap starting.
I20260709 12:59:08.270632   984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:08.274813   984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904: No bootstrap required, opened a new log
I20260709 12:59:08.276882   984 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0195911216294f3f843d11618de0e904" member_type: VOTER }
I20260709 12:59:08.277298   984 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:08.277608   984 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0195911216294f3f843d11618de0e904, State: Initialized, Role: FOLLOWER
I20260709 12:59:08.278193   984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0195911216294f3f843d11618de0e904" member_type: VOTER }
I20260709 12:59:08.278591   984 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:59:08.278774   984 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:59:08.278977   984 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:08.283843   984 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0195911216294f3f843d11618de0e904" member_type: VOTER }
I20260709 12:59:08.284407   984 leader_election.cc:304] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0195911216294f3f843d11618de0e904; no voters: 
I20260709 12:59:08.285635   984 leader_election.cc:290] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:59:08.286032   987 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:08.287350   987 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [term 1 LEADER]: Becoming Leader. State: Replica: 0195911216294f3f843d11618de0e904, State: Running, Role: LEADER
I20260709 12:59:08.288076   987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0195911216294f3f843d11618de0e904" member_type: VOTER }
I20260709 12:59:08.288563   984 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:59:08.295576   989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0195911216294f3f843d11618de0e904. Latest consensus state: current_term: 1 leader_uuid: "0195911216294f3f843d11618de0e904" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0195911216294f3f843d11618de0e904" member_type: VOTER } }
I20260709 12:59:08.295533   988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0195911216294f3f843d11618de0e904" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0195911216294f3f843d11618de0e904" member_type: VOTER } }
I20260709 12:59:08.296329   989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:08.296384   988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:08.298440   995 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:59:08.303361   995 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:59:08.306242   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:59:08.312086   995 catalog_manager.cc:1383] Generated new cluster ID: 1df89a4a160e4408a2a011d3b158ad85
I20260709 12:59:08.312402   995 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:59:08.325403   995 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:59:08.326669   995 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:59:08.346676   995 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904: Generated new TSK 0
I20260709 12:59:08.347306   995 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:59:08.373158   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:08.379310  1005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:08.380658  1006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:08.382977   431 server_base.cc:1061] running on GCE node
W20260709 12:59:08.383286  1008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:08.384155   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:08.384397   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:08.384544   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601948384532 us; error 0 us; skew 500 ppm
I20260709 12:59:08.385020   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:08.387321   431 webserver.cc:533] Webserver started at http://127.0.107.193:42623/ using document root <none> and password file <none>
I20260709 12:59:08.387801   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:08.387975   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:08.388199   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:08.389206   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-0-root/instance:
uuid: "a6ec7e01e34945bb848cd646a3a171f8"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-jk05"
I20260709 12:59:08.394078   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:59:08.397235  1013 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.398098   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:59:08.398356   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-0-root
uuid: "a6ec7e01e34945bb848cd646a3a171f8"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-jk05"
I20260709 12:59:08.398602   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:08.412580   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:08.413870   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:08.415252   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:08.417647   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:08.417837   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.418133   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:08.418322   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.459676   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:46129
I20260709 12:59:08.459769  1075 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:46129 every 8 connection(s)
I20260709 12:59:08.464661   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:08.471993  1080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:08.473863  1081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:08.474737   431 server_base.cc:1061] running on GCE node
W20260709 12:59:08.476402  1083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:08.477316   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:08.477818   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:08.478016   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601948477997 us; error 0 us; skew 500 ppm
I20260709 12:59:08.478575   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:08.481514   431 webserver.cc:533] Webserver started at http://127.0.107.194:37249/ using document root <none> and password file <none>
I20260709 12:59:08.482120   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:08.482316   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:08.482642   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:08.482705  1076 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36925
I20260709 12:59:08.483140  1076 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:08.483853  1076 heartbeater.cc:507] Master 127.0.107.254:36925 requested a full tablet report, sending...
I20260709 12:59:08.483909   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-1-root/instance:
uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-jk05"
I20260709 12:59:08.486028   949 ts_manager.cc:194] Registered new tserver with Master: a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129)
I20260709 12:59:08.487721   949 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46824
I20260709 12:59:08.489127   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:59:08.492542  1088 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.493489   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:59:08.493826   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-1-root
uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-jk05"
I20260709 12:59:08.494109   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:08.503245   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:08.504460   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:08.505998   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:08.508190   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:08.508422   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.508685   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:08.508899   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.553226   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:39335
I20260709 12:59:08.553325  1150 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:39335 every 8 connection(s)
I20260709 12:59:08.558348   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:08.566141  1154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:08.568665  1155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:08.570413  1157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:08.571311   431 server_base.cc:1061] running on GCE node
I20260709 12:59:08.572054  1151 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36925
I20260709 12:59:08.572412  1151 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:08.572455   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:08.572821   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:08.573025   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601948573006 us; error 0 us; skew 500 ppm
I20260709 12:59:08.573287  1151 heartbeater.cc:507] Master 127.0.107.254:36925 requested a full tablet report, sending...
I20260709 12:59:08.573779   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:08.575611   949 ts_manager.cc:194] Registered new tserver with Master: a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335)
I20260709 12:59:08.576650   431 webserver.cc:533] Webserver started at http://127.0.107.195:42461/ using document root <none> and password file <none>
I20260709 12:59:08.577137   949 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46834
I20260709 12:59:08.577219   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:08.577536   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:08.577838   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:08.579051   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-2-root/instance:
uuid: "c4958a1501dd46e4b8f154a28201eb7a"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-jk05"
I20260709 12:59:08.583858   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:59:08.587404  1162 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.588392   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:59:08.588655   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-2-root
uuid: "c4958a1501dd46e4b8f154a28201eb7a"
format_stamp: "Formatted at 2026-07-09 12:59:08 on dist-test-slave-jk05"
I20260709 12:59:08.588901   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783601935286233-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:08.606941   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:08.608130   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:08.609565   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:08.611665   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:08.611881   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.612215   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:08.612397   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:08.652374   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:40315
I20260709 12:59:08.652560  1224 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:40315 every 8 connection(s)
I20260709 12:59:08.666239  1225 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36925
I20260709 12:59:08.666661  1225 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:08.667315  1225 heartbeater.cc:507] Master 127.0.107.254:36925 requested a full tablet report, sending...
I20260709 12:59:08.669287   949 ts_manager.cc:194] Registered new tserver with Master: c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:08.670038   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014599725s
I20260709 12:59:08.670943   949 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46850
I20260709 12:59:08.696801   949 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46854:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:59:08.699837   949 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:59:08.775941  1041 tablet_service.cc:1511] Processing CreateTablet for tablet 0033bb4a59084e28ac4003e6845c1ae7 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:59:08.776660  1114 tablet_service.cc:1511] Processing CreateTablet for tablet 1793808c703a4b5fba0a518e39f7c7c3 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:59:08.777454  1041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0033bb4a59084e28ac4003e6845c1ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.778060  1114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1793808c703a4b5fba0a518e39f7c7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.777760  1115 tablet_service.cc:1511] Processing CreateTablet for tablet 5057bc9d1ba6441f8fcd0302355b2687 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:59:08.779107  1115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5057bc9d1ba6441f8fcd0302355b2687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.781774  1040 tablet_service.cc:1511] Processing CreateTablet for tablet 5057bc9d1ba6441f8fcd0302355b2687 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:59:08.782156  1113 tablet_service.cc:1511] Processing CreateTablet for tablet 086f7e48b7d44aabb6a2cf053f3afb95 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:59:08.783185  1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5057bc9d1ba6441f8fcd0302355b2687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.783555  1113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 086f7e48b7d44aabb6a2cf053f3afb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.786691  1039 tablet_service.cc:1511] Processing CreateTablet for tablet 1793808c703a4b5fba0a518e39f7c7c3 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:59:08.788141  1039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1793808c703a4b5fba0a518e39f7c7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.775861  1116 tablet_service.cc:1511] Processing CreateTablet for tablet 0033bb4a59084e28ac4003e6845c1ae7 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:59:08.790309  1116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0033bb4a59084e28ac4003e6845c1ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.791333  1038 tablet_service.cc:1511] Processing CreateTablet for tablet 086f7e48b7d44aabb6a2cf053f3afb95 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:59:08.792933  1038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 086f7e48b7d44aabb6a2cf053f3afb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.797981  1112 tablet_service.cc:1511] Processing CreateTablet for tablet c6e66356fcb14d2b825ab6666ac32d49 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:59:08.799425  1112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6e66356fcb14d2b825ab6666ac32d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.801843  1037 tablet_service.cc:1511] Processing CreateTablet for tablet c6e66356fcb14d2b825ab6666ac32d49 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:59:08.803228  1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6e66356fcb14d2b825ab6666ac32d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.812529  1190 tablet_service.cc:1511] Processing CreateTablet for tablet 0033bb4a59084e28ac4003e6845c1ae7 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:59:08.813989  1190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0033bb4a59084e28ac4003e6845c1ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.815022  1036 tablet_service.cc:1511] Processing CreateTablet for tablet 27555eb30a6241aca84fe4537aa8e33a (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:59:08.815613  1189 tablet_service.cc:1511] Processing CreateTablet for tablet 5057bc9d1ba6441f8fcd0302355b2687 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:59:08.816450  1036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27555eb30a6241aca84fe4537aa8e33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.816975  1189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5057bc9d1ba6441f8fcd0302355b2687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.819269  1187 tablet_service.cc:1511] Processing CreateTablet for tablet 086f7e48b7d44aabb6a2cf053f3afb95 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:59:08.820725  1187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 086f7e48b7d44aabb6a2cf053f3afb95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.822510  1186 tablet_service.cc:1511] Processing CreateTablet for tablet c6e66356fcb14d2b825ab6666ac32d49 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:59:08.823869  1186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6e66356fcb14d2b825ab6666ac32d49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.824554  1035 tablet_service.cc:1511] Processing CreateTablet for tablet 089f3cb6025f429e911a1f7f04eae9a7 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:59:08.825889  1035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 089f3cb6025f429e911a1f7f04eae9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.827658  1185 tablet_service.cc:1511] Processing CreateTablet for tablet 27555eb30a6241aca84fe4537aa8e33a (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:59:08.829054  1185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27555eb30a6241aca84fe4537aa8e33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.813680  1188 tablet_service.cc:1511] Processing CreateTablet for tablet 1793808c703a4b5fba0a518e39f7c7c3 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:59:08.833485  1188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1793808c703a4b5fba0a518e39f7c7c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.834121  1113 tablet_service.cc:1511] Processing CreateTablet for tablet 089f3cb6025f429e911a1f7f04eae9a7 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:59:08.835438  1113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 089f3cb6025f429e911a1f7f04eae9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.842721  1184 tablet_service.cc:1511] Processing CreateTablet for tablet 089f3cb6025f429e911a1f7f04eae9a7 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:59:08.844662  1184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 089f3cb6025f429e911a1f7f04eae9a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.844727  1115 tablet_service.cc:1511] Processing CreateTablet for tablet 27555eb30a6241aca84fe4537aa8e33a (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:59:08.846170  1115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27555eb30a6241aca84fe4537aa8e33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.857160  1034 tablet_service.cc:1511] Processing CreateTablet for tablet 848e80771a8c48d287bc912ac84bf065 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:59:08.860193  1034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 848e80771a8c48d287bc912ac84bf065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.882186  1240 tablet_bootstrap.cc:492] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3: Bootstrap starting.
I20260709 12:59:08.898746  1241 tablet_bootstrap.cc:492] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a: Bootstrap starting.
I20260709 12:59:08.904069  1114 tablet_service.cc:1511] Processing CreateTablet for tablet 848e80771a8c48d287bc912ac84bf065 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:59:08.907037  1114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 848e80771a8c48d287bc912ac84bf065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.916569  1187 tablet_service.cc:1511] Processing CreateTablet for tablet 848e80771a8c48d287bc912ac84bf065 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:59:08.918162  1187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 848e80771a8c48d287bc912ac84bf065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.920936  1240 tablet_bootstrap.cc:654] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:08.926725  1040 tablet_service.cc:1511] Processing CreateTablet for tablet a0dff6acdeff48a691083fdb124dfd20 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:59:08.928665  1040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0dff6acdeff48a691083fdb124dfd20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.942288  1185 tablet_service.cc:1511] Processing CreateTablet for tablet a0dff6acdeff48a691083fdb124dfd20 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:59:08.943747  1185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0dff6acdeff48a691083fdb124dfd20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.953039  1115 tablet_service.cc:1511] Processing CreateTablet for tablet 372642901bcb4c20a599bc5f223772f7 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:59:08.958562  1037 tablet_service.cc:1511] Processing CreateTablet for tablet 372642901bcb4c20a599bc5f223772f7 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:59:08.959916  1037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 372642901bcb4c20a599bc5f223772f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.956960  1115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 372642901bcb4c20a599bc5f223772f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.975190  1242 tablet_bootstrap.cc:492] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8: Bootstrap starting.
I20260709 12:59:08.975903  1240 tablet_bootstrap.cc:492] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3: No bootstrap required, opened a new log
I20260709 12:59:08.976266  1240 ts_tablet_manager.cc:1403] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent bootstrapping tablet: real 0.094s	user 0.016s	sys 0.018s
I20260709 12:59:08.978674  1240 raft_consensus.cc:359] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:08.979410  1240 raft_consensus.cc:385] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:08.979712  1240 raft_consensus.cc:740] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Initialized, Role: FOLLOWER
I20260709 12:59:08.980434  1240 consensus_queue.cc:260] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:08.981914  1113 tablet_service.cc:1511] Processing CreateTablet for tablet a0dff6acdeff48a691083fdb124dfd20 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:59:08.983433  1185 tablet_service.cc:1511] Processing CreateTablet for tablet 372642901bcb4c20a599bc5f223772f7 (DEFAULT_TABLE table=test-workload [id=8f606019a870418b83f9ec8095495a60]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:59:08.987875  1242 tablet_bootstrap.cc:654] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:08.990880  1113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0dff6acdeff48a691083fdb124dfd20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.991228  1185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 372642901bcb4c20a599bc5f223772f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:08.989144  1151 heartbeater.cc:499] Master 127.0.107.254:36925 was elected leader, sending a full tablet report...
I20260709 12:59:08.989711  1240 ts_tablet_manager.cc:1434] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.001s
I20260709 12:59:08.993947  1241 tablet_bootstrap.cc:654] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.000998  1240 tablet_bootstrap.cc:492] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3: Bootstrap starting.
I20260709 12:59:09.008237  1244 raft_consensus.cc:493] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.008797  1244 raft_consensus.cc:515] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.022578  1244 leader_election.cc:290] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.024478  1242 tablet_bootstrap.cc:492] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8: No bootstrap required, opened a new log
I20260709 12:59:09.024988  1242 ts_tablet_manager.cc:1403] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.004s
I20260709 12:59:09.025631  1240 tablet_bootstrap.cc:654] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.028080  1242 raft_consensus.cc:359] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.028803  1242 raft_consensus.cc:385] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.029106  1242 raft_consensus.cc:740] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.029946  1242 consensus_queue.cc:260] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.036590  1240 tablet_bootstrap.cc:492] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3: No bootstrap required, opened a new log
I20260709 12:59:09.036784  1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8" is_pre_election: true
I20260709 12:59:09.037243  1240 ts_tablet_manager.cc:1403] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent bootstrapping tablet: real 0.037s	user 0.005s	sys 0.011s
W20260709 12:59:09.040304  1090 leader_election.cc:343] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.043694  1240 raft_consensus.cc:359] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.046342  1240 raft_consensus.cc:385] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.046708  1241 tablet_bootstrap.cc:492] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a: No bootstrap required, opened a new log
I20260709 12:59:09.047266  1241 ts_tablet_manager.cc:1403] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a: Time spent bootstrapping tablet: real 0.149s	user 0.022s	sys 0.029s
I20260709 12:59:09.047847  1076 heartbeater.cc:499] Master 127.0.107.254:36925 was elected leader, sending a full tablet report...
I20260709 12:59:09.048396  1242 ts_tablet_manager.cc:1434] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8: Time spent starting tablet: real 0.023s	user 0.004s	sys 0.017s
I20260709 12:59:09.049374  1242 tablet_bootstrap.cc:492] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8: Bootstrap starting.
I20260709 12:59:09.050310  1241 raft_consensus.cc:359] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:09.051082  1241 raft_consensus.cc:385] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.051374  1241 raft_consensus.cc:740] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.046717  1240 raft_consensus.cc:740] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.052292  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a" is_pre_election: true
I20260709 12:59:09.052583  1240 consensus_queue.cc:260] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.052583  1241 consensus_queue.cc:260] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
W20260709 12:59:09.054522  1092 leader_election.cc:343] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.054999  1092 leader_election.cc:304] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3; no voters: a6ec7e01e34945bb848cd646a3a171f8, c4958a1501dd46e4b8f154a28201eb7a
I20260709 12:59:09.055496  1240 ts_tablet_manager.cc:1434] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:59:09.056406  1244 raft_consensus.cc:2749] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:09.059015  1240 tablet_bootstrap.cc:492] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3: Bootstrap starting.
I20260709 12:59:09.069832  1240 tablet_bootstrap.cc:654] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.071367  1242 tablet_bootstrap.cc:654] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.087991  1240 tablet_bootstrap.cc:492] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3: No bootstrap required, opened a new log
I20260709 12:59:09.088496  1240 ts_tablet_manager.cc:1403] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.016s
I20260709 12:59:09.091683  1240 raft_consensus.cc:359] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.092432  1240 raft_consensus.cc:385] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.092852  1240 raft_consensus.cc:740] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.093708  1240 consensus_queue.cc:260] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.096253  1240 ts_tablet_manager.cc:1434] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 12:59:09.097234  1240 tablet_bootstrap.cc:492] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3: Bootstrap starting.
I20260709 12:59:09.103735  1240 tablet_bootstrap.cc:654] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:09.107705  1092 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:40315, user_credentials={real_user=slave}} blocked reactor thread for 53291.1us
I20260709 12:59:09.110183  1225 heartbeater.cc:499] Master 127.0.107.254:36925 was elected leader, sending a full tablet report...
I20260709 12:59:09.112048  1241 ts_tablet_manager.cc:1434] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a: Time spent starting tablet: real 0.064s	user 0.020s	sys 0.041s
I20260709 12:59:09.113198  1240 tablet_bootstrap.cc:492] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3: No bootstrap required, opened a new log
I20260709 12:59:09.113722  1240 ts_tablet_manager.cc:1403] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.006s
I20260709 12:59:09.114195  1241 tablet_bootstrap.cc:492] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a: Bootstrap starting.
I20260709 12:59:09.115180  1242 tablet_bootstrap.cc:492] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8: No bootstrap required, opened a new log
I20260709 12:59:09.115710  1242 ts_tablet_manager.cc:1403] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.006s
I20260709 12:59:09.116606  1240 raft_consensus.cc:359] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.117347  1240 raft_consensus.cc:385] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.117688  1240 raft_consensus.cc:740] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.118507  1240 consensus_queue.cc:260] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.120693  1240 ts_tablet_manager.cc:1434] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:09.121636  1240 tablet_bootstrap.cc:492] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3: Bootstrap starting.
I20260709 12:59:09.121750  1241 tablet_bootstrap.cc:654] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.118897  1242 raft_consensus.cc:359] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.123974  1242 raft_consensus.cc:385] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.124377  1242 raft_consensus.cc:740] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.125361  1242 consensus_queue.cc:260] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.128472  1240 tablet_bootstrap.cc:654] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.137984  1242 ts_tablet_manager.cc:1434] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:59:09.139566  1242 tablet_bootstrap.cc:492] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8: Bootstrap starting.
I20260709 12:59:09.148948  1242 tablet_bootstrap.cc:654] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.153223  1241 tablet_bootstrap.cc:492] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a: No bootstrap required, opened a new log
I20260709 12:59:09.153221  1240 tablet_bootstrap.cc:492] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3: No bootstrap required, opened a new log
I20260709 12:59:09.153934  1241 ts_tablet_manager.cc:1403] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a: Time spent bootstrapping tablet: real 0.040s	user 0.021s	sys 0.016s
I20260709 12:59:09.153980  1240 ts_tablet_manager.cc:1403] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent bootstrapping tablet: real 0.033s	user 0.021s	sys 0.010s
I20260709 12:59:09.157205  1241 raft_consensus.cc:359] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.157070  1240 raft_consensus.cc:359] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:09.157927  1241 raft_consensus.cc:385] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.158046  1240 raft_consensus.cc:385] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.158313  1241 raft_consensus.cc:740] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.158478  1240 raft_consensus.cc:740] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.159291  1241 consensus_queue.cc:260] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.159512  1240 consensus_queue.cc:260] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:09.161878  1240 ts_tablet_manager.cc:1434] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:59:09.162946  1240 tablet_bootstrap.cc:492] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3: Bootstrap starting.
I20260709 12:59:09.164217  1241 ts_tablet_manager.cc:1434] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.004s
I20260709 12:59:09.165369  1241 tablet_bootstrap.cc:492] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a: Bootstrap starting.
I20260709 12:59:09.170240  1240 tablet_bootstrap.cc:654] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.172091  1241 tablet_bootstrap.cc:654] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.178546  1242 tablet_bootstrap.cc:492] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8: No bootstrap required, opened a new log
I20260709 12:59:09.179188  1242 ts_tablet_manager.cc:1403] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8: Time spent bootstrapping tablet: real 0.040s	user 0.027s	sys 0.009s
I20260709 12:59:09.180289  1241 tablet_bootstrap.cc:492] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a: No bootstrap required, opened a new log
I20260709 12:59:09.180781  1241 ts_tablet_manager.cc:1403] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.003s
I20260709 12:59:09.182842  1242 raft_consensus.cc:359] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.183631  1242 raft_consensus.cc:385] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.183938  1242 raft_consensus.cc:740] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.184249  1240 tablet_bootstrap.cc:492] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3: No bootstrap required, opened a new log
I20260709 12:59:09.184827  1240 ts_tablet_manager.cc:1403] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.002s
I20260709 12:59:09.184791  1242 consensus_queue.cc:260] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.183956  1241 raft_consensus.cc:359] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.186079  1241 raft_consensus.cc:385] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.186393  1241 raft_consensus.cc:740] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.187120  1249 raft_consensus.cc:493] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.187368  1241 consensus_queue.cc:260] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.187597  1249 raft_consensus.cc:515] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.189266  1241 ts_tablet_manager.cc:1434] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:59:09.188958  1240 raft_consensus.cc:359] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.189716  1240 raft_consensus.cc:385] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.190052  1240 raft_consensus.cc:740] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.190310  1241 tablet_bootstrap.cc:492] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a: Bootstrap starting.
I20260709 12:59:09.191118  1240 consensus_queue.cc:260] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.193348  1240 ts_tablet_manager.cc:1434] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:59:09.194577  1240 tablet_bootstrap.cc:492] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3: Bootstrap starting.
I20260709 12:59:09.197183  1242 ts_tablet_manager.cc:1434] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.002s
I20260709 12:59:09.198266  1242 tablet_bootstrap.cc:492] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8: Bootstrap starting.
I20260709 12:59:09.202225  1240 tablet_bootstrap.cc:654] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.203585  1249 leader_election.cc:290] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.205319  1242 tablet_bootstrap.cc:654] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.208598  1241 tablet_bootstrap.cc:654] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.216034  1240 tablet_bootstrap.cc:492] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3: No bootstrap required, opened a new log
I20260709 12:59:09.216662  1240 ts_tablet_manager.cc:1403] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.011s
I20260709 12:59:09.219437  1241 tablet_bootstrap.cc:492] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a: No bootstrap required, opened a new log
I20260709 12:59:09.222175  1241 ts_tablet_manager.cc:1403] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.005s
I20260709 12:59:09.220260  1240 raft_consensus.cc:359] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.223167  1240 raft_consensus.cc:385] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.223483  1240 raft_consensus.cc:740] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.225708  1242 tablet_bootstrap.cc:492] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8: No bootstrap required, opened a new log
I20260709 12:59:09.225649  1241 raft_consensus.cc:359] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.226668  1241 raft_consensus.cc:385] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.227070  1241 raft_consensus.cc:740] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.226305  1240 consensus_queue.cc:260] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.228003  1241 consensus_queue.cc:260] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.229123  1240 ts_tablet_manager.cc:1434] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 12:59:09.227213  1242 ts_tablet_manager.cc:1403] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.013s
I20260709 12:59:09.230346  1241 ts_tablet_manager.cc:1434] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:59:09.231397  1241 tablet_bootstrap.cc:492] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a: Bootstrap starting.
I20260709 12:59:09.232163  1242 raft_consensus.cc:359] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.232528  1250 raft_consensus.cc:493] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.232952  1242 raft_consensus.cc:385] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.233309  1242 raft_consensus.cc:740] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.233079  1250 raft_consensus.cc:515] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.234100  1242 consensus_queue.cc:260] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.236410  1242 ts_tablet_manager.cc:1434] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:59:09.237241  1240 tablet_bootstrap.cc:492] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3: Bootstrap starting.
I20260709 12:59:09.237377  1242 tablet_bootstrap.cc:492] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8: Bootstrap starting.
I20260709 12:59:09.239933  1250 leader_election.cc:290] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335)
I20260709 12:59:09.244457  1240 tablet_bootstrap.cc:654] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.245009  1242 tablet_bootstrap.cc:654] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.255568  1241 tablet_bootstrap.cc:654] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.259419  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e66356fcb14d2b825ab6666ac32d49" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" is_pre_election: true
I20260709 12:59:09.260046  1126 raft_consensus.cc:2468] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 0.
I20260709 12:59:09.265995  1240 tablet_bootstrap.cc:492] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3: No bootstrap required, opened a new log
I20260709 12:59:09.266528  1240 ts_tablet_manager.cc:1403] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.004s
I20260709 12:59:09.266053  1016 leader_election.cc:304] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, a7d1e1a9de284c5ab09ed49f577557c3; no voters: 
I20260709 12:59:09.268437  1249 raft_consensus.cc:2804] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.268867  1249 raft_consensus.cc:493] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.269275  1249 raft_consensus.cc:3060] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.269143  1240 raft_consensus.cc:359] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.269865  1240 raft_consensus.cc:385] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.270155  1240 raft_consensus.cc:740] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.273327  1240 consensus_queue.cc:260] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.277026  1240 ts_tablet_manager.cc:1434] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:59:09.278012  1240 tablet_bootstrap.cc:492] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3: Bootstrap starting.
I20260709 12:59:09.283865  1240 tablet_bootstrap.cc:654] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.285199  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1793808c703a4b5fba0a518e39f7c7c3" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" is_pre_election: true
I20260709 12:59:09.285936  1126 raft_consensus.cc:2468] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 0.
I20260709 12:59:09.287451  1165 leader_election.cc:304] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:09.288391  1250 raft_consensus.cc:2804] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.288741  1250 raft_consensus.cc:493] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.289091  1250 raft_consensus.cc:3060] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.293546  1251 raft_consensus.cc:493] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.294062  1251 raft_consensus.cc:515] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.296705  1249 raft_consensus.cc:515] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.298869  1251 leader_election.cc:290] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.309878  1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1793808c703a4b5fba0a518e39f7c7c3" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8" is_pre_election: true
I20260709 12:59:09.310629  1051 raft_consensus.cc:2468] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 0.
W20260709 12:59:09.311014   997 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:09.312179  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a" is_pre_election: true
I20260709 12:59:09.312808  1200 raft_consensus.cc:2468] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7d1e1a9de284c5ab09ed49f577557c3 in term 0.
I20260709 12:59:09.317922  1092 leader_election.cc:304] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:09.318856  1251 raft_consensus.cc:2804] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.319222  1251 raft_consensus.cc:493] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.320070  1251 raft_consensus.cc:3060] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.319236  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e66356fcb14d2b825ab6666ac32d49" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a" is_pre_election: true
W20260709 12:59:09.322209  1017 leader_election.cc:343] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.323776  1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8" is_pre_election: true
I20260709 12:59:09.328106  1242 tablet_bootstrap.cc:492] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8: No bootstrap required, opened a new log
I20260709 12:59:09.328699  1241 tablet_bootstrap.cc:492] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a: No bootstrap required, opened a new log
I20260709 12:59:09.329363  1241 ts_tablet_manager.cc:1403] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a: Time spent bootstrapping tablet: real 0.098s	user 0.016s	sys 0.019s
I20260709 12:59:09.330917  1251 raft_consensus.cc:515] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
W20260709 12:59:09.326184  1090 leader_election.cc:343] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.333518  1251 leader_election.cc:290] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 election: Requested vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.333662  1250 raft_consensus.cc:515] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.335843  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:09.328673  1242 ts_tablet_manager.cc:1403] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8: Time spent bootstrapping tablet: real 0.091s	user 0.005s	sys 0.025s
I20260709 12:59:09.336450  1200 raft_consensus.cc:3060] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.337407  1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
I20260709 12:59:09.339664  1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1793808c703a4b5fba0a518e39f7c7c3" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
I20260709 12:59:09.340314  1051 raft_consensus.cc:3060] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.340629  1242 raft_consensus.cc:359] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
W20260709 12:59:09.342507  1090 leader_election.cc:343] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.344383  1249 leader_election.cc:290] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 election: Requested vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.348526  1242 raft_consensus.cc:385] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.348939  1242 raft_consensus.cc:740] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.350083  1051 raft_consensus.cc:2468] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 1.
I20260709 12:59:09.349824  1242 consensus_queue.cc:260] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:09.350248  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1793808c703a4b5fba0a518e39f7c7c3" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:09.350816  1126 raft_consensus.cc:3060] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.351626  1164 leader_election.cc:304] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:09.344264  1241 raft_consensus.cc:359] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.345114  1250 leader_election.cc:290] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 1 election: Requested vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335)
I20260709 12:59:09.353341  1272 raft_consensus.cc:2804] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.346946  1240 tablet_bootstrap.cc:492] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3: No bootstrap required, opened a new log
I20260709 12:59:09.354172  1240 ts_tablet_manager.cc:1403] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent bootstrapping tablet: real 0.076s	user 0.027s	sys 0.024s
I20260709 12:59:09.355072  1242 ts_tablet_manager.cc:1434] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:59:09.356060  1242 tablet_bootstrap.cc:492] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8: Bootstrap starting.
I20260709 12:59:09.346343  1199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e66356fcb14d2b825ab6666ac32d49" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:09.356703  1240 raft_consensus.cc:359] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.357357  1240 raft_consensus.cc:385] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:59:09.357637  1017 leader_election.cc:343] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.357786  1240 raft_consensus.cc:740] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.347952  1200 raft_consensus.cc:2468] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7d1e1a9de284c5ab09ed49f577557c3 in term 1.
I20260709 12:59:09.358582  1240 consensus_queue.cc:260] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.365903  1125 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6e66356fcb14d2b825ab6666ac32d49" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:09.366178  1092 leader_election.cc:304] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: a6ec7e01e34945bb848cd646a3a171f8
I20260709 12:59:09.366565  1125 raft_consensus.cc:3060] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.366958  1240 ts_tablet_manager.cc:1434] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 12:59:09.359035  1250 raft_consensus.cc:493] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.367429  1250 raft_consensus.cc:515] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:09.353508  1241 raft_consensus.cc:385] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.369632  1250 leader_election.cc:290] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129)
I20260709 12:59:09.369019  1241 raft_consensus.cc:740] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.371254  1241 consensus_queue.cc:260] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.372328  1251 raft_consensus.cc:2804] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.372812  1251 raft_consensus.cc:697] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 LEADER]: Becoming Leader. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Running, Role: LEADER
I20260709 12:59:09.373625  1251 consensus_queue.cc:237] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.375278  1244 raft_consensus.cc:493] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.375761  1244 raft_consensus.cc:515] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.377750  1244 leader_election.cc:290] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.378638  1125 raft_consensus.cc:2468] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 1.
I20260709 12:59:09.379345  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" is_pre_election: true
I20260709 12:59:09.380007  1124 raft_consensus.cc:2468] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 0.
I20260709 12:59:09.380100  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372642901bcb4c20a599bc5f223772f7" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a" is_pre_election: true
I20260709 12:59:09.380892  1244 raft_consensus.cc:493] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.381676  1241 ts_tablet_manager.cc:1434] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 12:59:09.381345  1165 leader_election.cc:304] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:09.381830  1240 tablet_bootstrap.cc:492] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3: Bootstrap starting.
I20260709 12:59:09.381351  1244 raft_consensus.cc:515] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.383339  1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372642901bcb4c20a599bc5f223772f7" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8" is_pre_election: true
I20260709 12:59:09.383993  1126 raft_consensus.cc:2468] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 1.
I20260709 12:59:09.384832  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8" is_pre_election: true
I20260709 12:59:09.385043  1244 leader_election.cc:290] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.385633  1250 raft_consensus.cc:2804] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.385535  1050 raft_consensus.cc:2468] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 0.
I20260709 12:59:09.386482  1250 raft_consensus.cc:493] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.386904  1250 raft_consensus.cc:3060] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.387264  1199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27555eb30a6241aca84fe4537aa8e33a" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a" is_pre_election: true
I20260709 12:59:09.388052  1199 raft_consensus.cc:2468] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7d1e1a9de284c5ab09ed49f577557c3 in term 0.
I20260709 12:59:09.388849  1272 raft_consensus.cc:697] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 1 LEADER]: Becoming Leader. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Running, Role: LEADER
I20260709 12:59:09.389649  1092 leader_election.cc:304] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:09.393584  1244 raft_consensus.cc:2804] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.394018  1244 raft_consensus.cc:493] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.394348  1244 raft_consensus.cc:3060] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.394462  1272 consensus_queue.cc:237] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
W20260709 12:59:09.390573  1090 leader_election.cc:343] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.397025  1240 tablet_bootstrap.cc:654] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:09.397152  1092 leader_election.cc:343] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.392110  1016 leader_election.cc:304] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, a7d1e1a9de284c5ab09ed49f577557c3; no voters: c4958a1501dd46e4b8f154a28201eb7a
I20260709 12:59:09.382817  1241 tablet_bootstrap.cc:492] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a: Bootstrap starting.
I20260709 12:59:09.399623  1249 raft_consensus.cc:2804] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.390431  1242 tablet_bootstrap.cc:654] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.399936  1092 leader_election.cc:304] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3; no voters: a6ec7e01e34945bb848cd646a3a171f8, c4958a1501dd46e4b8f154a28201eb7a
I20260709 12:59:09.401757  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27555eb30a6241aca84fe4537aa8e33a" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8" is_pre_election: true
I20260709 12:59:09.402072   948 catalog_manager.cc:5697] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:09.403535  1250 raft_consensus.cc:515] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:09.408357  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:09.408959  1126 raft_consensus.cc:3060] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.410583  1244 raft_consensus.cc:515] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
W20260709 12:59:09.403510  1090 leader_election.cc:343] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.411739  1251 raft_consensus.cc:2749] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:09.419668  1249 raft_consensus.cc:697] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Becoming Leader. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:09.423473  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
I20260709 12:59:09.424171  1050 raft_consensus.cc:3060] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.426688  1241 tablet_bootstrap.cc:654] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.429517  1250 leader_election.cc:290] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 1 election: Requested vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129)
I20260709 12:59:09.430097   949 catalog_manager.cc:5697] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a reported cstate change: term changed from 0 to 1, leader changed from <none> to c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "c4958a1501dd46e4b8f154a28201eb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:09.432080  1126 raft_consensus.cc:2468] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 1.
I20260709 12:59:09.433777  1165 leader_election.cc:304] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:09.439337  1250 raft_consensus.cc:2804] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.439863  1250 raft_consensus.cc:697] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 LEADER]: Becoming Leader. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Running, Role: LEADER
I20260709 12:59:09.441740  1050 raft_consensus.cc:2468] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 1.
I20260709 12:59:09.437592  1249 consensus_queue.cc:237] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.442962  1241 tablet_bootstrap.cc:492] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a: No bootstrap required, opened a new log
I20260709 12:59:09.443116  1242 tablet_bootstrap.cc:492] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8: No bootstrap required, opened a new log
I20260709 12:59:09.443480  1241 ts_tablet_manager.cc:1403] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a: Time spent bootstrapping tablet: real 0.061s	user 0.007s	sys 0.007s
I20260709 12:59:09.443753  1242 ts_tablet_manager.cc:1403] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8: Time spent bootstrapping tablet: real 0.088s	user 0.023s	sys 0.005s
I20260709 12:59:09.444919  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27555eb30a6241aca84fe4537aa8e33a" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
I20260709 12:59:09.445497  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27555eb30a6241aca84fe4537aa8e33a" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:09.446113  1200 raft_consensus.cc:3060] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.440902  1250 consensus_queue.cc:237] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:09.447556  1240 tablet_bootstrap.cc:492] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3: No bootstrap required, opened a new log
I20260709 12:59:09.447988  1240 ts_tablet_manager.cc:1403] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent bootstrapping tablet: real 0.066s	user 0.019s	sys 0.001s
I20260709 12:59:09.450315  1242 raft_consensus.cc:359] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.451056  1242 raft_consensus.cc:385] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.451340  1242 raft_consensus.cc:740] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.451573  1240 raft_consensus.cc:359] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.452433  1240 raft_consensus.cc:385] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.452795  1240 raft_consensus.cc:740] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.448391  1244 leader_election.cc:290] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 election: Requested vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.454558  1242 consensus_queue.cc:260] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.456326  1242 ts_tablet_manager.cc:1434] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:59:09.457180  1242 tablet_bootstrap.cc:492] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8: Bootstrap starting.
I20260709 12:59:09.458003  1240 consensus_queue.cc:260] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.459860  1200 raft_consensus.cc:2468] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7d1e1a9de284c5ab09ed49f577557c3 in term 1.
W20260709 12:59:09.460882  1090 leader_election.cc:343] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:09.461606  1240 ts_tablet_manager.cc:1434] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:59:09.461503  1092 leader_election.cc:304] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: a6ec7e01e34945bb848cd646a3a171f8
I20260709 12:59:09.462318  1251 raft_consensus.cc:2804] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.462811  1251 raft_consensus.cc:697] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 LEADER]: Becoming Leader. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Running, Role: LEADER
I20260709 12:59:09.464078  1242 tablet_bootstrap.cc:654] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.463603  1251 consensus_queue.cc:237] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.473560  1241 raft_consensus.cc:359] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.474740  1241 raft_consensus.cc:385] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.473825   946 catalog_manager.cc:5697] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:09.475155  1241 raft_consensus.cc:740] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.476339  1241 consensus_queue.cc:260] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.479884  1241 ts_tablet_manager.cc:1434] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.000s
I20260709 12:59:09.481570  1241 tablet_bootstrap.cc:492] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a: Bootstrap starting.
I20260709 12:59:09.490276  1241 tablet_bootstrap.cc:654] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.503643  1249 raft_consensus.cc:493] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.504372  1249 raft_consensus.cc:515] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.507145  1242 tablet_bootstrap.cc:492] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8: No bootstrap required, opened a new log
I20260709 12:59:09.507893  1242 ts_tablet_manager.cc:1403] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.010s
I20260709 12:59:09.508538  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086f7e48b7d44aabb6a2cf053f3afb95" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" is_pre_election: true
I20260709 12:59:09.509461  1126 raft_consensus.cc:2468] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 0.
I20260709 12:59:09.511118  1016 leader_election.cc:304] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, a7d1e1a9de284c5ab09ed49f577557c3; no voters: 
I20260709 12:59:09.512065  1276 raft_consensus.cc:2804] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.512595  1276 raft_consensus.cc:493] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.512372  1242 raft_consensus.cc:359] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.513128  1276 raft_consensus.cc:3060] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.513530  1242 raft_consensus.cc:385] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.514020  1242 raft_consensus.cc:740] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.516710  1242 consensus_queue.cc:260] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.518144  1241 tablet_bootstrap.cc:492] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a: No bootstrap required, opened a new log
I20260709 12:59:09.518638  1241 ts_tablet_manager.cc:1403] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a: Time spent bootstrapping tablet: real 0.037s	user 0.021s	sys 0.015s
I20260709 12:59:09.519215  1249 leader_election.cc:290] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.519596  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086f7e48b7d44aabb6a2cf053f3afb95" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a" is_pre_election: true
I20260709 12:59:09.520431  1200 raft_consensus.cc:2468] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 0.
I20260709 12:59:09.522569  1241 raft_consensus.cc:359] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.523612  1241 raft_consensus.cc:385] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.524014  1241 raft_consensus.cc:740] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.525215  1241 consensus_queue.cc:260] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.528875  1241 ts_tablet_manager.cc:1434] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:59:09.528816  1276 raft_consensus.cc:515] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.530347  1241 tablet_bootstrap.cc:492] T c6e66356fcb14d2b825ab6666ac32d49 P c4958a1501dd46e4b8f154a28201eb7a: Bootstrap starting.
I20260709 12:59:09.530821   948 catalog_manager.cc:5697] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a reported cstate change: term changed from 0 to 1, leader changed from <none> to c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195). New cstate: current_term: 1 leader_uuid: "c4958a1501dd46e4b8f154a28201eb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:09.532425  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086f7e48b7d44aabb6a2cf053f3afb95" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:09.533046  1126 raft_consensus.cc:3060] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.540129  1242 ts_tablet_manager.cc:1434] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8: Time spent starting tablet: real 0.032s	user 0.015s	sys 0.013s
I20260709 12:59:09.542069  1242 tablet_bootstrap.cc:492] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8: Bootstrap starting.
I20260709 12:59:09.543135  1276 leader_election.cc:290] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 election: Requested vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.544158  1241 tablet_bootstrap.cc:654] T c6e66356fcb14d2b825ab6666ac32d49 P c4958a1501dd46e4b8f154a28201eb7a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.544301  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "086f7e48b7d44aabb6a2cf053f3afb95" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:09.544884  1200 raft_consensus.cc:3060] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.542456  1126 raft_consensus.cc:2468] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 1.
I20260709 12:59:09.547264  1016 leader_election.cc:304] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, a7d1e1a9de284c5ab09ed49f577557c3; no voters: 
I20260709 12:59:09.548163  1276 raft_consensus.cc:2804] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.548851  1276 raft_consensus.cc:697] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Becoming Leader. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:09.549685  1276 consensus_queue.cc:237] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.552444  1200 raft_consensus.cc:2468] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 1.
I20260709 12:59:09.552990  1242 tablet_bootstrap.cc:654] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.561914   949 catalog_manager.cc:5697] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:09.567488  1242 tablet_bootstrap.cc:492] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8: No bootstrap required, opened a new log
I20260709 12:59:09.568272  1242 ts_tablet_manager.cc:1403] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.009s
I20260709 12:59:09.570349  1289 raft_consensus.cc:493] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.570842  1289 raft_consensus.cc:515] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.573014  1289 leader_election.cc:290] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.576217  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" is_pre_election: true
I20260709 12:59:09.576684  1126 raft_consensus.cc:2468] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 0.
I20260709 12:59:09.577776  1016 leader_election.cc:304] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, a7d1e1a9de284c5ab09ed49f577557c3; no voters: 
I20260709 12:59:09.578902  1289 raft_consensus.cc:2804] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.579337  1289 raft_consensus.cc:493] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.579756  1289 raft_consensus.cc:3060] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.579630  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a" is_pre_election: true
I20260709 12:59:09.580577  1200 raft_consensus.cc:2468] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 0.
I20260709 12:59:09.580473   949 catalog_manager.cc:5697] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:09.571992  1242 raft_consensus.cc:359] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.583606  1242 raft_consensus.cc:385] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.583981  1242 raft_consensus.cc:740] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.585041  1242 consensus_queue.cc:260] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.587772  1242 ts_tablet_manager.cc:1434] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:59:09.588861  1242 tablet_bootstrap.cc:492] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8: Bootstrap starting.
I20260709 12:59:09.590329  1241 tablet_bootstrap.cc:492] T c6e66356fcb14d2b825ab6666ac32d49 P c4958a1501dd46e4b8f154a28201eb7a: No bootstrap required, opened a new log
I20260709 12:59:09.590910  1241 ts_tablet_manager.cc:1403] T c6e66356fcb14d2b825ab6666ac32d49 P c4958a1501dd46e4b8f154a28201eb7a: Time spent bootstrapping tablet: real 0.061s	user 0.029s	sys 0.020s
I20260709 12:59:09.590806  1289 raft_consensus.cc:515] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.593657  1241 raft_consensus.cc:359] T c6e66356fcb14d2b825ab6666ac32d49 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.594034  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:09.594293  1241 raft_consensus.cc:385] T c6e66356fcb14d2b825ab6666ac32d49 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.594543  1242 tablet_bootstrap.cc:654] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.594606  1126 raft_consensus.cc:3060] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.594604  1241 raft_consensus.cc:740] T c6e66356fcb14d2b825ab6666ac32d49 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.595700  1241 consensus_queue.cc:260] T c6e66356fcb14d2b825ab6666ac32d49 P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.598412  1276 raft_consensus.cc:493] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.598914  1276 raft_consensus.cc:515] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.601068  1276 leader_election.cc:290] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.601501  1241 ts_tablet_manager.cc:1434] T c6e66356fcb14d2b825ab6666ac32d49 P c4958a1501dd46e4b8f154a28201eb7a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:59:09.601769  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848e80771a8c48d287bc912ac84bf065" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" is_pre_election: true
I20260709 12:59:09.602262  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848e80771a8c48d287bc912ac84bf065" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a" is_pre_election: true
I20260709 12:59:09.602407  1241 tablet_bootstrap.cc:492] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a: Bootstrap starting.
I20260709 12:59:09.602361  1124 raft_consensus.cc:2468] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 0.
I20260709 12:59:09.602896  1200 raft_consensus.cc:2468] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 0.
I20260709 12:59:09.603663  1016 leader_election.cc:304] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, a7d1e1a9de284c5ab09ed49f577557c3; no voters: 
I20260709 12:59:09.604964  1276 raft_consensus.cc:2804] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.605347  1276 raft_consensus.cc:493] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.605710  1276 raft_consensus.cc:3060] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.608307  1126 raft_consensus.cc:2468] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 1.
I20260709 12:59:09.611312  1289 leader_election.cc:290] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 election: Requested vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.611321  1016 leader_election.cc:304] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, a7d1e1a9de284c5ab09ed49f577557c3; no voters: 
I20260709 12:59:09.612253  1289 raft_consensus.cc:2804] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.613365  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:09.613579  1289 raft_consensus.cc:697] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Becoming Leader. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:09.614037  1200 raft_consensus.cc:3060] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.614668  1289 consensus_queue.cc:237] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.621484  1241 tablet_bootstrap.cc:654] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.622460  1242 tablet_bootstrap.cc:492] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8: No bootstrap required, opened a new log
I20260709 12:59:09.622572  1200 raft_consensus.cc:2468] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 1.
I20260709 12:59:09.623030  1242 ts_tablet_manager.cc:1403] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.016s
I20260709 12:59:09.626457  1242 raft_consensus.cc:359] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.627063  1276 raft_consensus.cc:515] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.627377  1242 raft_consensus.cc:385] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.627894  1242 raft_consensus.cc:740] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.629035  1242 consensus_queue.cc:260] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.631582  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848e80771a8c48d287bc912ac84bf065" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:09.631763  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848e80771a8c48d287bc912ac84bf065" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:09.631884  1242 ts_tablet_manager.cc:1434] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:59:09.632293  1276 leader_election.cc:290] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 election: Requested vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.632172  1200 raft_consensus.cc:3060] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.633450  1242 tablet_bootstrap.cc:492] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8: Bootstrap starting.
I20260709 12:59:09.635118   946 catalog_manager.cc:5697] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:09.632230  1126 raft_consensus.cc:3060] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.639545  1200 raft_consensus.cc:2468] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 1.
I20260709 12:59:09.640806  1017 leader_election.cc:304] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:09.641701  1289 raft_consensus.cc:2804] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.642138  1242 tablet_bootstrap.cc:654] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.642201  1289 raft_consensus.cc:697] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Becoming Leader. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:09.643143  1289 consensus_queue.cc:237] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.650447  1126 raft_consensus.cc:2468] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 1.
I20260709 12:59:09.658102  1241 tablet_bootstrap.cc:492] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a: No bootstrap required, opened a new log
I20260709 12:59:09.658694  1241 ts_tablet_manager.cc:1403] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a: Time spent bootstrapping tablet: real 0.056s	user 0.030s	sys 0.014s
I20260709 12:59:09.658092  1242 tablet_bootstrap.cc:492] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8: No bootstrap required, opened a new log
I20260709 12:59:09.659246  1242 ts_tablet_manager.cc:1403] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.000s
I20260709 12:59:09.661294  1241 raft_consensus.cc:359] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.662014  1241 raft_consensus.cc:385] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.661703  1242 raft_consensus.cc:359] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.662276  1241 raft_consensus.cc:740] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.662400  1242 raft_consensus.cc:385] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.662662  1242 raft_consensus.cc:740] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.663098  1241 consensus_queue.cc:260] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.663396  1242 consensus_queue.cc:260] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.665113  1241 ts_tablet_manager.cc:1434] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:59:09.665113  1242 ts_tablet_manager.cc:1434] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:59:09.666878  1241 tablet_bootstrap.cc:492] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a: Bootstrap starting.
I20260709 12:59:09.666077   946 catalog_manager.cc:5697] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:09.668391  1289 raft_consensus.cc:493] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.668821  1289 raft_consensus.cc:515] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.671386  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" is_pre_election: true
I20260709 12:59:09.671583  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a" is_pre_election: true
I20260709 12:59:09.672289  1200 raft_consensus.cc:2393] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a6ec7e01e34945bb848cd646a3a171f8 in current term 1: Already voted for candidate a7d1e1a9de284c5ab09ed49f577557c3 in this term.
I20260709 12:59:09.673326  1241 tablet_bootstrap.cc:654] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:09.673293  1017 leader_election.cc:304] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8; no voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a
I20260709 12:59:09.674256  1276 raft_consensus.cc:3060] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.678433  1241 tablet_bootstrap.cc:492] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a: No bootstrap required, opened a new log
I20260709 12:59:09.678880  1241 ts_tablet_manager.cc:1403] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a: Time spent bootstrapping tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 12:59:09.680374  1276 raft_consensus.cc:2749] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:09.681345  1289 leader_election.cc:290] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.681666  1241 raft_consensus.cc:359] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.682320  1241 raft_consensus.cc:385] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:09.682571  1241 raft_consensus.cc:740] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Initialized, Role: FOLLOWER
I20260709 12:59:09.683321  1241 consensus_queue.cc:260] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.685396  1241 ts_tablet_manager.cc:1434] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:59:09.694418  1289 raft_consensus.cc:493] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.694891  1289 raft_consensus.cc:515] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.697036  1289 leader_election.cc:290] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.697831  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372642901bcb4c20a599bc5f223772f7" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" is_pre_election: true
I20260709 12:59:09.698136  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372642901bcb4c20a599bc5f223772f7" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a" is_pre_election: true
I20260709 12:59:09.698385  1126 raft_consensus.cc:2468] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 0.
I20260709 12:59:09.698767  1200 raft_consensus.cc:2468] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 0.
I20260709 12:59:09.699486  1016 leader_election.cc:304] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, a7d1e1a9de284c5ab09ed49f577557c3; no voters: 
I20260709 12:59:09.700305  1289 raft_consensus.cc:2804] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.700641  1289 raft_consensus.cc:493] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.700882  1289 raft_consensus.cc:3060] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.705972  1289 raft_consensus.cc:515] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.707082  1276 raft_consensus.cc:493] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:09.707876  1289 leader_election.cc:290] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 election: Requested vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.707623  1276 raft_consensus.cc:515] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.708974  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372642901bcb4c20a599bc5f223772f7" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:09.709579  1200 raft_consensus.cc:3060] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.710795  1276 leader_election.cc:290] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.710661  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0dff6acdeff48a691083fdb124dfd20" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" is_pre_election: true
I20260709 12:59:09.711160  1199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0dff6acdeff48a691083fdb124dfd20" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a" is_pre_election: true
I20260709 12:59:09.711454  1124 raft_consensus.cc:2468] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 0.
I20260709 12:59:09.711796  1199 raft_consensus.cc:2468] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 0.
I20260709 12:59:09.708856  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372642901bcb4c20a599bc5f223772f7" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:09.712891  1017 leader_election.cc:304] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:09.713261  1126 raft_consensus.cc:3060] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.714156  1276 raft_consensus.cc:2804] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:09.714529  1276 raft_consensus.cc:493] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:09.714941  1276 raft_consensus.cc:3060] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.716177  1200 raft_consensus.cc:2468] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 1.
I20260709 12:59:09.717281  1017 leader_election.cc:304] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:09.718182  1289 raft_consensus.cc:2804] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.718642  1289 raft_consensus.cc:697] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Becoming Leader. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:09.719442  1289 consensus_queue.cc:237] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.720556  1126 raft_consensus.cc:2468] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 1.
I20260709 12:59:09.721799  1276 raft_consensus.cc:515] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.724089  1276 leader_election.cc:290] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 election: Requested vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:09.724895  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0dff6acdeff48a691083fdb124dfd20" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:09.725548  1126 raft_consensus.cc:3060] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.727022  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0dff6acdeff48a691083fdb124dfd20" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:09.727545  1200 raft_consensus.cc:3060] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.729012   946 catalog_manager.cc:5697] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:09.732218  1200 raft_consensus.cc:2468] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 1.
I20260709 12:59:09.732448  1126 raft_consensus.cc:2468] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 1.
I20260709 12:59:09.733367  1017 leader_election.cc:304] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:09.734241  1276 raft_consensus.cc:2804] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:09.734768  1276 raft_consensus.cc:697] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Becoming Leader. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:09.735558  1276 consensus_queue.cc:237] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:09.743366   946 catalog_manager.cc:5697] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:09.847445  1250 consensus_queue.cc:1048] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:09.855612  1272 consensus_queue.cc:1048] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:09.884497  1251 consensus_queue.cc:1048] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:09.892256  1272 consensus_queue.cc:1048] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:09.900897  1272 consensus_queue.cc:1048] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:09.902333  1307 consensus_queue.cc:1048] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:09.937156  1307 consensus_queue.cc:1048] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:09.946540  1200 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7"
dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
mode: GRACEFUL
new_leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:46366
I20260709 12:59:09.947069  1200 raft_consensus.cc:606] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 LEADER]: Received request to transfer leadership to TS a6ec7e01e34945bb848cd646a3a171f8
I20260709 12:59:09.947319  1050 raft_consensus.cc:3060] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.953256  1276 consensus_queue.cc:1048] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:09.962914  1307 consensus_queue.cc:1048] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:09.967106  1289 consensus_queue.cc:1048] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:09.969055  1199 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1793808c703a4b5fba0a518e39f7c7c3"
dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
mode: GRACEFUL
new_leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:46366
I20260709 12:59:09.969741  1199 raft_consensus.cc:606] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 1 LEADER]: Received request to transfer leadership to TS a6ec7e01e34945bb848cd646a3a171f8
I20260709 12:59:09.973191  1199 raft_consensus.cc:3060] T c6e66356fcb14d2b825ab6666ac32d49 P c4958a1501dd46e4b8f154a28201eb7a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:09.986426  1249 consensus_queue.cc:1048] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:09.999995  1249 consensus_queue.cc:1048] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.017202  1051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "848e80771a8c48d287bc912ac84bf065"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
mode: GRACEFUL
new_leader_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
 from {username='slave'} at 127.0.0.1:41964
I20260709 12:59:10.017776  1051 raft_consensus.cc:606] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Received request to transfer leadership to TS a7d1e1a9de284c5ab09ed49f577557c3
I20260709 12:59:10.022423  1051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0dff6acdeff48a691083fdb124dfd20"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
mode: GRACEFUL
new_leader_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
 from {username='slave'} at 127.0.0.1:41964
I20260709 12:59:10.022861  1051 raft_consensus.cc:606] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Received request to transfer leadership to TS a7d1e1a9de284c5ab09ed49f577557c3
I20260709 12:59:10.027297  1051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "372642901bcb4c20a599bc5f223772f7"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
mode: GRACEFUL
new_leader_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
 from {username='slave'} at 127.0.0.1:41964
I20260709 12:59:10.027745  1051 raft_consensus.cc:606] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Received request to transfer leadership to TS c4958a1501dd46e4b8f154a28201eb7a
I20260709 12:59:10.132164  1249 consensus_queue.cc:1048] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.142330  1249 consensus_queue.cc:1048] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.165354  1289 consensus_queue.cc:1048] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.176095  1249 consensus_queue.cc:1048] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:10.184111  1249 raft_consensus.cc:993] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8: : Instructing follower a7d1e1a9de284c5ab09ed49f577557c3 to start an election
I20260709 12:59:10.184546  1276 raft_consensus.cc:1081] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Signalling peer a7d1e1a9de284c5ab09ed49f577557c3 to start an election
I20260709 12:59:10.186147  1126 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0dff6acdeff48a691083fdb124dfd20"
dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
 from {username='slave'} at 127.0.0.1:48642
I20260709 12:59:10.186765  1126 raft_consensus.cc:493] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:10.187150  1126 raft_consensus.cc:3060] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.191999  1126 raft_consensus.cc:515] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:10.193961  1126 leader_election.cc:290] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 2 election: Requested vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:10.194643  1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0dff6acdeff48a691083fdb124dfd20" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
I20260709 12:59:10.195210  1199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0dff6acdeff48a691083fdb124dfd20" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:10.195250  1051 raft_consensus.cc:3055] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:10.195895  1199 raft_consensus.cc:3060] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.195843  1051 raft_consensus.cc:740] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:10.196810  1051 consensus_queue.cc:260] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:10.198154  1051 raft_consensus.cc:3060] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.203130  1199 raft_consensus.cc:2468] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7d1e1a9de284c5ab09ed49f577557c3 in term 2.
I20260709 12:59:10.204449  1092 leader_election.cc:304] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:10.204712  1051 raft_consensus.cc:2468] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7d1e1a9de284c5ab09ed49f577557c3 in term 2.
I20260709 12:59:10.205240  1307 raft_consensus.cc:2804] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:10.205744  1307 raft_consensus.cc:697] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 LEADER]: Becoming Leader. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Running, Role: LEADER
I20260709 12:59:10.206377  1307 consensus_queue.cc:237] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:10.212600   949 catalog_manager.cc:5697] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 reported cstate change: term changed from 1 to 2, leader changed from a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193) to a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:10.223004  1276 consensus_queue.cc:1048] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.233866  1289 consensus_queue.cc:1048] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.241780  1249 raft_consensus.cc:993] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8: : Instructing follower a7d1e1a9de284c5ab09ed49f577557c3 to start an election
I20260709 12:59:10.242311  1289 raft_consensus.cc:1081] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Signalling peer a7d1e1a9de284c5ab09ed49f577557c3 to start an election
I20260709 12:59:10.243886  1126 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "848e80771a8c48d287bc912ac84bf065"
dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
 from {username='slave'} at 127.0.0.1:48642
I20260709 12:59:10.244509  1126 raft_consensus.cc:493] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:10.244901  1126 raft_consensus.cc:3060] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.249089  1289 consensus_queue.cc:1048] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.250615  1126 raft_consensus.cc:515] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:10.252663  1126 leader_election.cc:290] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 2 election: Requested vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:10.253805  1199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848e80771a8c48d287bc912ac84bf065" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:10.254418  1199 raft_consensus.cc:3060] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.254210  1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848e80771a8c48d287bc912ac84bf065" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
I20260709 12:59:10.255044  1051 raft_consensus.cc:3055] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:10.255482  1051 raft_consensus.cc:740] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:10.256531  1051 consensus_queue.cc:260] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:10.258584  1051 raft_consensus.cc:3060] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.260957  1199 raft_consensus.cc:2468] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7d1e1a9de284c5ab09ed49f577557c3 in term 2.
I20260709 12:59:10.262094  1092 leader_election.cc:304] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:10.262970  1307 raft_consensus.cc:2804] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:10.263490  1307 raft_consensus.cc:697] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 LEADER]: Becoming Leader. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Running, Role: LEADER
I20260709 12:59:10.264578  1307 consensus_queue.cc:237] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:10.270205  1249 consensus_queue.cc:1048] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:10.271724  1051 raft_consensus.cc:2468] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7d1e1a9de284c5ab09ed49f577557c3 in term 2.
I20260709 12:59:10.276077   949 catalog_manager.cc:5697] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 reported cstate change: term changed from 1 to 2, leader changed from a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193) to a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:10.282678  1276 raft_consensus.cc:993] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8: : Instructing follower c4958a1501dd46e4b8f154a28201eb7a to start an election
I20260709 12:59:10.283192  1249 raft_consensus.cc:1081] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Signalling peer c4958a1501dd46e4b8f154a28201eb7a to start an election
I20260709 12:59:10.284742  1199 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "372642901bcb4c20a599bc5f223772f7"
dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
 from {username='slave'} at 127.0.0.1:46354
I20260709 12:59:10.285312  1199 raft_consensus.cc:493] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:10.285728  1199 raft_consensus.cc:3060] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.292322  1199 raft_consensus.cc:515] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:10.294265  1199 leader_election.cc:290] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 2 election: Requested vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335)
I20260709 12:59:10.294966  1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372642901bcb4c20a599bc5f223772f7" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
I20260709 12:59:10.295557  1051 raft_consensus.cc:3055] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:10.295912  1051 raft_consensus.cc:740] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:10.296169  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "372642901bcb4c20a599bc5f223772f7" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:10.296790  1124 raft_consensus.cc:3060] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.296674  1051 consensus_queue.cc:260] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:10.297982  1051 raft_consensus.cc:3060] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.302287  1124 raft_consensus.cc:2468] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 2.
I20260709 12:59:10.303367  1165 leader_election.cc:304] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:10.303529  1051 raft_consensus.cc:2468] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 2.
I20260709 12:59:10.304176  1273 raft_consensus.cc:2804] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:10.304724  1273 raft_consensus.cc:697] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [term 2 LEADER]: Becoming Leader. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Running, Role: LEADER
I20260709 12:59:10.305529  1273 consensus_queue.cc:237] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:10.311762   946 catalog_manager.cc:5697] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a reported cstate change: term changed from 1 to 2, leader changed from a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193) to c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195). New cstate: current_term: 2 leader_uuid: "c4958a1501dd46e4b8f154a28201eb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:10.454766  1250 raft_consensus.cc:993] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a: : Instructing follower a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:10.455204  1273 raft_consensus.cc:1081] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 LEADER]: Signalling peer a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:10.456679  1051 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:41956
I20260709 12:59:10.457116  1051 raft_consensus.cc:493] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:10.457368  1051 raft_consensus.cc:3060] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.463819  1250 raft_consensus.cc:993] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a: : Instructing follower a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:10.464339  1250 raft_consensus.cc:1081] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 1 LEADER]: Signalling peer a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:10.464864  1051 raft_consensus.cc:515] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:10.465976  1050 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1793808c703a4b5fba0a518e39f7c7c3"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:41956
I20260709 12:59:10.466459  1050 raft_consensus.cc:493] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:10.466809  1050 raft_consensus.cc:3060] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.467787  1199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:10.467348  1051 leader_election.cc:290] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 2 election: Requested vote from peers c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315), a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335)
I20260709 12:59:10.468403  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:10.468405  1199 raft_consensus.cc:3055] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:10.469211  1124 raft_consensus.cc:3060] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.469592  1199 raft_consensus.cc:740] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Running, Role: LEADER
I20260709 12:59:10.470428  1199 consensus_queue.cc:260] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:10.471333  1199 raft_consensus.cc:3060] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.474231  1050 raft_consensus.cc:515] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:10.476084  1199 raft_consensus.cc:2468] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 2.
I20260709 12:59:10.476382  1124 raft_consensus.cc:2468] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 2.
I20260709 12:59:10.477270  1017 leader_election.cc:304] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:10.477174  1050 leader_election.cc:290] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 2 election: Requested vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:10.477846  1199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1793808c703a4b5fba0a518e39f7c7c3" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:10.478193  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1793808c703a4b5fba0a518e39f7c7c3" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:10.478477  1199 raft_consensus.cc:3055] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:10.478787  1126 raft_consensus.cc:3060] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.478892  1199 raft_consensus.cc:740] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Running, Role: LEADER
I20260709 12:59:10.478235  1249 raft_consensus.cc:2804] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:10.479595  1249 raft_consensus.cc:697] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Becoming Leader. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:10.479863  1199 consensus_queue.cc:260] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:10.480412  1249 consensus_queue.cc:237] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:10.480983  1199 raft_consensus.cc:3060] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:10.485991  1126 raft_consensus.cc:2468] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 2.
I20260709 12:59:10.487337  1199 raft_consensus.cc:2468] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 2.
I20260709 12:59:10.488641  1017 leader_election.cc:304] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:10.489462  1276 raft_consensus.cc:2804] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:10.489974  1276 raft_consensus.cc:697] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Becoming Leader. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:10.490797  1276 consensus_queue.cc:237] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:10.493752   946 catalog_manager.cc:5697] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 reported cstate change: term changed from 1 to 2, leader changed from c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195) to a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:10.505098   946 catalog_manager.cc:5697] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 reported cstate change: term changed from 1 to 2, leader changed from c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195) to a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:10.702544  1199 raft_consensus.cc:1275] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Refusing update from remote peer a7d1e1a9de284c5ab09ed49f577557c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:10.703783  1244 consensus_queue.cc:1048] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:10.710938  1050 raft_consensus.cc:1275] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Refusing update from remote peer a7d1e1a9de284c5ab09ed49f577557c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:10.712456  1251 consensus_queue.cc:1048] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:10.716408  1051 raft_consensus.cc:1275] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Refusing update from remote peer a7d1e1a9de284c5ab09ed49f577557c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:10.717914  1251 consensus_queue.cc:1048] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:10.726053  1199 raft_consensus.cc:1275] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Refusing update from remote peer a7d1e1a9de284c5ab09ed49f577557c3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:10.728116  1251 consensus_queue.cc:1048] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:10.758266  1050 raft_consensus.cc:1275] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Refusing update from remote peer c4958a1501dd46e4b8f154a28201eb7a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:10.759407  1250 consensus_queue.cc:1048] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:10.766705  1126 raft_consensus.cc:1275] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Refusing update from remote peer c4958a1501dd46e4b8f154a28201eb7a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:10.768242  1328 consensus_queue.cc:1048] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:10.911238  1126 raft_consensus.cc:1275] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:10.912732  1329 consensus_queue.cc:1048] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:10.921054  1199 raft_consensus.cc:1275] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:10.922758  1289 consensus_queue.cc:1048] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:11.038816  1199 raft_consensus.cc:1275] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:11.040153  1289 consensus_queue.cc:1048] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:11.048195  1126 raft_consensus.cc:1275] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:11.049818  1249 consensus_queue.cc:1048] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:13.059711   431 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 12:59:13.060708   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:13.077204  1050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
mode: GRACEFUL
new_leader_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
 from {username='slave'} at 127.0.0.1:55668
I20260709 12:59:13.077781  1050 raft_consensus.cc:606] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Received request to transfer leadership to TS c4958a1501dd46e4b8f154a28201eb7a
I20260709 12:59:13.080080  1050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
mode: GRACEFUL
new_leader_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
 from {username='slave'} at 127.0.0.1:55668
I20260709 12:59:13.080622  1050 raft_consensus.cc:606] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Received request to transfer leadership to TS c4958a1501dd46e4b8f154a28201eb7a
I20260709 12:59:13.081779   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:59:13.082208   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:13.268610  1345 raft_consensus.cc:993] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8: : Instructing follower c4958a1501dd46e4b8f154a28201eb7a to start an election
I20260709 12:59:13.269094  1289 raft_consensus.cc:1081] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Signalling peer c4958a1501dd46e4b8f154a28201eb7a to start an election
I20260709 12:59:13.270704  1200 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687"
dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
 from {username='slave'} at 127.0.0.1:46354
I20260709 12:59:13.271216  1200 raft_consensus.cc:493] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:13.271546  1200 raft_consensus.cc:3060] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:13.335819  1345 raft_consensus.cc:993] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8: : Instructing follower c4958a1501dd46e4b8f154a28201eb7a to start an election
I20260709 12:59:13.336287  1345 raft_consensus.cc:1081] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Signalling peer c4958a1501dd46e4b8f154a28201eb7a to start an election
I20260709 12:59:13.337791  1199 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7"
dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
 from {username='slave'} at 127.0.0.1:46354
I20260709 12:59:13.338253  1199 raft_consensus.cc:493] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:13.338513  1199 raft_consensus.cc:3060] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:13.346736  1200 raft_consensus.cc:515] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:13.351346  1200 leader_election.cc:290] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 2 election: Requested vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335)
I20260709 12:59:13.352185  1199 raft_consensus.cc:515] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:13.352830  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:13.353376  1124 raft_consensus.cc:3060] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:13.353332  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
I20260709 12:59:13.354039  1050 raft_consensus.cc:3055] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:13.354351  1050 raft_consensus.cc:740] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:13.354846  1199 leader_election.cc:290] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 3 election: Requested vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129)
I20260709 12:59:13.355186  1050 consensus_queue.cc:260] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:13.356467  1050 raft_consensus.cc:3060] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:13.357015  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:13.357129  1051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
I20260709 12:59:13.357580  1126 raft_consensus.cc:3060] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:13.357635  1051 raft_consensus.cc:3055] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:13.358028  1051 raft_consensus.cc:740] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:13.358796  1051 consensus_queue.cc:260] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:13.359876  1051 raft_consensus.cc:3060] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:13.448112  1051 raft_consensus.cc:2468] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 3.
I20260709 12:59:13.448397  1126 raft_consensus.cc:2468] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 3.
I20260709 12:59:13.448081  1050 raft_consensus.cc:2468] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 2.
I20260709 12:59:13.448073  1124 raft_consensus.cc:2468] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 2.
I20260709 12:59:13.449812  1164 leader_election.cc:304] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:13.450655  1165 leader_election.cc:304] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:13.451643  1341 raft_consensus.cc:2804] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:13.453904  1346 raft_consensus.cc:2804] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:13.455338  1341 raft_consensus.cc:697] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 3 LEADER]: Becoming Leader. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Running, Role: LEADER
I20260709 12:59:13.456497  1346 raft_consensus.cc:697] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 2 LEADER]: Becoming Leader. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Running, Role: LEADER
I20260709 12:59:13.456301  1341 consensus_queue.cc:237] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:13.457326  1346 consensus_queue.cc:237] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:13.465945   949 catalog_manager.cc:5697] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a reported cstate change: term changed from 2 to 3, leader changed from a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193) to c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195). New cstate: current_term: 3 leader_uuid: "c4958a1501dd46e4b8f154a28201eb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:13.467118   949 catalog_manager.cc:5697] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a reported cstate change: term changed from 1 to 2, leader changed from a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193) to c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195). New cstate: current_term: 2 leader_uuid: "c4958a1501dd46e4b8f154a28201eb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:13.872756  1050 raft_consensus.cc:1275] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Refusing update from remote peer c4958a1501dd46e4b8f154a28201eb7a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:13.873993  1346 consensus_queue.cc:1048] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:13.883548  1126 raft_consensus.cc:1275] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Refusing update from remote peer c4958a1501dd46e4b8f154a28201eb7a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:13.885053  1341 consensus_queue.cc:1048] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:14.028788  1126 raft_consensus.cc:1275] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Refusing update from remote peer c4958a1501dd46e4b8f154a28201eb7a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:14.030054  1341 consensus_queue.cc:1048] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:14.033051  1050 raft_consensus.cc:1275] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Refusing update from remote peer c4958a1501dd46e4b8f154a28201eb7a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:14.034380  1341 consensus_queue.cc:1048] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:14.082942   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:14.094049  1199 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7"
dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
mode: GRACEFUL
new_leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:46366
I20260709 12:59:14.094602  1199 raft_consensus.cc:606] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 3 LEADER]: Received request to transfer leadership to TS a6ec7e01e34945bb848cd646a3a171f8
I20260709 12:59:14.099557  1199 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687"
dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
mode: GRACEFUL
new_leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:46366
I20260709 12:59:14.100117  1199 raft_consensus.cc:606] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 2 LEADER]: Received request to transfer leadership to TS a6ec7e01e34945bb848cd646a3a171f8
I20260709 12:59:14.127030  1126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27555eb30a6241aca84fe4537aa8e33a"
dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
mode: GRACEFUL
new_leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:40078
I20260709 12:59:14.127552  1126 raft_consensus.cc:606] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 LEADER]: Received request to transfer leadership to TS a6ec7e01e34945bb848cd646a3a171f8
I20260709 12:59:14.132459  1126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7"
dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
mode: GRACEFUL
new_leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:40078
I20260709 12:59:14.132936  1126 raft_consensus.cc:606] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 LEADER]: Received request to transfer leadership to TS a6ec7e01e34945bb848cd646a3a171f8
I20260709 12:59:14.138156  1126 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "848e80771a8c48d287bc912ac84bf065"
dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
mode: GRACEFUL
new_leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:40078
I20260709 12:59:14.138619  1126 raft_consensus.cc:606] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 LEADER]: Received request to transfer leadership to TS a6ec7e01e34945bb848cd646a3a171f8
I20260709 12:59:14.325505  1365 raft_consensus.cc:993] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3: : Instructing follower a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:14.327234  1338 raft_consensus.cc:1081] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 LEADER]: Signalling peer a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:14.329113  1049 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "848e80771a8c48d287bc912ac84bf065"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:41944
I20260709 12:59:14.329654  1049 raft_consensus.cc:493] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:14.330044  1049 raft_consensus.cc:3060] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:14.330067  1341 raft_consensus.cc:993] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a: : Instructing follower a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:14.330503  1346 raft_consensus.cc:1081] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 3 LEADER]: Signalling peer a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:14.331019  1366 raft_consensus.cc:993] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3: : Instructing follower a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:14.331482  1338 raft_consensus.cc:1081] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 LEADER]: Signalling peer a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:14.332213  1051 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:41956
I20260709 12:59:14.332747  1051 raft_consensus.cc:493] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:14.333055  1051 raft_consensus.cc:3060] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:14.333055  1050 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:41944
I20260709 12:59:14.333812  1050 raft_consensus.cc:493] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:14.334602  1050 raft_consensus.cc:3060] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:14.337535  1049 raft_consensus.cc:515] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:14.337762  1051 raft_consensus.cc:515] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:14.339787  1049 leader_election.cc:290] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 3 election: Requested vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:14.341046  1051 leader_election.cc:290] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 4 election: Requested vote from peers c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315), a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335)
I20260709 12:59:14.341811  1200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848e80771a8c48d287bc912ac84bf065" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:14.341673  1050 raft_consensus.cc:515] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:14.342136  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:14.342456  1200 raft_consensus.cc:3060] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:14.342681  1124 raft_consensus.cc:3060] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:14.341399  1199 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0033bb4a59084e28ac4003e6845c1ae7" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:14.343297  1199 raft_consensus.cc:3055] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:14.343608  1199 raft_consensus.cc:740] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Running, Role: LEADER
I20260709 12:59:14.343773  1050 leader_election.cc:290] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 2 election: Requested vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:14.344391  1199 consensus_queue.cc:260] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:14.345645  1199 raft_consensus.cc:3060] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:14.348289  1124 raft_consensus.cc:2468] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 4.
I20260709 12:59:14.349279  1126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "848e80771a8c48d287bc912ac84bf065" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:14.349876  1126 raft_consensus.cc:3055] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:14.351162  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:14.351263  1200 raft_consensus.cc:2468] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 3.
I20260709 12:59:14.351472  1126 raft_consensus.cc:740] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Running, Role: LEADER
I20260709 12:59:14.351760  1198 raft_consensus.cc:3060] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:14.352285  1126 consensus_queue.cc:260] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:14.352556  1017 leader_election.cc:304] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:14.353173  1199 raft_consensus.cc:2468] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 4.
I20260709 12:59:14.350656  1016 leader_election.cc:304] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, a7d1e1a9de284c5ab09ed49f577557c3; no voters: 
I20260709 12:59:14.353505  1360 raft_consensus.cc:2804] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:14.350466  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:14.354087  1345 raft_consensus.cc:2804] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:14.354236  1124 raft_consensus.cc:3055] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:14.354648  1124 raft_consensus.cc:740] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Running, Role: LEADER
I20260709 12:59:14.355865  1124 consensus_queue.cc:260] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:14.357117  1124 raft_consensus.cc:3060] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:14.357573  1126 raft_consensus.cc:3060] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:14.359151  1198 raft_consensus.cc:2468] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 2.
I20260709 12:59:14.360601  1017 leader_election.cc:304] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:14.367295  1345 raft_consensus.cc:697] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 4 LEADER]: Becoming Leader. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:14.368623  1126 raft_consensus.cc:2468] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 3.
I20260709 12:59:14.368739  1360 raft_consensus.cc:697] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 LEADER]: Becoming Leader. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:14.369302  1345 consensus_queue.cc:237] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } }
I20260709 12:59:14.370378  1360 consensus_queue.cc:237] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:14.369786  1368 raft_consensus.cc:2804] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:14.373287  1368 raft_consensus.cc:697] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Becoming Leader. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:14.374465  1368 consensus_queue.cc:237] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:14.376943  1124 raft_consensus.cc:2468] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 2.
I20260709 12:59:14.393267   948 catalog_manager.cc:5697] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 reported cstate change: term changed from 3 to 4, leader changed from c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195) to a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:14.394690   948 catalog_manager.cc:5697] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 reported cstate change: term changed from 2 to 3, leader changed from a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194) to a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:14.411406   948 catalog_manager.cc:5697] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 reported cstate change: term changed from 1 to 2, leader changed from a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194) to a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:14.536821  1341 raft_consensus.cc:993] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a: : Instructing follower a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:14.537460  1346 raft_consensus.cc:1081] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 2 LEADER]: Signalling peer a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:14.539238  1050 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:41956
I20260709 12:59:14.539675  1050 raft_consensus.cc:493] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:14.539919  1050 raft_consensus.cc:3060] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:14.544516  1050 raft_consensus.cc:515] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:14.546370  1050 leader_election.cc:290] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 3 election: Requested vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:14.547214  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:14.547495  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:14.547777  1124 raft_consensus.cc:3060] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:14.548134  1198 raft_consensus.cc:3055] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:14.548451  1198 raft_consensus.cc:740] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Running, Role: LEADER
I20260709 12:59:14.549299  1198 consensus_queue.cc:260] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:14.550367  1198 raft_consensus.cc:3060] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:14.553602  1124 raft_consensus.cc:2468] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 3.
I20260709 12:59:14.554980  1016 leader_election.cc:304] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, a7d1e1a9de284c5ab09ed49f577557c3; no voters: 
I20260709 12:59:14.555688  1360 raft_consensus.cc:2804] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:14.556105  1360 raft_consensus.cc:697] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 LEADER]: Becoming Leader. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:14.556483  1198 raft_consensus.cc:2468] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 3.
I20260709 12:59:14.556870  1360 consensus_queue.cc:237] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:14.562896   948 catalog_manager.cc:5697] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 reported cstate change: term changed from 2 to 3, leader changed from c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195) to a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:14.682367  1366 raft_consensus.cc:993] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3: : Instructing follower a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:14.682799  1366 raft_consensus.cc:1081] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 LEADER]: Signalling peer a6ec7e01e34945bb848cd646a3a171f8 to start an election
I20260709 12:59:14.684226  1050 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "27555eb30a6241aca84fe4537aa8e33a"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
 from {username='slave'} at 127.0.0.1:41944
I20260709 12:59:14.684701  1050 raft_consensus.cc:493] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:14.684958  1050 raft_consensus.cc:3060] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:14.689741  1050 raft_consensus.cc:515] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:14.691572  1050 leader_election.cc:290] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 2 election: Requested vote from peers a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:14.692399  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27555eb30a6241aca84fe4537aa8e33a" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:14.692644  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27555eb30a6241aca84fe4537aa8e33a" candidate_uuid: "a6ec7e01e34945bb848cd646a3a171f8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:14.693076  1124 raft_consensus.cc:3055] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:14.693231  1198 raft_consensus.cc:3060] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:14.693569  1124 raft_consensus.cc:740] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Running, Role: LEADER
I20260709 12:59:14.694394  1124 consensus_queue.cc:260] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:14.695364  1124 raft_consensus.cc:3060] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:14.698570  1198 raft_consensus.cc:2468] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 2.
I20260709 12:59:14.699926  1017 leader_election.cc:304] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6ec7e01e34945bb848cd646a3a171f8, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:14.700489  1124 raft_consensus.cc:2468] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6ec7e01e34945bb848cd646a3a171f8 in term 2.
I20260709 12:59:14.700747  1360 raft_consensus.cc:2804] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:14.701283  1360 raft_consensus.cc:697] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Becoming Leader. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:14.702152  1360 consensus_queue.cc:237] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:14.708894   948 catalog_manager.cc:5697] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 reported cstate change: term changed from 1 to 2, leader changed from a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194) to a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "a6ec7e01e34945bb848cd646a3a171f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:14.782270  1124 raft_consensus.cc:1275] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:14.783560  1360 consensus_queue.cc:1048] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:14.788636  1198 raft_consensus.cc:1275] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:14.790544  1345 consensus_queue.cc:1048] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:14.823359  1124 raft_consensus.cc:1275] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 4 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:14.824745  1368 consensus_queue.cc:1048] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:14.832763  1198 raft_consensus.cc:1275] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 4 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:14.834614  1345 consensus_queue.cc:1048] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:14.917110  1198 raft_consensus.cc:1275] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:14.918658  1360 consensus_queue.cc:1048] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:14.927913  1124 raft_consensus.cc:1275] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:14.929520  1345 consensus_queue.cc:1048] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:15.007952  1198 raft_consensus.cc:1275] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:15.009286  1345 consensus_queue.cc:1048] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:15.017001  1124 raft_consensus.cc:1275] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:15.018489  1345 consensus_queue.cc:1048] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:15.245038  1124 raft_consensus.cc:1275] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:15.246323  1345 consensus_queue.cc:1048] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:15.255339  1198 raft_consensus.cc:1275] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Refusing update from remote peer a6ec7e01e34945bb848cd646a3a171f8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:15.257227  1360 consensus_queue.cc:1048] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:17.146252   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:17.154219  1050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
mode: GRACEFUL
new_leader_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
 from {username='slave'} at 127.0.0.1:55668
I20260709 12:59:17.154894  1050 raft_consensus.cc:606] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Received request to transfer leadership to TS c4958a1501dd46e4b8f154a28201eb7a
I20260709 12:59:17.157580  1050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1793808c703a4b5fba0a518e39f7c7c3"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
mode: GRACEFUL
new_leader_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
 from {username='slave'} at 127.0.0.1:55668
I20260709 12:59:17.158028  1050 raft_consensus.cc:606] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Received request to transfer leadership to TS a7d1e1a9de284c5ab09ed49f577557c3
I20260709 12:59:17.160364  1050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27555eb30a6241aca84fe4537aa8e33a"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
mode: GRACEFUL
new_leader_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
 from {username='slave'} at 127.0.0.1:55668
I20260709 12:59:17.160844  1050 raft_consensus.cc:606] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Received request to transfer leadership to TS a7d1e1a9de284c5ab09ed49f577557c3
I20260709 12:59:17.163164  1050 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687"
dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
mode: GRACEFUL
new_leader_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
 from {username='slave'} at 127.0.0.1:55668
I20260709 12:59:17.163596  1050 raft_consensus.cc:606] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 LEADER]: Received request to transfer leadership to TS c4958a1501dd46e4b8f154a28201eb7a
I20260709 12:59:17.164543   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:59:17.164992   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:17.289324  1386 raft_consensus.cc:993] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8: : Instructing follower a7d1e1a9de284c5ab09ed49f577557c3 to start an election
I20260709 12:59:17.289872  1368 raft_consensus.cc:1081] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Signalling peer a7d1e1a9de284c5ab09ed49f577557c3 to start an election
I20260709 12:59:17.291466  1124 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1793808c703a4b5fba0a518e39f7c7c3"
dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
 from {username='slave'} at 127.0.0.1:48642
I20260709 12:59:17.291976  1124 raft_consensus.cc:493] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:17.292312  1124 raft_consensus.cc:3060] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:17.299211  1124 raft_consensus.cc:515] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:17.301137  1124 leader_election.cc:290] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 3 election: Requested vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:17.302117  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1793808c703a4b5fba0a518e39f7c7c3" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
I20260709 12:59:17.302409  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1793808c703a4b5fba0a518e39f7c7c3" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:17.302745  1050 raft_consensus.cc:3055] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:17.303013  1198 raft_consensus.cc:3060] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:17.303164  1050 raft_consensus.cc:740] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:17.304049  1050 consensus_queue.cc:260] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:17.306308  1050 raft_consensus.cc:3060] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:17.309558  1198 raft_consensus.cc:2468] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7d1e1a9de284c5ab09ed49f577557c3 in term 3.
I20260709 12:59:17.310462  1386 raft_consensus.cc:993] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8: : Instructing follower a7d1e1a9de284c5ab09ed49f577557c3 to start an election
I20260709 12:59:17.310923  1386 raft_consensus.cc:1081] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Signalling peer a7d1e1a9de284c5ab09ed49f577557c3 to start an election
I20260709 12:59:17.310956  1092 leader_election.cc:304] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:17.312062  1366 raft_consensus.cc:2804] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:17.312436  1124 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "27555eb30a6241aca84fe4537aa8e33a"
dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
 from {username='slave'} at 127.0.0.1:48642
I20260709 12:59:17.312639  1050 raft_consensus.cc:2468] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7d1e1a9de284c5ab09ed49f577557c3 in term 3.
I20260709 12:59:17.312871  1124 raft_consensus.cc:493] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:17.313787  1366 raft_consensus.cc:697] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 LEADER]: Becoming Leader. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Running, Role: LEADER
I20260709 12:59:17.313954  1124 raft_consensus.cc:3060] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:17.314916  1366 consensus_queue.cc:237] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:17.320183  1124 raft_consensus.cc:515] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:17.322001  1124 leader_election.cc:290] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 3 election: Requested vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195:40315)
I20260709 12:59:17.322643  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27555eb30a6241aca84fe4537aa8e33a" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
I20260709 12:59:17.323226  1050 raft_consensus.cc:3055] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:17.323112  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27555eb30a6241aca84fe4537aa8e33a" candidate_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
I20260709 12:59:17.323606  1050 raft_consensus.cc:740] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:17.323803  1198 raft_consensus.cc:3060] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:17.323508   949 catalog_manager.cc:5697] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 reported cstate change: term changed from 2 to 3, leader changed from a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193) to a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:17.324633  1050 consensus_queue.cc:260] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:17.325728  1050 raft_consensus.cc:3060] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:17.330832  1198 raft_consensus.cc:2468] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7d1e1a9de284c5ab09ed49f577557c3 in term 3.
I20260709 12:59:17.332525  1092 leader_election.cc:304] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:17.333405  1050 raft_consensus.cc:2468] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7d1e1a9de284c5ab09ed49f577557c3 in term 3.
I20260709 12:59:17.333562  1366 raft_consensus.cc:2804] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:17.334524  1366 raft_consensus.cc:697] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 LEADER]: Becoming Leader. State: Replica: a7d1e1a9de284c5ab09ed49f577557c3, State: Running, Role: LEADER
I20260709 12:59:17.335544  1366 consensus_queue.cc:237] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:17.344799   949 catalog_manager.cc:5697] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 reported cstate change: term changed from 2 to 3, leader changed from a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193) to a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:17.348762  1368 raft_consensus.cc:993] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8: : Instructing follower c4958a1501dd46e4b8f154a28201eb7a to start an election
I20260709 12:59:17.349269  1386 raft_consensus.cc:1081] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Signalling peer c4958a1501dd46e4b8f154a28201eb7a to start an election
I20260709 12:59:17.350874  1198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7"
dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
 from {username='slave'} at 127.0.0.1:46354
I20260709 12:59:17.351438  1198 raft_consensus.cc:493] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:17.351828  1198 raft_consensus.cc:3060] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:17.356801  1198 raft_consensus.cc:515] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:17.358633  1198 leader_election.cc:290] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 3 election: Requested vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335)
I20260709 12:59:17.359452  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
I20260709 12:59:17.359761  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "089f3cb6025f429e911a1f7f04eae9a7" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:17.360021  1050 raft_consensus.cc:3055] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:17.360275  1124 raft_consensus.cc:3060] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:17.360349  1050 raft_consensus.cc:740] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:17.361162  1050 consensus_queue.cc:260] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:17.362092  1050 raft_consensus.cc:3060] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:17.364681  1124 raft_consensus.cc:2468] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 3.
I20260709 12:59:17.365864  1165 leader_election.cc:304] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:17.366544  1050 raft_consensus.cc:2468] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 3.
I20260709 12:59:17.366708  1346 raft_consensus.cc:2804] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:17.368105  1346 raft_consensus.cc:697] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 3 LEADER]: Becoming Leader. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Running, Role: LEADER
I20260709 12:59:17.368909  1346 consensus_queue.cc:237] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:17.375995   949 catalog_manager.cc:5697] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a reported cstate change: term changed from 2 to 3, leader changed from a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193) to c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195). New cstate: current_term: 3 leader_uuid: "c4958a1501dd46e4b8f154a28201eb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:17.475860  1386 raft_consensus.cc:993] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8: : Instructing follower c4958a1501dd46e4b8f154a28201eb7a to start an election
I20260709 12:59:17.476318  1368 raft_consensus.cc:1081] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 LEADER]: Signalling peer c4958a1501dd46e4b8f154a28201eb7a to start an election
I20260709 12:59:17.477828  1198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687"
dest_uuid: "c4958a1501dd46e4b8f154a28201eb7a"
 from {username='slave'} at 127.0.0.1:46354
I20260709 12:59:17.478266  1198 raft_consensus.cc:493] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:17.478547  1198 raft_consensus.cc:3060] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:17.483059  1198 raft_consensus.cc:515] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:17.484755  1198 leader_election.cc:290] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 4 election: Requested vote from peers a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129), a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335)
I20260709 12:59:17.485543  1050 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a6ec7e01e34945bb848cd646a3a171f8"
I20260709 12:59:17.485901  1124 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5057bc9d1ba6441f8fcd0302355b2687" candidate_uuid: "c4958a1501dd46e4b8f154a28201eb7a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a7d1e1a9de284c5ab09ed49f577557c3"
I20260709 12:59:17.486080  1050 raft_consensus.cc:3055] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:59:17.486387  1124 raft_consensus.cc:3060] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:17.486413  1050 raft_consensus.cc:740] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a6ec7e01e34945bb848cd646a3a171f8, State: Running, Role: LEADER
I20260709 12:59:17.487293  1050 consensus_queue.cc:260] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:17.488147  1050 raft_consensus.cc:3060] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:59:17.490947  1124 raft_consensus.cc:2468] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 4.
I20260709 12:59:17.491925  1165 leader_election.cc:304] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7d1e1a9de284c5ab09ed49f577557c3, c4958a1501dd46e4b8f154a28201eb7a; no voters: 
I20260709 12:59:17.492686  1346 raft_consensus.cc:2804] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:59:17.493002  1050 raft_consensus.cc:2468] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4958a1501dd46e4b8f154a28201eb7a in term 4.
I20260709 12:59:17.493279  1346 raft_consensus.cc:697] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 4 LEADER]: Becoming Leader. State: Replica: c4958a1501dd46e4b8f154a28201eb7a, State: Running, Role: LEADER
I20260709 12:59:17.494249  1346 consensus_queue.cc:237] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } }
I20260709 12:59:17.501709   949 catalog_manager.cc:5697] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a reported cstate change: term changed from 3 to 4, leader changed from a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193) to c4958a1501dd46e4b8f154a28201eb7a (127.0.107.195). New cstate: current_term: 4 leader_uuid: "c4958a1501dd46e4b8f154a28201eb7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:17.770802  1198 raft_consensus.cc:1275] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Refusing update from remote peer a7d1e1a9de284c5ab09ed49f577557c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:17.772298  1366 consensus_queue.cc:1048] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:17.783129  1050 raft_consensus.cc:1275] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Refusing update from remote peer a7d1e1a9de284c5ab09ed49f577557c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:17.784796  1387 consensus_queue.cc:1048] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:17.817591  1050 raft_consensus.cc:1275] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Refusing update from remote peer a7d1e1a9de284c5ab09ed49f577557c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:17.818734  1387 consensus_queue.cc:1048] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:17.827270  1198 raft_consensus.cc:1275] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Refusing update from remote peer a7d1e1a9de284c5ab09ed49f577557c3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:17.829404  1366 consensus_queue.cc:1048] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4958a1501dd46e4b8f154a28201eb7a" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 40315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:17.853269  1124 raft_consensus.cc:1275] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Refusing update from remote peer c4958a1501dd46e4b8f154a28201eb7a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:17.854488  1346 consensus_queue.cc:1048] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:17.864198  1050 raft_consensus.cc:1275] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Refusing update from remote peer c4958a1501dd46e4b8f154a28201eb7a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:17.865998  1346 consensus_queue.cc:1048] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:17.988005  1050 raft_consensus.cc:1275] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 4 FOLLOWER]: Refusing update from remote peer c4958a1501dd46e4b8f154a28201eb7a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:17.989177  1407 consensus_queue.cc:1048] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6ec7e01e34945bb848cd646a3a171f8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:17.999209  1124 raft_consensus.cc:1275] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 4 FOLLOWER]: Refusing update from remote peer c4958a1501dd46e4b8f154a28201eb7a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:59:18.000664  1394 consensus_queue.cc:1048] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7d1e1a9de284c5ab09ed49f577557c3" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:18.165886   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:18.172106   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 12:59:18.196103   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:18.196892   431 tablet_replica.cc:333] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8: stopping tablet replica
I20260709 12:59:18.197592   431 raft_consensus.cc:2243] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.198057   431 raft_consensus.cc:2272] T 27555eb30a6241aca84fe4537aa8e33a P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.200474   431 tablet_replica.cc:333] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8: stopping tablet replica
I20260709 12:59:18.201009   431 raft_consensus.cc:2243] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.201540   431 raft_consensus.cc:2272] T 372642901bcb4c20a599bc5f223772f7 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.203241   431 tablet_replica.cc:333] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8: stopping tablet replica
I20260709 12:59:18.203783   431 raft_consensus.cc:2243] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.204655   431 raft_consensus.cc:2272] T 0033bb4a59084e28ac4003e6845c1ae7 P a6ec7e01e34945bb848cd646a3a171f8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.206489   431 tablet_replica.cc:333] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8: stopping tablet replica
I20260709 12:59:18.206981   431 raft_consensus.cc:2243] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.207401   431 raft_consensus.cc:2272] T a0dff6acdeff48a691083fdb124dfd20 P a6ec7e01e34945bb848cd646a3a171f8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.209075   431 tablet_replica.cc:333] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8: stopping tablet replica
I20260709 12:59:18.209666   431 raft_consensus.cc:2243] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.210064   431 raft_consensus.cc:2272] T 089f3cb6025f429e911a1f7f04eae9a7 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.212477   431 tablet_replica.cc:333] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8: stopping tablet replica
I20260709 12:59:18.213014   431 raft_consensus.cc:2243] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.213461   431 raft_consensus.cc:2272] T 5057bc9d1ba6441f8fcd0302355b2687 P a6ec7e01e34945bb848cd646a3a171f8 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.215637   431 tablet_replica.cc:333] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8: stopping tablet replica
I20260709 12:59:18.216154   431 raft_consensus.cc:2243] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.216897   431 raft_consensus.cc:2272] T 848e80771a8c48d287bc912ac84bf065 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.218590   431 tablet_replica.cc:333] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8: stopping tablet replica
I20260709 12:59:18.219092   431 raft_consensus.cc:2243] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.220011   431 raft_consensus.cc:2272] T 086f7e48b7d44aabb6a2cf053f3afb95 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.222389   431 tablet_replica.cc:333] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8: stopping tablet replica
I20260709 12:59:18.223099   431 raft_consensus.cc:2243] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.223595   431 raft_consensus.cc:2272] T 1793808c703a4b5fba0a518e39f7c7c3 P a6ec7e01e34945bb848cd646a3a171f8 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:18.224395  1090 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
I20260709 12:59:18.226223   431 tablet_replica.cc:333] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8: stopping tablet replica
I20260709 12:59:18.226716   431 raft_consensus.cc:2243] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.227458   431 raft_consensus.cc:2272] T c6e66356fcb14d2b825ab6666ac32d49 P a6ec7e01e34945bb848cd646a3a171f8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:18.227458  1090 consensus_peers.cc:597] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 -> Peer a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129): Couldn't send request to peer a6ec7e01e34945bb848cd646a3a171f8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:18.257113   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 12:59:18.278471   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 12:59:18.289460  1164 consensus_peers.cc:597] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a -> Peer a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129): Couldn't send request to peer a6ec7e01e34945bb848cd646a3a171f8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46129: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:18.300032  1165 consensus_peers.cc:597] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a -> Peer a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335): Couldn't send request to peer a7d1e1a9de284c5ab09ed49f577557c3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:18.307301  1090 consensus_peers.cc:597] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 -> Peer a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129): Couldn't send request to peer a6ec7e01e34945bb848cd646a3a171f8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46129: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:18.317870   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:18.318545   431 tablet_replica.cc:333] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3: stopping tablet replica
I20260709 12:59:18.319249   431 raft_consensus.cc:2243] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.319777   431 raft_consensus.cc:2272] T 372642901bcb4c20a599bc5f223772f7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.321727   431 tablet_replica.cc:333] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3: stopping tablet replica
I20260709 12:59:18.322283   431 raft_consensus.cc:2243] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.323038   431 raft_consensus.cc:2272] T 27555eb30a6241aca84fe4537aa8e33a P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.325240   431 tablet_replica.cc:333] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3: stopping tablet replica
I20260709 12:59:18.325851   431 raft_consensus.cc:2243] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.326282   431 raft_consensus.cc:2272] T 0033bb4a59084e28ac4003e6845c1ae7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.328002   431 tablet_replica.cc:333] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3: stopping tablet replica
I20260709 12:59:18.328486   431 raft_consensus.cc:2243] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.328881   431 raft_consensus.cc:2272] T c6e66356fcb14d2b825ab6666ac32d49 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.330637   431 tablet_replica.cc:333] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3: stopping tablet replica
I20260709 12:59:18.331146   431 raft_consensus.cc:2243] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.331864   431 raft_consensus.cc:2272] T 1793808c703a4b5fba0a518e39f7c7c3 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.333945   431 tablet_replica.cc:333] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3: stopping tablet replica
I20260709 12:59:18.334466   431 raft_consensus.cc:2243] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.334909   431 raft_consensus.cc:2272] T 848e80771a8c48d287bc912ac84bf065 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.338078   431 tablet_replica.cc:333] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3: stopping tablet replica
I20260709 12:59:18.338693   431 raft_consensus.cc:2243] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.339212   431 raft_consensus.cc:2272] T 086f7e48b7d44aabb6a2cf053f3afb95 P a7d1e1a9de284c5ab09ed49f577557c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.341764   431 tablet_replica.cc:333] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3: stopping tablet replica
I20260709 12:59:18.342621   431 raft_consensus.cc:2243] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:59:18.343346  1164 consensus_peers.cc:597] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a -> Peer a6ec7e01e34945bb848cd646a3a171f8 (127.0.107.193:46129): Couldn't send request to peer a6ec7e01e34945bb848cd646a3a171f8. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46129: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:18.343705   431 raft_consensus.cc:2272] T a0dff6acdeff48a691083fdb124dfd20 P a7d1e1a9de284c5ab09ed49f577557c3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.346089   431 tablet_replica.cc:333] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3: stopping tablet replica
I20260709 12:59:18.346537   431 raft_consensus.cc:2243] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.346905   431 raft_consensus.cc:2272] T 089f3cb6025f429e911a1f7f04eae9a7 P a7d1e1a9de284c5ab09ed49f577557c3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.348935   431 tablet_replica.cc:333] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3: stopping tablet replica
I20260709 12:59:18.349385   431 raft_consensus.cc:2243] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.349839   431 raft_consensus.cc:2272] T 5057bc9d1ba6441f8fcd0302355b2687 P a7d1e1a9de284c5ab09ed49f577557c3 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.379065   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 12:59:18.397086   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 12:59:18.420769   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:59:18.421411   431 tablet_replica.cc:333] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a: stopping tablet replica
I20260709 12:59:18.422094   431 raft_consensus.cc:2243] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.422928   431 raft_consensus.cc:2272] T 372642901bcb4c20a599bc5f223772f7 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.424945   431 tablet_replica.cc:333] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a: stopping tablet replica
I20260709 12:59:18.425644   431 raft_consensus.cc:2243] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.426199   431 raft_consensus.cc:2272] T 848e80771a8c48d287bc912ac84bf065 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.428552   431 tablet_replica.cc:333] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a: stopping tablet replica
I20260709 12:59:18.429384   431 raft_consensus.cc:2243] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.429996   431 raft_consensus.cc:2272] T 086f7e48b7d44aabb6a2cf053f3afb95 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:18.430747  1165 consensus_peers.cc:597] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a -> Peer a7d1e1a9de284c5ab09ed49f577557c3 (127.0.107.194:39335): Couldn't send request to peer a7d1e1a9de284c5ab09ed49f577557c3. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:39335: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:18.432545   431 tablet_replica.cc:333] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a: stopping tablet replica
I20260709 12:59:18.433327   431 raft_consensus.cc:2243] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.433873   431 raft_consensus.cc:2272] T 27555eb30a6241aca84fe4537aa8e33a P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.436414   431 tablet_replica.cc:333] T c6e66356fcb14d2b825ab6666ac32d49 P c4958a1501dd46e4b8f154a28201eb7a: stopping tablet replica
I20260709 12:59:18.437175   431 raft_consensus.cc:2243] T c6e66356fcb14d2b825ab6666ac32d49 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.437722   431 raft_consensus.cc:2272] T c6e66356fcb14d2b825ab6666ac32d49 P c4958a1501dd46e4b8f154a28201eb7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.439780   431 tablet_replica.cc:333] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a: stopping tablet replica
I20260709 12:59:18.440246   431 raft_consensus.cc:2243] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.440635   431 raft_consensus.cc:2272] T 1793808c703a4b5fba0a518e39f7c7c3 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.442746   431 tablet_replica.cc:333] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a: stopping tablet replica
I20260709 12:59:18.443222   431 raft_consensus.cc:2243] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.443589   431 raft_consensus.cc:2272] T a0dff6acdeff48a691083fdb124dfd20 P c4958a1501dd46e4b8f154a28201eb7a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.445040   431 tablet_replica.cc:333] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a: stopping tablet replica
I20260709 12:59:18.445538   431 raft_consensus.cc:2243] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.446205   431 raft_consensus.cc:2272] T 5057bc9d1ba6441f8fcd0302355b2687 P c4958a1501dd46e4b8f154a28201eb7a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.448001   431 tablet_replica.cc:333] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a: stopping tablet replica
I20260709 12:59:18.448439   431 raft_consensus.cc:2243] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.449074   431 raft_consensus.cc:2272] T 089f3cb6025f429e911a1f7f04eae9a7 P c4958a1501dd46e4b8f154a28201eb7a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.450887   431 tablet_replica.cc:333] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a: stopping tablet replica
I20260709 12:59:18.451303   431 raft_consensus.cc:2243] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:18.451656   431 raft_consensus.cc:2272] T 0033bb4a59084e28ac4003e6845c1ae7 P c4958a1501dd46e4b8f154a28201eb7a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.479033   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 12:59:18.496655   431 master.cc:562] Master@127.0.107.254:36925 shutting down...
I20260709 12:59:18.517463   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:59:18.518052   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:18.518389   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0195911216294f3f843d11618de0e904: stopping tablet replica
I20260709 12:59:18.537774   431 master.cc:584] Master@127.0.107.254:36925 shutdown complete.
a6ec7e01e34945bb848cd646a3a171f8  5
a7d1e1a9de284c5ab09ed49f577557c3  4
c4958a1501dd46e4b8f154a28201eb7a  1
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10412 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:59:18.578536   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:36375
I20260709 12:59:18.579630   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:18.586037  1417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:18.586073  1416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:18.587088  1419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:18.587144   431 server_base.cc:1061] running on GCE node
I20260709 12:59:18.588586   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:18.588766   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:18.588881   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601958588870 us; error 0 us; skew 500 ppm
I20260709 12:59:18.589758   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:18.592165   431 webserver.cc:533] Webserver started at http://127.0.107.254:42861/ using document root <none> and password file <none>
I20260709 12:59:18.592612   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:18.592760   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:18.592989   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:18.594151   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/master-0-root/instance:
uuid: "1ca716594c6348a883b88d48068b8416"
format_stamp: "Formatted at 2026-07-09 12:59:18 on dist-test-slave-jk05"
I20260709 12:59:18.598616   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:59:18.602092  1424 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:18.602922   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:59:18.603236   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/master-0-root
uuid: "1ca716594c6348a883b88d48068b8416"
format_stamp: "Formatted at 2026-07-09 12:59:18 on dist-test-slave-jk05"
I20260709 12:59:18.603542   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:18.624964   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:18.626320   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:18.664425   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:36375
I20260709 12:59:18.664547  1475 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:36375 every 8 connection(s)
I20260709 12:59:18.668445  1476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:18.678349  1476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Bootstrap starting.
I20260709 12:59:18.682914  1476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:18.687215  1476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: No bootstrap required, opened a new log
I20260709 12:59:18.689347  1476 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca716594c6348a883b88d48068b8416" member_type: VOTER }
I20260709 12:59:18.689749  1476 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:18.689926  1476 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ca716594c6348a883b88d48068b8416, State: Initialized, Role: FOLLOWER
I20260709 12:59:18.690544  1476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca716594c6348a883b88d48068b8416" member_type: VOTER }
I20260709 12:59:18.690969  1476 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:59:18.691164  1476 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:59:18.691376  1476 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:18.695983  1476 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca716594c6348a883b88d48068b8416" member_type: VOTER }
I20260709 12:59:18.696537  1476 leader_election.cc:304] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ca716594c6348a883b88d48068b8416; no voters: 
I20260709 12:59:18.697794  1476 leader_election.cc:290] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:59:18.698256  1479 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:18.699656  1479 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [term 1 LEADER]: Becoming Leader. State: Replica: 1ca716594c6348a883b88d48068b8416, State: Running, Role: LEADER
I20260709 12:59:18.700408  1479 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca716594c6348a883b88d48068b8416" member_type: VOTER }
I20260709 12:59:18.700902  1476 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:59:18.705139  1480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ca716594c6348a883b88d48068b8416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca716594c6348a883b88d48068b8416" member_type: VOTER } }
I20260709 12:59:18.705967  1480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:18.706390  1481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ca716594c6348a883b88d48068b8416. Latest consensus state: current_term: 1 leader_uuid: "1ca716594c6348a883b88d48068b8416" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ca716594c6348a883b88d48068b8416" member_type: VOTER } }
I20260709 12:59:18.707203  1481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [sys.catalog]: This master's current role is: LEADER
I20260709 12:59:18.713551  1486 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:59:18.721290  1486 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:59:18.722460   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:59:18.730546  1486 catalog_manager.cc:1383] Generated new cluster ID: ec41e2b038064970a91ea87c28f76db5
I20260709 12:59:18.730834  1486 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:59:18.774194  1486 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:59:18.775498  1486 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:59:18.787623  1486 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Generated new TSK 0
I20260709 12:59:18.788319  1486 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:59:18.853823   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:18.860476  1497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:18.861572  1498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:18.863049   431 server_base.cc:1061] running on GCE node
W20260709 12:59:18.863899  1500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:18.864938   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:18.865197   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:18.865392   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601958865365 us; error 0 us; skew 500 ppm
I20260709 12:59:18.866029   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:18.868454   431 webserver.cc:533] Webserver started at http://127.0.107.193:44505/ using document root <none> and password file <none>
I20260709 12:59:18.868963   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:18.869184   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:18.869529   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:18.870661   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/instance:
uuid: "83f6f5810950417195f707ff308a8125"
format_stamp: "Formatted at 2026-07-09 12:59:18 on dist-test-slave-jk05"
I20260709 12:59:18.875607   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:59:18.879216  1505 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:18.880167   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:59:18.880553   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root
uuid: "83f6f5810950417195f707ff308a8125"
format_stamp: "Formatted at 2026-07-09 12:59:18 on dist-test-slave-jk05"
I20260709 12:59:18.880860   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:18.895030   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:18.896360   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:18.897941   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:18.900432   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:18.900700   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:18.900954   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:18.901150   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:18.945190   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:43459
I20260709 12:59:18.945290  1567 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:43459 every 8 connection(s)
I20260709 12:59:18.950194   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:18.957625  1572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:18.958954  1573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:18.960630  1575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:18.962311   431 server_base.cc:1061] running on GCE node
I20260709 12:59:18.963456   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:18.963742   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:18.963967   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601958963925 us; error 0 us; skew 500 ppm
I20260709 12:59:18.964581   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:18.964876  1568 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36375
I20260709 12:59:18.965307  1568 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:18.966274  1568 heartbeater.cc:507] Master 127.0.107.254:36375 requested a full tablet report, sending...
I20260709 12:59:18.967338   431 webserver.cc:533] Webserver started at http://127.0.107.194:41641/ using document root <none> and password file <none>
I20260709 12:59:18.967944   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:18.968179   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:18.968458   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:18.968894  1441 ts_manager.cc:194] Registered new tserver with Master: 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
I20260709 12:59:18.969754   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/instance:
uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
format_stamp: "Formatted at 2026-07-09 12:59:18 on dist-test-slave-jk05"
I20260709 12:59:18.970759  1441 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55356
I20260709 12:59:18.974500   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:59:18.977943  1580 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:18.978726   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:59:18.979033   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root
uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
format_stamp: "Formatted at 2026-07-09 12:59:18 on dist-test-slave-jk05"
I20260709 12:59:18.979314   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:18.990079   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:18.991267   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:18.992718   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:18.995018   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:18.995250   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:18.995493   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:18.995672   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:19.040793   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:43887
I20260709 12:59:19.040884  1642 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:43887 every 8 connection(s)
I20260709 12:59:19.045524   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:19.052668  1646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:19.054397  1647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:19.057354   431 server_base.cc:1061] running on GCE node
W20260709 12:59:19.058225  1649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:19.058768  1643 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36375
I20260709 12:59:19.059140  1643 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:19.059859  1643 heartbeater.cc:507] Master 127.0.107.254:36375 requested a full tablet report, sending...
I20260709 12:59:19.061138   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:19.061462   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:19.061653   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601959061607 us; error 0 us; skew 500 ppm
I20260709 12:59:19.062098  1441 ts_manager.cc:194] Registered new tserver with Master: 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:19.062232   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:19.063740  1441 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55368
I20260709 12:59:19.065066   431 webserver.cc:533] Webserver started at http://127.0.107.195:34655/ using document root <none> and password file <none>
I20260709 12:59:19.065680   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:19.065904   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:19.066188   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:19.067342   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/instance:
uuid: "984c9089d9804652846f3caeb609d56e"
format_stamp: "Formatted at 2026-07-09 12:59:19 on dist-test-slave-jk05"
I20260709 12:59:19.072094   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:59:19.075547  1654 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:19.076381   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:59:19.076735   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root
uuid: "984c9089d9804652846f3caeb609d56e"
format_stamp: "Formatted at 2026-07-09 12:59:19 on dist-test-slave-jk05"
I20260709 12:59:19.077106   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:19.110562   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:19.111809   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:19.113273   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:19.115604   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:19.115854   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:19.116103   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:19.116284   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:19.156942   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:33859
I20260709 12:59:19.157091  1716 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:33859 every 8 connection(s)
I20260709 12:59:19.178007  1717 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36375
I20260709 12:59:19.178406  1717 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:19.179121  1717 heartbeater.cc:507] Master 127.0.107.254:36375 requested a full tablet report, sending...
I20260709 12:59:19.181270  1441 ts_manager.cc:194] Registered new tserver with Master: 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:19.181684   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.021538113s
I20260709 12:59:19.182804  1441 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55374
I20260709 12:59:19.214176  1441 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55390:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:59:19.221206  1441 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:59:19.441565  1682 tablet_service.cc:1511] Processing CreateTablet for tablet 85ae4f3774324fcf9745a42f30ffe08b (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:59:19.443097  1682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85ae4f3774324fcf9745a42f30ffe08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.446353  1681 tablet_service.cc:1511] Processing CreateTablet for tablet caae9b2a426d42fb8c2353e1c978d0fb (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:59:19.446871  1608 tablet_service.cc:1511] Processing CreateTablet for tablet 85ae4f3774324fcf9745a42f30ffe08b (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:59:19.447824  1681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caae9b2a426d42fb8c2353e1c978d0fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.448079  1608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85ae4f3774324fcf9745a42f30ffe08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.450541  1607 tablet_service.cc:1511] Processing CreateTablet for tablet caae9b2a426d42fb8c2353e1c978d0fb (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:59:19.451958  1607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caae9b2a426d42fb8c2353e1c978d0fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.451912  1679 tablet_service.cc:1511] Processing CreateTablet for tablet ab5b971190d342d2b2568c30f3c0c81f (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:59:19.453373  1679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5b971190d342d2b2568c30f3c0c81f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.454588  1606 tablet_service.cc:1511] Processing CreateTablet for tablet 674516b0482b47aaa9380469245f9d2e (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:59:19.455948  1606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674516b0482b47aaa9380469245f9d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.458726  1605 tablet_service.cc:1511] Processing CreateTablet for tablet ab5b971190d342d2b2568c30f3c0c81f (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:59:19.460110  1605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5b971190d342d2b2568c30f3c0c81f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.462687  1678 tablet_service.cc:1511] Processing CreateTablet for tablet 978c5dd19ae34aa4b4b42b8c3ccfb52a (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:59:19.463176  1676 tablet_service.cc:1511] Processing CreateTablet for tablet 3ab2e02ed3c849e9bda035a57add301d (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:59:19.464084  1678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 978c5dd19ae34aa4b4b42b8c3ccfb52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.467383  1677 tablet_service.cc:1511] Processing CreateTablet for tablet 488823792cd54090a36c758519a28875 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:59:19.450544  1680 tablet_service.cc:1511] Processing CreateTablet for tablet 674516b0482b47aaa9380469245f9d2e (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:59:19.469967  1676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ab2e02ed3c849e9bda035a57add301d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.469877  1675 tablet_service.cc:1511] Processing CreateTablet for tablet ae2bf844acee4ce5b28b92217256b084 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:59:19.470650  1680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674516b0482b47aaa9380469245f9d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.471236  1675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae2bf844acee4ce5b28b92217256b084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.474615  1604 tablet_service.cc:1511] Processing CreateTablet for tablet 978c5dd19ae34aa4b4b42b8c3ccfb52a (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:59:19.476198  1604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 978c5dd19ae34aa4b4b42b8c3ccfb52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.476548  1601 tablet_service.cc:1511] Processing CreateTablet for tablet ae2bf844acee4ce5b28b92217256b084 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:59:19.474615  1602 tablet_service.cc:1511] Processing CreateTablet for tablet 3ab2e02ed3c849e9bda035a57add301d (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:59:19.480041  1603 tablet_service.cc:1511] Processing CreateTablet for tablet 488823792cd54090a36c758519a28875 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:59:19.480859  1674 tablet_service.cc:1511] Processing CreateTablet for tablet 474d3b57c33542aba459b97733122bf0 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:59:19.482785  1601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae2bf844acee4ce5b28b92217256b084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.484112  1673 tablet_service.cc:1511] Processing CreateTablet for tablet c7154d65b3794a0e87f1ae971e939fec (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:59:19.485193  1602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ab2e02ed3c849e9bda035a57add301d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.487880  1533 tablet_service.cc:1511] Processing CreateTablet for tablet 85ae4f3774324fcf9745a42f30ffe08b (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:59:19.490444  1677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 488823792cd54090a36c758519a28875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.490689  1532 tablet_service.cc:1511] Processing CreateTablet for tablet caae9b2a426d42fb8c2353e1c978d0fb (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:59:19.495267  1530 tablet_service.cc:1511] Processing CreateTablet for tablet ab5b971190d342d2b2568c30f3c0c81f (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:59:19.498263  1529 tablet_service.cc:1511] Processing CreateTablet for tablet 978c5dd19ae34aa4b4b42b8c3ccfb52a (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:59:19.499365  1599 tablet_service.cc:1511] Processing CreateTablet for tablet c7154d65b3794a0e87f1ae971e939fec (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:59:19.493037  1531 tablet_service.cc:1511] Processing CreateTablet for tablet 674516b0482b47aaa9380469245f9d2e (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:59:19.501454  1600 tablet_service.cc:1511] Processing CreateTablet for tablet 474d3b57c33542aba459b97733122bf0 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:59:19.505527  1674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 474d3b57c33542aba459b97733122bf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.489321  1533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85ae4f3774324fcf9745a42f30ffe08b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.506884  1673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7154d65b3794a0e87f1ae971e939fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.507302  1532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caae9b2a426d42fb8c2353e1c978d0fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.507958  1603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 488823792cd54090a36c758519a28875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.509675  1600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 474d3b57c33542aba459b97733122bf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.510901  1531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674516b0482b47aaa9380469245f9d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.512557  1599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7154d65b3794a0e87f1ae971e939fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.517808  1529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 978c5dd19ae34aa4b4b42b8c3ccfb52a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.521921  1530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5b971190d342d2b2568c30f3c0c81f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.542635  1528 tablet_service.cc:1511] Processing CreateTablet for tablet 488823792cd54090a36c758519a28875 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:59:19.546455  1528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 488823792cd54090a36c758519a28875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.547137  1527 tablet_service.cc:1511] Processing CreateTablet for tablet 3ab2e02ed3c849e9bda035a57add301d (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:59:19.548573  1527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ab2e02ed3c849e9bda035a57add301d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.549114  1682 tablet_service.cc:1511] Processing CreateTablet for tablet 3ce476a5b0534bc5b6915606f6d37f30 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:59:19.550460  1682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ce476a5b0534bc5b6915606f6d37f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.566772  1526 tablet_service.cc:1511] Processing CreateTablet for tablet ae2bf844acee4ce5b28b92217256b084 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:59:19.568290  1526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae2bf844acee4ce5b28b92217256b084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.575268  1525 tablet_service.cc:1511] Processing CreateTablet for tablet 474d3b57c33542aba459b97733122bf0 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:59:19.576869  1525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 474d3b57c33542aba459b97733122bf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.579191  1524 tablet_service.cc:1511] Processing CreateTablet for tablet c7154d65b3794a0e87f1ae971e939fec (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:59:19.582239  1524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7154d65b3794a0e87f1ae971e939fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.584262  1682 tablet_service.cc:1511] Processing CreateTablet for tablet 969ec621630549bb9b288fdb50e46fbe (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:59:19.587345  1682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 969ec621630549bb9b288fdb50e46fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.615113  1678 tablet_service.cc:1511] Processing CreateTablet for tablet da6820413cb240fd9a4bd8655e6762f7 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:59:19.616966  1678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da6820413cb240fd9a4bd8655e6762f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.623481  1602 tablet_service.cc:1511] Processing CreateTablet for tablet 3ce476a5b0534bc5b6915606f6d37f30 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:59:19.624867  1602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ce476a5b0534bc5b6915606f6d37f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.635324  1680 tablet_service.cc:1511] Processing CreateTablet for tablet 49db1a2fe0dd40eba2ac450e306a6937 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:59:19.641896  1733 tablet_bootstrap.cc:492] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:19.646896  1606 tablet_service.cc:1511] Processing CreateTablet for tablet c53638cee15242e6a8f5dff3907d70c2 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:59:19.648218  1733 tablet_bootstrap.cc:654] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:19.653128  1732 tablet_bootstrap.cc:492] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:19.664188  1674 tablet_service.cc:1511] Processing CreateTablet for tablet ed741b21bc0447a0896904a04e535f8a (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:59:19.666734  1604 tablet_service.cc:1511] Processing CreateTablet for tablet da6820413cb240fd9a4bd8655e6762f7 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:59:19.668411  1606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c53638cee15242e6a8f5dff3907d70c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.680619  1528 tablet_service.cc:1511] Processing CreateTablet for tablet 3ce476a5b0534bc5b6915606f6d37f30 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:59:19.682653  1528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ce476a5b0534bc5b6915606f6d37f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.685124  1674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed741b21bc0447a0896904a04e535f8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.687125  1608 tablet_service.cc:1511] Processing CreateTablet for tablet 49db1a2fe0dd40eba2ac450e306a6937 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:59:19.688145  1532 tablet_service.cc:1511] Processing CreateTablet for tablet da6820413cb240fd9a4bd8655e6762f7 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:59:19.709204  1608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49db1a2fe0dd40eba2ac450e306a6937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.710103  1604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da6820413cb240fd9a4bd8655e6762f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.704948  1680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49db1a2fe0dd40eba2ac450e306a6937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.711954  1675 tablet_service.cc:1511] Processing CreateTablet for tablet c53638cee15242e6a8f5dff3907d70c2 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:59:19.713368  1675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c53638cee15242e6a8f5dff3907d70c2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:19.714068  1489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:19.717077  1600 tablet_service.cc:1511] Processing CreateTablet for tablet 27181dd872d647748dc1fe8c2ac6c12b (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:59:19.716655  1676 tablet_service.cc:1511] Processing CreateTablet for tablet eed65fd1add844e7821095e2215fb393 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:59:19.718475  1600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27181dd872d647748dc1fe8c2ac6c12b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.717396  1679 tablet_service.cc:1511] Processing CreateTablet for tablet 12adfb7855f340329f110249c207f409 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:59:19.735180  1599 tablet_service.cc:1511] Processing CreateTablet for tablet 969ec621630549bb9b288fdb50e46fbe (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:59:19.738035  1599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 969ec621630549bb9b288fdb50e46fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.718161  1607 tablet_service.cc:1511] Processing CreateTablet for tablet ed741b21bc0447a0896904a04e535f8a (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:59:19.741990  1607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed741b21bc0447a0896904a04e535f8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.742475  1682 tablet_service.cc:1511] Processing CreateTablet for tablet 4d7cdb06308e4f21a67e6a685abdc179 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:59:19.707547  1531 tablet_service.cc:1511] Processing CreateTablet for tablet 49db1a2fe0dd40eba2ac450e306a6937 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:59:19.738077  1532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da6820413cb240fd9a4bd8655e6762f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.741993  1732 tablet_bootstrap.cc:654] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:19.756549  1681 tablet_service.cc:1511] Processing CreateTablet for tablet 034eef09c5fd47c9b010158a104b4028 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:59:19.762765  1530 tablet_service.cc:1511] Processing CreateTablet for tablet c53638cee15242e6a8f5dff3907d70c2 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:59:19.764329  1530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c53638cee15242e6a8f5dff3907d70c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.764542  1677 tablet_service.cc:1511] Processing CreateTablet for tablet e2c52c22aef94ec6b8226f64dc4baca4 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:59:19.722337  1673 tablet_service.cc:1511] Processing CreateTablet for tablet 27181dd872d647748dc1fe8c2ac6c12b (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:59:19.770378  1734 tablet_bootstrap.cc:492] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:19.778625  1676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eed65fd1add844e7821095e2215fb393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.782850  1531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 49db1a2fe0dd40eba2ac450e306a6937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.787808  1679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12adfb7855f340329f110249c207f409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.788578  1673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27181dd872d647748dc1fe8c2ac6c12b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.795975  1601 tablet_service.cc:1511] Processing CreateTablet for tablet 12adfb7855f340329f110249c207f409 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:59:19.797387  1601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12adfb7855f340329f110249c207f409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.807966  1733 tablet_bootstrap.cc:492] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:19.808905  1733 ts_tablet_manager.cc:1403] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.167s	user 0.007s	sys 0.013s
I20260709 12:59:19.801785  1677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2c52c22aef94ec6b8226f64dc4baca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.809522  1527 tablet_service.cc:1511] Processing CreateTablet for tablet 27181dd872d647748dc1fe8c2ac6c12b (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:59:19.810683  1682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d7cdb06308e4f21a67e6a685abdc179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.816118  1734 tablet_bootstrap.cc:654] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:19.822979  1681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034eef09c5fd47c9b010158a104b4028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.826280  1524 tablet_service.cc:1511] Processing CreateTablet for tablet eed65fd1add844e7821095e2215fb393 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:59:19.812008  1733 raft_consensus.cc:359] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:19.829978  1732 tablet_bootstrap.cc:492] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:19.835207  1732 ts_tablet_manager.cc:1403] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.182s	user 0.008s	sys 0.010s
I20260709 12:59:19.840404  1732 raft_consensus.cc:359] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:19.841176  1732 raft_consensus.cc:385] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:19.788288  1603 tablet_service.cc:1511] Processing CreateTablet for tablet eed65fd1add844e7821095e2215fb393 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:59:19.843488  1603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eed65fd1add844e7821095e2215fb393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.841619  1533 tablet_service.cc:1511] Processing CreateTablet for tablet 12adfb7855f340329f110249c207f409 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:59:19.841827  1525 tablet_service.cc:1511] Processing CreateTablet for tablet ed741b21bc0447a0896904a04e535f8a (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:59:19.831135  1733 raft_consensus.cc:385] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:19.834273  1678 tablet_service.cc:1511] Processing CreateTablet for tablet ad1245d9a7bf4a21ac3d2291898dd976 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:59:19.846127  1526 tablet_service.cc:1511] Processing CreateTablet for tablet 969ec621630549bb9b288fdb50e46fbe (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:59:19.846894  1678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad1245d9a7bf4a21ac3d2291898dd976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.846151  1733 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:19.850468  1733 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:19.852276  1602 tablet_service.cc:1511] Processing CreateTablet for tablet 034eef09c5fd47c9b010158a104b4028 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:59:19.854826  1606 tablet_service.cc:1511] Processing CreateTablet for tablet e2c52c22aef94ec6b8226f64dc4baca4 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:59:19.856976  1680 tablet_service.cc:1511] Processing CreateTablet for tablet 1a69ce55a73e464a962e15e73fd1db2c (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:59:19.857646  1732 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:19.858434  1680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a69ce55a73e464a962e15e73fd1db2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.858503  1732 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:19.860263  1524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eed65fd1add844e7821095e2215fb393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.862524  1607 tablet_service.cc:1511] Processing CreateTablet for tablet 1a69ce55a73e464a962e15e73fd1db2c (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:59:19.864312  1675 tablet_service.cc:1511] Processing CreateTablet for tablet 0a1244fbe0514ba59a0164883a315bbe (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:59:19.865015  1608 tablet_service.cc:1511] Processing CreateTablet for tablet ad1245d9a7bf4a21ac3d2291898dd976 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:59:19.866739  1533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12adfb7855f340329f110249c207f409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.845120  1605 tablet_service.cc:1511] Processing CreateTablet for tablet 4d7cdb06308e4f21a67e6a685abdc179 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:59:19.868120  1527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27181dd872d647748dc1fe8c2ac6c12b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.865691  1675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a1244fbe0514ba59a0164883a315bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.872686  1602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034eef09c5fd47c9b010158a104b4028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.884881  1673 tablet_service.cc:1511] Processing CreateTablet for tablet 380b2dca2259476bbe1f22b2eb110fd0 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:59:19.896168  1673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 380b2dca2259476bbe1f22b2eb110fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.896344  1599 tablet_service.cc:1511] Processing CreateTablet for tablet 0a1244fbe0514ba59a0164883a315bbe (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:59:19.900871  1606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2c52c22aef94ec6b8226f64dc4baca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.908147  1643 heartbeater.cc:499] Master 127.0.107.254:36375 was elected leader, sending a full tablet report...
I20260709 12:59:19.908541  1732 ts_tablet_manager.cc:1434] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.073s	user 0.002s	sys 0.008s
I20260709 12:59:19.909595  1608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad1245d9a7bf4a21ac3d2291898dd976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.921465  1526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 969ec621630549bb9b288fdb50e46fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.924777  1734 tablet_bootstrap.cc:492] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:19.925324  1734 ts_tablet_manager.cc:1403] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.222s	user 0.009s	sys 0.012s
I20260709 12:59:19.924091  1733 ts_tablet_manager.cc:1434] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.115s	user 0.013s	sys 0.029s
I20260709 12:59:19.926383  1531 tablet_service.cc:1511] Processing CreateTablet for tablet 034eef09c5fd47c9b010158a104b4028 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:59:19.927359  1717 heartbeater.cc:499] Master 127.0.107.254:36375 was elected leader, sending a full tablet report...
I20260709 12:59:19.928800  1531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034eef09c5fd47c9b010158a104b4028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.928467  1734 raft_consensus.cc:359] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:19.930263  1734 raft_consensus.cc:385] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:19.930586  1734 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:19.931484  1734 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:19.927299  1529 tablet_service.cc:1511] Processing CreateTablet for tablet 4d7cdb06308e4f21a67e6a685abdc179 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:59:19.933687  1525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed741b21bc0447a0896904a04e535f8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.937793  1607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a69ce55a73e464a962e15e73fd1db2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.940099  1733 tablet_bootstrap.cc:492] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:19.943248  1529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d7cdb06308e4f21a67e6a685abdc179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.956279  1605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d7cdb06308e4f21a67e6a685abdc179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.961669  1732 tablet_bootstrap.cc:492] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:19.975466  1568 heartbeater.cc:499] Master 127.0.107.254:36375 was elected leader, sending a full tablet report...
I20260709 12:59:19.988006  1599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a1244fbe0514ba59a0164883a315bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:19.993933  1600 tablet_service.cc:1511] Processing CreateTablet for tablet 380b2dca2259476bbe1f22b2eb110fd0 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:59:19.995376  1600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 380b2dca2259476bbe1f22b2eb110fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.001827  1681 tablet_service.cc:1511] Processing CreateTablet for tablet 3b24dad799cc469c9d10c4833dffe30c (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:59:20.003218  1681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b24dad799cc469c9d10c4833dffe30c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.005754  1674 tablet_service.cc:1511] Processing CreateTablet for tablet 6087b5252cf44609bcf98bb0925b1666 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:59:20.010303  1604 tablet_service.cc:1511] Processing CreateTablet for tablet 532fefba367345759887da9fcc9872e3 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:59:20.011063  1532 tablet_service.cc:1511] Processing CreateTablet for tablet e2c52c22aef94ec6b8226f64dc4baca4 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:59:20.011804  1604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 532fefba367345759887da9fcc9872e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.012301  1532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2c52c22aef94ec6b8226f64dc4baca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.012027  1526 tablet_service.cc:1511] Processing CreateTablet for tablet 1a69ce55a73e464a962e15e73fd1db2c (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:59:20.013231  1526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a69ce55a73e464a962e15e73fd1db2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.014456  1734 ts_tablet_manager.cc:1434] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.089s	user 0.011s	sys 0.022s
I20260709 12:59:20.008864  1530 tablet_service.cc:1511] Processing CreateTablet for tablet ad1245d9a7bf4a21ac3d2291898dd976 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:59:19.997396  1676 tablet_service.cc:1511] Processing CreateTablet for tablet 532fefba367345759887da9fcc9872e3 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:59:20.018939  1734 tablet_bootstrap.cc:492] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:20.022070  1528 tablet_service.cc:1511] Processing CreateTablet for tablet 0a1244fbe0514ba59a0164883a315bbe (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:59:20.023048  1606 tablet_service.cc:1511] Processing CreateTablet for tablet 3b24dad799cc469c9d10c4833dffe30c (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:59:20.026496  1679 tablet_service.cc:1511] Processing CreateTablet for tablet 8343e6d9eb5348b99ed8b5dc38a6087d (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:59:20.047210  1528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a1244fbe0514ba59a0164883a315bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.030000  1525 tablet_service.cc:1511] Processing CreateTablet for tablet 380b2dca2259476bbe1f22b2eb110fd0 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:59:20.047807  1606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b24dad799cc469c9d10c4833dffe30c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.010289  1674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6087b5252cf44609bcf98bb0925b1666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.048625  1676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 532fefba367345759887da9fcc9872e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.049901  1525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 380b2dca2259476bbe1f22b2eb110fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.052080  1679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8343e6d9eb5348b99ed8b5dc38a6087d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.066510  1530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad1245d9a7bf4a21ac3d2291898dd976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.089726  1603 tablet_service.cc:1511] Processing CreateTablet for tablet 6087b5252cf44609bcf98bb0925b1666 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:59:20.091564  1603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6087b5252cf44609bcf98bb0925b1666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.115581  1739 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.116304  1739 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:20.125654  1428 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 54469.6us
I20260709 12:59:20.134836  1739 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:20.138691  1599 tablet_service.cc:1511] Processing CreateTablet for tablet 8343e6d9eb5348b99ed8b5dc38a6087d (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:59:20.161290  1602 tablet_service.cc:1511] Processing CreateTablet for tablet f03fb00780d54c5fbc0be2c8a1b8d4aa (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:59:20.143325  1733 tablet_bootstrap.cc:654] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.179910  1599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8343e6d9eb5348b99ed8b5dc38a6087d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.185977  1679 tablet_service.cc:1511] Processing CreateTablet for tablet f03fb00780d54c5fbc0be2c8a1b8d4aa (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:59:20.187376  1732 tablet_bootstrap.cc:654] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.188109  1527 tablet_service.cc:1511] Processing CreateTablet for tablet 532fefba367345759887da9fcc9872e3 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:59:20.192255  1527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 532fefba367345759887da9fcc9872e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.187515  1679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f03fb00780d54c5fbc0be2c8a1b8d4aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.202934  1682 tablet_service.cc:1511] Processing CreateTablet for tablet 960d36351aeb45a08c60a7d839c40e6d (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:59:20.203763  1533 tablet_service.cc:1511] Processing CreateTablet for tablet 3b24dad799cc469c9d10c4833dffe30c (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:59:20.205246  1602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f03fb00780d54c5fbc0be2c8a1b8d4aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.205785  1677 tablet_service.cc:1511] Processing CreateTablet for tablet 0695e4119591411b8adab45b5925e8eb (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:59:20.207865  1678 tablet_service.cc:1511] Processing CreateTablet for tablet 7ba0620abe844f5bb71ba9dd947c3b9c (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:59:20.213730  1531 tablet_service.cc:1511] Processing CreateTablet for tablet 6087b5252cf44609bcf98bb0925b1666 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:59:20.215061  1531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6087b5252cf44609bcf98bb0925b1666. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.218384  1525 tablet_service.cc:1511] Processing CreateTablet for tablet f03fb00780d54c5fbc0be2c8a1b8d4aa (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:59:20.219921  1525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f03fb00780d54c5fbc0be2c8a1b8d4aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.233862  1531 tablet_service.cc:1511] Processing CreateTablet for tablet 0695e4119591411b8adab45b5925e8eb (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:59:20.235289  1740 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.235832  1740 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:20.238600  1740 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:20.216881  1524 tablet_service.cc:1511] Processing CreateTablet for tablet 8343e6d9eb5348b99ed8b5dc38a6087d (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:59:20.235297  1531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0695e4119591411b8adab45b5925e8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.234541  1675 tablet_service.cc:1511] Processing CreateTablet for tablet b833a9bd5950418885dc620b52d4558f (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:59:20.257737  1682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 960d36351aeb45a08c60a7d839c40e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.274821  1608 tablet_service.cc:1511] Processing CreateTablet for tablet fa01a3d0bb094d0d80ca50f0287dbb27 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:59:20.276242  1608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa01a3d0bb094d0d80ca50f0287dbb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.243417  1605 tablet_service.cc:1511] Processing CreateTablet for tablet 34b2ba71a23e4ba3becae445ee7598c5 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:59:20.279034  1605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34b2ba71a23e4ba3becae445ee7598c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.289251  1600 tablet_service.cc:1511] Processing CreateTablet for tablet 960d36351aeb45a08c60a7d839c40e6d (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:59:20.290729  1600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 960d36351aeb45a08c60a7d839c40e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.243417  1674 tablet_service.cc:1511] Processing CreateTablet for tablet 34b2ba71a23e4ba3becae445ee7598c5 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:59:20.295167  1738 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.295738  1738 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:20.296243  1530 tablet_service.cc:1511] Processing CreateTablet for tablet fa01a3d0bb094d0d80ca50f0287dbb27 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:59:20.294186  1680 tablet_service.cc:1511] Processing CreateTablet for tablet fa01a3d0bb094d0d80ca50f0287dbb27 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:59:20.303144  1738 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:20.306279  1734 tablet_bootstrap.cc:654] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.310143  1533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b24dad799cc469c9d10c4833dffe30c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.322854  1678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ba0620abe844f5bb71ba9dd947c3b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.339823  1673 tablet_service.cc:1511] Processing CreateTablet for tablet 214bc9e9284a4502844dbdc3395843fa (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:59:20.342298  1601 tablet_service.cc:1511] Processing CreateTablet for tablet 0695e4119591411b8adab45b5925e8eb (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:59:20.343686  1601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0695e4119591411b8adab45b5925e8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.347328  1681 tablet_service.cc:1511] Processing CreateTablet for tablet 13adc029302444abaaffd4bed11760a3 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:59:20.362658  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:20.362740  1526 tablet_service.cc:1511] Processing CreateTablet for tablet 34b2ba71a23e4ba3becae445ee7598c5 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:59:20.364324  1528 tablet_service.cc:1511] Processing CreateTablet for tablet 960d36351aeb45a08c60a7d839c40e6d (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260709 12:59:20.367326  1584 leader_election.cc:343] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.369565  1527 tablet_service.cc:1511] Processing CreateTablet for tablet 7ba0620abe844f5bb71ba9dd947c3b9c (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:59:20.374163  1606 tablet_service.cc:1511] Processing CreateTablet for tablet 7ba0620abe844f5bb71ba9dd947c3b9c (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:59:20.376976  1606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ba0620abe844f5bb71ba9dd947c3b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.352972  1680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa01a3d0bb094d0d80ca50f0287dbb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.354099  1524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8343e6d9eb5348b99ed8b5dc38a6087d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.392925  1733 tablet_bootstrap.cc:492] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:20.413264  1732 tablet_bootstrap.cc:492] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:20.422135  1528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 960d36351aeb45a08c60a7d839c40e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.388494  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:20.423312  1525 tablet_service.cc:1511] Processing CreateTablet for tablet b833a9bd5950418885dc620b52d4558f (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:59:20.394837  1676 tablet_service.cc:1511] Processing CreateTablet for tablet e0058494c1f14737a2e0e2b8f4697969 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:59:20.429987  1603 tablet_service.cc:1511] Processing CreateTablet for tablet 13adc029302444abaaffd4bed11760a3 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:59:20.431294  1603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13adc029302444abaaffd4bed11760a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.421592  1733 ts_tablet_manager.cc:1403] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.486s	user 0.012s	sys 0.023s
I20260709 12:59:20.421299  1675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b833a9bd5950418885dc620b52d4558f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.427417  1732 ts_tablet_manager.cc:1403] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.475s	user 0.022s	sys 0.010s
I20260709 12:59:20.435468  1526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34b2ba71a23e4ba3becae445ee7598c5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:20.436995  1582 leader_election.cc:343] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.455729  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:20.456959  1739 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.439522  1529 tablet_service.cc:1511] Processing CreateTablet for tablet 13adc029302444abaaffd4bed11760a3 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:59:20.400130  1604 tablet_service.cc:1511] Processing CreateTablet for tablet b833a9bd5950418885dc620b52d4558f (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:59:20.446193  1607 tablet_service.cc:1511] Processing CreateTablet for tablet e4d8a1326e854e51b5539e7addc91fe5 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:59:20.385555  1602 tablet_service.cc:1511] Processing CreateTablet for tablet 214bc9e9284a4502844dbdc3395843fa (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:59:20.461206  1607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4d8a1326e854e51b5539e7addc91fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.462399  1732 raft_consensus.cc:359] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:20.462951  1734 tablet_bootstrap.cc:492] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:20.463183  1732 raft_consensus.cc:385] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.463546  1732 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.463646  1734 ts_tablet_manager.cc:1403] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.445s	user 0.013s	sys 0.018s
I20260709 12:59:20.464322  1732 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:20.466171  1604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b833a9bd5950418885dc620b52d4558f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.467172  1734 raft_consensus.cc:359] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:20.469213  1732 ts_tablet_manager.cc:1434] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:59:20.449558  1605 tablet_service.cc:1511] Processing CreateTablet for tablet e0058494c1f14737a2e0e2b8f4697969 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:59:20.404377  1679 tablet_service.cc:1511] Processing CreateTablet for tablet e4d8a1326e854e51b5539e7addc91fe5 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:59:20.478227  1676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0058494c1f14737a2e0e2b8f4697969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.481478  1530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fa01a3d0bb094d0d80ca50f0287dbb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.425836  1532 tablet_service.cc:1511] Processing CreateTablet for tablet 214bc9e9284a4502844dbdc3395843fa (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:59:20.486167  1602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 214bc9e9284a4502844dbdc3395843fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.499933  1673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 214bc9e9284a4502844dbdc3395843fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.473308  1734 raft_consensus.cc:385] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.507546  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:20.510635  1525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b833a9bd5950418885dc620b52d4558f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.511126  1605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0058494c1f14737a2e0e2b8f4697969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.526623  1606 tablet_service.cc:1511] Processing CreateTablet for tablet 81301f84b53f41349ac6bca78f1a77f4 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
W20260709 12:59:20.527276  1509 leader_election.cc:343] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.513597  1732 tablet_bootstrap.cc:492] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:20.526803  1599 tablet_service.cc:1511] Processing CreateTablet for tablet 88cbab659ea041148748114fae549ae0 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:59:20.528144  1532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 214bc9e9284a4502844dbdc3395843fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.528698  1599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88cbab659ea041148748114fae549ae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.536633  1527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ba0620abe844f5bb71ba9dd947c3b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.537061  1607 tablet_service.cc:1511] Processing CreateTablet for tablet ebc9ea99300141e883fcf2f1b0d7a744 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:59:20.538589  1607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc9ea99300141e883fcf2f1b0d7a744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.512238  1733 raft_consensus.cc:359] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 12:59:20.541985  1733 raft_consensus.cc:385] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.543131  1733 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.545308  1678 tablet_service.cc:1511] Processing CreateTablet for tablet 81301f84b53f41349ac6bca78f1a77f4 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:59:20.550171  1529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13adc029302444abaaffd4bed11760a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.517007  1734 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.555436  1734 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:20.557998  1734 ts_tablet_manager.cc:1434] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.094s	user 0.002s	sys 0.005s
I20260709 12:59:20.558920  1734 tablet_bootstrap.cc:492] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:20.567603  1600 tablet_service.cc:1511] Processing CreateTablet for tablet bb1171fd6323498ea16dfea3e5356fe0 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:59:20.567057  1733 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 12:59:20.570268  1606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81301f84b53f41349ac6bca78f1a77f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.573031  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:20.582195  1677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0695e4119591411b8adab45b5925e8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.584185  1674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34b2ba71a23e4ba3becae445ee7598c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.586546  1678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81301f84b53f41349ac6bca78f1a77f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.587425  1681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13adc029302444abaaffd4bed11760a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.591197  1733 ts_tablet_manager.cc:1434] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.138s	user 0.006s	sys 0.003s
I20260709 12:59:20.592892  1679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4d8a1326e854e51b5539e7addc91fe5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:20.605213  1507 leader_election.cc:343] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.605865  1507 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:20.607231  1740 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.606985  1601 tablet_service.cc:1511] Processing CreateTablet for tablet 22357a3949344a1792e5bae4a0291987 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:59:20.608784  1601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22357a3949344a1792e5bae4a0291987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.553035  1608 tablet_service.cc:1511] Processing CreateTablet for tablet 59d952214bc24204a149e6f1a9af3184 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:59:20.621776  1608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d952214bc24204a149e6f1a9af3184. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:20.641554  1428 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 43802.1us
I20260709 12:59:20.649227  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:20.650956  1732 tablet_bootstrap.cc:654] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:20.651131  1658 leader_election.cc:343] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.657666  1733 tablet_bootstrap.cc:492] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:20.674654  1604 tablet_service.cc:1511] Processing CreateTablet for tablet c65847e774e445909d83c69bf486b7fc (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:59:20.676532  1526 tablet_service.cc:1511] Processing CreateTablet for tablet e4d8a1326e854e51b5539e7addc91fe5 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:59:20.683300  1526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4d8a1326e854e51b5539e7addc91fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.689718  1533 tablet_service.cc:1511] Processing CreateTablet for tablet 81301f84b53f41349ac6bca78f1a77f4 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:59:20.695801  1602 tablet_service.cc:1511] Processing CreateTablet for tablet fb767ced66f24d579d3a9c116606ef2f (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:59:20.677246  1676 tablet_service.cc:1511] Processing CreateTablet for tablet 88cbab659ea041148748114fae549ae0 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:59:20.697243  1676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88cbab659ea041148748114fae549ae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.697827  1600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb1171fd6323498ea16dfea3e5356fe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.684136  1605 tablet_service.cc:1511] Processing CreateTablet for tablet 2f07ede48ffe4a5eb0b162da083502c2 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:59:20.700752  1605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f07ede48ffe4a5eb0b162da083502c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.694566  1531 tablet_service.cc:1511] Processing CreateTablet for tablet e0058494c1f14737a2e0e2b8f4697969 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:59:20.701251  1533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81301f84b53f41349ac6bca78f1a77f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.701725  1607 tablet_service.cc:1511] Processing CreateTablet for tablet b7e984cd665a4768933ff09e8ba0e07b (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:59:20.710697  1607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7e984cd665a4768933ff09e8ba0e07b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.715787  1524 tablet_service.cc:1511] Processing CreateTablet for tablet ebc9ea99300141e883fcf2f1b0d7a744 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
W20260709 12:59:20.721462  1489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:20.723860  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 12:59:20.725520  1658 leader_election.cc:343] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.725574  1673 tablet_service.cc:1511] Processing CreateTablet for tablet ebc9ea99300141e883fcf2f1b0d7a744 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:59:20.726128  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:20.727123  1673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc9ea99300141e883fcf2f1b0d7a744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.727283  1733 tablet_bootstrap.cc:654] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.727336  1738 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.729645  1531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0058494c1f14737a2e0e2b8f4697969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.661051  1603 tablet_service.cc:1511] Processing CreateTablet for tablet a35934c9d0d04fbeac6fa3a58fa8fd63 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:59:20.733496  1524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ebc9ea99300141e883fcf2f1b0d7a744. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.733542  1602 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb767ced66f24d579d3a9c116606ef2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.750346  1739 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.750864  1739 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:20.752997  1739 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:20.754698  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:20.755553  1529 tablet_service.cc:1511] Processing CreateTablet for tablet bb1171fd6323498ea16dfea3e5356fe0 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:59:20.756893  1529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb1171fd6323498ea16dfea3e5356fe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.756815  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:20.693302  1527 tablet_service.cc:1511] Processing CreateTablet for tablet 88cbab659ea041148748114fae549ae0 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
W20260709 12:59:20.758682  1584 leader_election.cc:343] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.759131  1527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88cbab659ea041148748114fae549ae0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:20.759562  1582 leader_election.cc:343] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.759991  1582 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:20.761175  1739 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.749507  1734 tablet_bootstrap.cc:700] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Time spent opening tablet: real 0.113s	user 0.003s	sys 0.000s
I20260709 12:59:20.762032  1532 tablet_service.cc:1511] Processing CreateTablet for tablet 59d952214bc24204a149e6f1a9af3184 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:59:20.763255  1734 tablet_bootstrap.cc:654] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.764006  1532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d952214bc24204a149e6f1a9af3184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.777498  1603 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a35934c9d0d04fbeac6fa3a58fa8fd63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.778699  1604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65847e774e445909d83c69bf486b7fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.780357  1528 tablet_service.cc:1511] Processing CreateTablet for tablet 22357a3949344a1792e5bae4a0291987 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:59:20.780462  1681 tablet_service.cc:1511] Processing CreateTablet for tablet 59d952214bc24204a149e6f1a9af3184 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:59:20.781873  1528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22357a3949344a1792e5bae4a0291987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.786479  1525 tablet_service.cc:1511] Processing CreateTablet for tablet a35934c9d0d04fbeac6fa3a58fa8fd63 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:59:20.790356  1525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a35934c9d0d04fbeac6fa3a58fa8fd63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.799063  1608 tablet_service.cc:1511] Processing CreateTablet for tablet c8377b7234e4472ea27005f16dede669 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:59:20.800493  1608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8377b7234e4472ea27005f16dede669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.781873  1681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d952214bc24204a149e6f1a9af3184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.806869  1526 tablet_service.cc:1511] Processing CreateTablet for tablet c65847e774e445909d83c69bf486b7fc (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:59:20.808298  1526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65847e774e445909d83c69bf486b7fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.777298  1599 tablet_service.cc:1511] Processing CreateTablet for tablet e1ef0a9c8bca48b6898df70834e97c4c (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:59:20.818894  1675 tablet_service.cc:1511] Processing CreateTablet for tablet bb1171fd6323498ea16dfea3e5356fe0 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:59:20.837287  1740 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.837836  1740 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:20.838332  1732 tablet_bootstrap.cc:492] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:20.838858  1732 ts_tablet_manager.cc:1403] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.326s	user 0.028s	sys 0.006s
I20260709 12:59:20.839910  1740 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:20.841255  1675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb1171fd6323498ea16dfea3e5356fe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.842288  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:20.842674  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 12:59:20.843825  1507 leader_election.cc:343] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:20.844444  1509 leader_election.cc:343] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.847304  1732 raft_consensus.cc:359] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:20.848096  1732 raft_consensus.cc:385] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.847110  1509 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:20.852905  1740 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.854700  1606 tablet_service.cc:1511] Processing CreateTablet for tablet c601f0c0dfb54023907d2722495668cb (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:59:20.862915  1678 tablet_service.cc:1511] Processing CreateTablet for tablet 2f07ede48ffe4a5eb0b162da083502c2 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:59:20.849066  1530 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54754 (request call id 32) took 1045 ms. Trace:
I20260709 12:59:20.859229  1599 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ef0a9c8bca48b6898df70834e97c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.850952  1732 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.866007  1530 rpcz_store.cc:276] 0709 12:59:19.803521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:20.294887 (+491366us) service_pool.cc:224] Handling call
0709 12:59:20.298367 (+  3480us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:59:20.298519 (+   152us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:59:20.651434 (+352915us) tablet_metadata.cc:665] Metadata flushed
0709 12:59:20.831695 (+180261us) tablet_replica.cc:206] Creating consensus instance
0709 12:59:20.849003 (+ 17308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:59:20.869854  1733 tablet_bootstrap.cc:492] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:20.870539  1733 ts_tablet_manager.cc:1403] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.278s	user 0.009s	sys 0.015s
I20260709 12:59:20.871418  1601 tablet_service.cc:1511] Processing CreateTablet for tablet 21b84a6a3d5b4e8b95867a87260513a1 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:59:20.860311  1677 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40104 (request call id 31) took 1080 ms. Trace:
I20260709 12:59:20.873744  1733 raft_consensus.cc:359] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:20.873961  1732 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:20.873143  1677 rpcz_store.cc:276] 0709 12:59:19.778771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:20.196056 (+417285us) service_pool.cc:224] Handling call
0709 12:59:20.206600 (+ 10544us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:59:20.206692 (+    92us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:59:20.742873 (+536181us) tablet_metadata.cc:665] Metadata flushed
0709 12:59:20.834389 (+ 91516us) tablet_replica.cc:206] Creating consensus instance
0709 12:59:20.859532 (+ 25143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:59:20.877761  1606 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c601f0c0dfb54023907d2722495668cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.878266  1531 tablet_service.cc:1511] Processing CreateTablet for tablet fb767ced66f24d579d3a9c116606ef2f (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:59:20.879070  1533 tablet_service.cc:1511] Processing CreateTablet for tablet 2f07ede48ffe4a5eb0b162da083502c2 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:59:20.880226  1530 tablet_service.cc:1511] Processing CreateTablet for tablet b7e984cd665a4768933ff09e8ba0e07b (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:59:20.884920  1528 tablet_service.cc:1511] Processing CreateTablet for tablet e1ef0a9c8bca48b6898df70834e97c4c (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:59:20.885073  1604 tablet_service.cc:1511] Processing CreateTablet for tablet afa53a120a654b379c23e60544c86b5e (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:59:20.853884  1679 tablet_service.cc:1511] Processing CreateTablet for tablet fb767ced66f24d579d3a9c116606ef2f (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:59:20.902576  1677 tablet_service.cc:1511] Processing CreateTablet for tablet e1ef0a9c8bca48b6898df70834e97c4c (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:59:20.889470  1676 tablet_service.cc:1511] Processing CreateTablet for tablet c65847e774e445909d83c69bf486b7fc (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:59:20.852697  1674 tablet_service.cc:1511] Processing CreateTablet for tablet 22357a3949344a1792e5bae4a0291987 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:59:20.891430  1680 tablet_service.cc:1511] Processing CreateTablet for tablet a35934c9d0d04fbeac6fa3a58fa8fd63 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:59:20.905062  1732 ts_tablet_manager.cc:1434] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.066s	user 0.008s	sys 0.000s
I20260709 12:59:20.906292  1732 tablet_bootstrap.cc:492] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:20.892609  1678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f07ede48ffe4a5eb0b162da083502c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.882268  1607 tablet_service.cc:1511] Processing CreateTablet for tablet 63835fd2de1c4dd4b0cea71fe7f09ed5 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:59:20.901368  1530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7e984cd665a4768933ff09e8ba0e07b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.910183  1600 tablet_service.cc:1511] Processing CreateTablet for tablet 67b2c0e67e8f4d0c9f640a2ce4fb6041 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:59:20.911810  1533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f07ede48ffe4a5eb0b162da083502c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.916366  1532 tablet_service.cc:1511] Processing CreateTablet for tablet c8377b7234e4472ea27005f16dede669 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:59:20.921927  1604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afa53a120a654b379c23e60544c86b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.922668  1531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb767ced66f24d579d3a9c116606ef2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.925895  1679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb767ced66f24d579d3a9c116606ef2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.929664  1677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ef0a9c8bca48b6898df70834e97c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.929840  1601 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b84a6a3d5b4e8b95867a87260513a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.933774  1600 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67b2c0e67e8f4d0c9f640a2ce4fb6041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.934489  1674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22357a3949344a1792e5bae4a0291987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.890182  1733 raft_consensus.cc:385] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:20.929823  1532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8377b7234e4472ea27005f16dede669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.942314  1607 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63835fd2de1c4dd4b0cea71fe7f09ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.944882  1738 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:20.945570  1680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a35934c9d0d04fbeac6fa3a58fa8fd63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.945413  1738 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 12:59:20.947510  1738 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:20.949162  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:20.949483  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 12:59:20.950837  1658 leader_election.cc:343] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:20.951530  1658 leader_election.cc:343] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:20.951937  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:20.952641  1732 tablet_bootstrap.cc:654] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:20.953094  1738 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:20.938290  1733 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:20.967764  1528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ef0a9c8bca48b6898df70834e97c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.969472  1676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65847e774e445909d83c69bf486b7fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.964428  1682 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40104 (request call id 34) took 1079 ms. Trace:
I20260709 12:59:20.981130  1527 tablet_service.cc:1511] Processing CreateTablet for tablet 63835fd2de1c4dd4b0cea71fe7f09ed5 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:59:20.981513  1682 rpcz_store.cc:276] 0709 12:59:19.884040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:20.201955 (+317915us) service_pool.cc:224] Handling call
0709 12:59:20.203835 (+  1880us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:59:20.203934 (+    99us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:59:20.815169 (+611235us) tablet_metadata.cc:665] Metadata flushed
0709 12:59:20.888343 (+ 73174us) tablet_replica.cc:206] Creating consensus instance
0709 12:59:20.963125 (+ 74782us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:59:20.983644  1681 tablet_service.cc:1511] Processing CreateTablet for tablet b7e984cd665a4768933ff09e8ba0e07b (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:59:20.987758  1681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7e984cd665a4768933ff09e8ba0e07b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.985700  1527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63835fd2de1c4dd4b0cea71fe7f09ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:20.984467  1682 tablet_service.cc:1511] Processing CreateTablet for tablet c8377b7234e4472ea27005f16dede669 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:59:20.998574  1673 tablet_service.cc:1511] Processing CreateTablet for tablet 63835fd2de1c4dd4b0cea71fe7f09ed5 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:59:20.999413  1524 tablet_service.cc:1511] Processing CreateTablet for tablet 67b2c0e67e8f4d0c9f640a2ce4fb6041 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:59:21.004360  1677 tablet_service.cc:1511] Processing CreateTablet for tablet c601f0c0dfb54023907d2722495668cb (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:59:21.005384  1739 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.010089  1673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63835fd2de1c4dd4b0cea71fe7f09ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.009665  1739 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.014670  1739 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.017266  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:21.018193  1692 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:21.019548  1582 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:21.020481  1739 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.022745  1739 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.023201  1739 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.022398  1732 tablet_bootstrap.cc:492] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:21.022285  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.029927  1732 ts_tablet_manager.cc:1403] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.124s	user 0.012s	sys 0.003s
I20260709 12:59:21.032805  1733 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.033798  1738 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.034047  1732 raft_consensus.cc:359] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 12:59:21.021970  1524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67b2c0e67e8f4d0c9f640a2ce4fb6041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.025044  1526 tablet_service.cc:1511] Processing CreateTablet for tablet afa53a120a654b379c23e60544c86b5e (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:59:21.037603  1732 raft_consensus.cc:385] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.038152  1732 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.034277  1738 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.038456  1529 tablet_service.cc:1511] Processing CreateTablet for tablet c601f0c0dfb54023907d2722495668cb (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:59:21.039952  1529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c601f0c0dfb54023907d2722495668cb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:59:21.030854  1584 leader_election.cc:343] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.042217  1526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afa53a120a654b379c23e60544c86b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.043241  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.043860  1738 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 12:59:21.044579  1658 leader_election.cc:343] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.029852  1532 tablet_service.cc:1511] Processing CreateTablet for tablet 21b84a6a3d5b4e8b95867a87260513a1 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:59:21.045325  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 12:59:21.048250  1658 leader_election.cc:343] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.048796  1733 ts_tablet_manager.cc:1434] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.178s	user 0.007s	sys 0.000s
I20260709 12:59:21.049901  1733 tablet_bootstrap.cc:492] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:21.048714  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:21.049829  1678 tablet_service.cc:1511] Processing CreateTablet for tablet 67b2c0e67e8f4d0c9f640a2ce4fb6041 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:59:21.050705  1738 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:21.057894  1734 tablet_bootstrap.cc:492] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:21.058471  1734 ts_tablet_manager.cc:1403] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.500s	user 0.040s	sys 0.024s
I20260709 12:59:21.061657  1734 raft_consensus.cc:359] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.062407  1734 raft_consensus.cc:385] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.062711  1734 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.063450  1734 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.067977  1677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c601f0c0dfb54023907d2722495668cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.070787  1678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67b2c0e67e8f4d0c9f640a2ce4fb6041. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.073730  1682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8377b7234e4472ea27005f16dede669. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.078449  1679 tablet_service.cc:1511] Processing CreateTablet for tablet 21b84a6a3d5b4e8b95867a87260513a1 (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:59:21.080032  1679 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b84a6a3d5b4e8b95867a87260513a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.082129  1739 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.082481  1732 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 12:59:21.085325  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:21.086407  1543 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.086550  1734 ts_tablet_manager.cc:1434] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:59:21.087114  1532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21b84a6a3d5b4e8b95867a87260513a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.087625  1734 tablet_bootstrap.cc:492] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:21.089936  1733 tablet_bootstrap.cc:654] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.097779  1674 tablet_service.cc:1511] Processing CreateTablet for tablet afa53a120a654b379c23e60544c86b5e (DEFAULT_TABLE table=test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:59:21.110692  1674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afa53a120a654b379c23e60544c86b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:21.119182  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:21.119815  1692 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.105732  1543 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:21.125674  1739 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.129594  1584 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:21.130322  1692 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:21.131134  1734 tablet_bootstrap.cc:654] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.146140  1739 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.146775  1739 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:21.148368  1739 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.161733  1732 ts_tablet_manager.cc:1434] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.130s	user 0.018s	sys 0.012s
I20260709 12:59:21.162639  1675 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:40104 (request call id 45) took 1094 ms. Trace:
I20260709 12:59:21.162801  1732 tablet_bootstrap.cc:492] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:21.163066  1675 rpcz_store.cc:276] 0709 12:59:20.068176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:59:20.804630 (+736454us) service_pool.cc:224] Handling call
0709 12:59:20.834722 (+ 30092us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:59:20.840855 (+  6133us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:59:21.056310 (+215455us) tablet_metadata.cc:665] Metadata flushed
0709 12:59:21.160343 (+104033us) tablet_replica.cc:206] Creating consensus instance
0709 12:59:21.162583 (+  2240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:59:21.175992  1733 tablet_bootstrap.cc:492] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:21.176496  1733 ts_tablet_manager.cc:1403] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.127s	user 0.014s	sys 0.003s
I20260709 12:59:21.178839  1733 raft_consensus.cc:359] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.180150  1733 raft_consensus.cc:385] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.180675  1733 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.181674  1733 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.182487  1732 tablet_bootstrap.cc:654] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.183566  1440 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:21.187171  1733 ts_tablet_manager.cc:1434] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:59:21.190220  1733 tablet_bootstrap.cc:492] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:21.192122  1734 tablet_bootstrap.cc:492] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:21.192651  1734 ts_tablet_manager.cc:1403] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.105s	user 0.014s	sys 0.010s
I20260709 12:59:21.194864  1734 raft_consensus.cc:359] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.195562  1734 raft_consensus.cc:385] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.195854  1734 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.197615  1733 tablet_bootstrap.cc:654] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.198297  1734 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.201030  1734 ts_tablet_manager.cc:1434] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:59:21.202229  1734 tablet_bootstrap.cc:492] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:21.205240  1732 tablet_bootstrap.cc:492] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:21.206005  1732 ts_tablet_manager.cc:1403] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.006s
I20260709 12:59:21.209753  1734 tablet_bootstrap.cc:654] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.209453  1732 raft_consensus.cc:359] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.210186  1732 raft_consensus.cc:385] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.210522  1732 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.211390  1732 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.213706  1732 ts_tablet_manager.cc:1434] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 12:59:21.214658  1732 tablet_bootstrap.cc:492] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:21.216320  1734 tablet_bootstrap.cc:492] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:21.216801  1734 ts_tablet_manager.cc:1403] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.015s	user 0.001s	sys 0.011s
I20260709 12:59:21.216905  1733 tablet_bootstrap.cc:492] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:21.217373  1733 ts_tablet_manager.cc:1403] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.005s
I20260709 12:59:21.220191  1734 raft_consensus.cc:359] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.220940  1734 raft_consensus.cc:385] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.221253  1734 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.221515  1733 raft_consensus.cc:359] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.222254  1733 raft_consensus.cc:385] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.222139  1734 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.222606  1733 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.223487  1733 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.226130  1733 ts_tablet_manager.cc:1434] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:59:21.226400  1732 tablet_bootstrap.cc:654] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.227124  1733 tablet_bootstrap.cc:492] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:21.227596  1734 ts_tablet_manager.cc:1434] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:59:21.228585  1734 tablet_bootstrap.cc:492] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:21.234879  1733 tablet_bootstrap.cc:654] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.237298  1734 tablet_bootstrap.cc:654] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.237995  1763 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.238679  1763 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 12:59:21.242332  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.242645  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:21.243312  1692 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:21.243886  1732 tablet_bootstrap.cc:492] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
W20260709 12:59:21.244200  1584 leader_election.cc:343] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.244379  1732 ts_tablet_manager.cc:1403] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.011s
I20260709 12:59:21.244771  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:21.246039  1739 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.246546  1739 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.247005  1739 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.247519  1732 raft_consensus.cc:359] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.247931  1763 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.248385  1732 raft_consensus.cc:385] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.248723  1732 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.249521  1732 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.251317  1732 ts_tablet_manager.cc:1434] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:59:21.252206  1732 tablet_bootstrap.cc:492] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:21.258170  1732 tablet_bootstrap.cc:654] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.257961  1739 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 12:59:21.261850  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:21.262720  1692 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.263410  1763 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.263890  1763 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.265832  1761 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.266312  1761 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.268913  1761 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.261852  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:21.271015  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.271685  1542 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
W20260709 12:59:21.272912  1584 leader_election.cc:343] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.273672  1584 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:21.275310  1761 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.275677  1761 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.276012  1761 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.276232  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:21.277174  1691 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:21.279899  1734 tablet_bootstrap.cc:492] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:21.280051  1739 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.280853  1734 ts_tablet_manager.cc:1403] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.052s	user 0.013s	sys 0.001s
I20260709 12:59:21.282199  1692 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:21.282667  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.283339  1542 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:21.283476  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:21.284600  1739 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.284610  1584 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:21.286864  1734 raft_consensus.cc:359] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.287453  1763 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.287670  1734 raft_consensus.cc:385] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.288009  1734 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.288172  1763 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.289327  1734 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.291445  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 12:59:21.292824  1582 leader_election.cc:343] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.293576  1733 tablet_bootstrap.cc:492] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:21.293722  1763 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.294262  1763 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.294579  1733 ts_tablet_manager.cc:1403] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.068s	user 0.022s	sys 0.012s
I20260709 12:59:21.296799  1761 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.301167  1734 ts_tablet_manager.cc:1434] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 12:59:21.301574  1733 raft_consensus.cc:359] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.302680  1733 raft_consensus.cc:385] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.303059  1733 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.304064  1733 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.307255  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:21.307756  1733 ts_tablet_manager.cc:1434] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 12:59:21.307888  1692 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.309275  1733 tablet_bootstrap.cc:492] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:21.310678  1761 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.312606  1732 tablet_bootstrap.cc:492] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:21.313139  1732 ts_tablet_manager.cc:1403] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.061s	user 0.040s	sys 0.016s
I20260709 12:59:21.314299  1739 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:21.315184  1739 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 12:59:21.316372  1733 tablet_bootstrap.cc:654] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.316651  1732 raft_consensus.cc:359] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.317386  1732 raft_consensus.cc:385] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.317740  1732 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.318432  1732 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.321171  1732 ts_tablet_manager.cc:1434] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:59:21.322530  1732 tablet_bootstrap.cc:492] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:21.335916  1734 tablet_bootstrap.cc:492] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:21.342559  1732 tablet_bootstrap.cc:654] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.345103  1692 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:21.338094  1763 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.340034  1733 tablet_bootstrap.cc:492] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:21.346294  1733 ts_tablet_manager.cc:1403] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.037s	user 0.011s	sys 0.020s
I20260709 12:59:21.346302  1582 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:21.349887  1733 raft_consensus.cc:359] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.350687  1733 raft_consensus.cc:385] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.350879  1734 tablet_bootstrap.cc:654] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.350508  1777 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.351102  1733 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
W20260709 12:59:21.340412  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 55893us
I20260709 12:59:21.351943  1777 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:21.353345  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:21.353199  1777 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.353936  1542 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.351943  1733 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.360582  1732 tablet_bootstrap.cc:492] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:21.361227  1732 ts_tablet_manager.cc:1403] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.039s	user 0.023s	sys 0.005s
I20260709 12:59:21.362293  1542 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:21.364388  1732 raft_consensus.cc:359] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.365097  1732 raft_consensus.cc:385] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.365270  1733 ts_tablet_manager.cc:1434] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.002s
I20260709 12:59:21.366597  1733 tablet_bootstrap.cc:492] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:21.366858  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:21.367548  1542 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.368438  1763 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.369213  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 12:59:21.370522  1582 leader_election.cc:343] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.374634  1733 tablet_bootstrap.cc:654] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.365360  1732 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.375988  1732 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.378089  1732 ts_tablet_manager.cc:1434] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:59:21.379009  1732 tablet_bootstrap.cc:492] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:21.379709  1733 tablet_bootstrap.cc:492] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:21.380155  1733 ts_tablet_manager.cc:1403] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.001s
I20260709 12:59:21.377280  1441 catalog_manager.cc:5697] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:21.383474  1542 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:21.383350  1733 raft_consensus.cc:359] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.384065  1733 raft_consensus.cc:385] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.384341  1733 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.384914  1732 tablet_bootstrap.cc:654] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.384859  1584 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 12:59:21.385016  1733 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.386708  1763 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.387187  1763 raft_consensus.cc:697] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:21.387423  1733 ts_tablet_manager.cc:1434] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:59:21.388084  1734 tablet_bootstrap.cc:492] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:21.388347  1733 tablet_bootstrap.cc:492] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:21.387989  1763 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.388988  1734 ts_tablet_manager.cc:1403] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.053s	user 0.017s	sys 0.005s
I20260709 12:59:21.391990  1734 raft_consensus.cc:359] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 12:59:21.392714  1734 raft_consensus.cc:385] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.393009  1734 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.395118  1733 tablet_bootstrap.cc:654] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.397575  1734 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 12:59:21.399981  1734 ts_tablet_manager.cc:1434] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:59:21.408232  1734 tablet_bootstrap.cc:492] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:21.408587  1439 catalog_manager.cc:5697] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:21.411572  1733 tablet_bootstrap.cc:492] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:21.412170  1733 ts_tablet_manager.cc:1403] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.016s
I20260709 12:59:21.415663  1733 raft_consensus.cc:359] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.416443  1733 raft_consensus.cc:385] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.416778  1733 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.417718  1732 tablet_bootstrap.cc:492] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:21.417536  1733 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.418179  1732 ts_tablet_manager.cc:1403] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.039s	user 0.006s	sys 0.010s
I20260709 12:59:21.419790  1733 ts_tablet_manager.cc:1434] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:21.421007  1733 tablet_bootstrap.cc:492] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:21.423243  1734 tablet_bootstrap.cc:654] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.424665  1732 raft_consensus.cc:359] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.425365  1732 raft_consensus.cc:385] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.425685  1732 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.426362  1732 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.428002  1733 tablet_bootstrap.cc:654] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.428366  1732 ts_tablet_manager.cc:1434] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:59:21.429281  1732 tablet_bootstrap.cc:492] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:21.433787  1734 tablet_bootstrap.cc:492] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:21.434443  1734 ts_tablet_manager.cc:1403] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.000s
I20260709 12:59:21.435865  1732 tablet_bootstrap.cc:654] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.428790  1441 catalog_manager.cc:5697] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:21.438084  1734 raft_consensus.cc:359] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.438802  1734 raft_consensus.cc:385] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.439049  1734 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.440970  1734 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.442132  1733 tablet_bootstrap.cc:492] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:21.442651  1733 ts_tablet_manager.cc:1403] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260709 12:59:21.445744  1734 ts_tablet_manager.cc:1434] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:59:21.445389  1733 raft_consensus.cc:359] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.446197  1733 raft_consensus.cc:385] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.446522  1733 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.447110  1760 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.447388  1733 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.448052  1760 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.450353  1733 ts_tablet_manager.cc:1434] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:59:21.451382  1733 tablet_bootstrap.cc:492] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:21.451184  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.451807  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:21.452158  1542 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:21.452452  1618 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:21.453788  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:21.454993  1738 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.455425  1738 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.455799  1738 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.458436  1733 tablet_bootstrap.cc:654] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.461547  1760 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:21.466684  1738 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.470187  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:21.470568  1733 tablet_bootstrap.cc:492] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:21.470821  1542 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.471669  1733 ts_tablet_manager.cc:1403] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.011s
I20260709 12:59:21.474611  1733 raft_consensus.cc:359] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.475379  1733 raft_consensus.cc:385] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.475716  1733 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.476431  1733 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.479197  1733 ts_tablet_manager.cc:1434] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:21.479359  1542 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:21.480383  1733 tablet_bootstrap.cc:492] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:21.480618  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:21.479280  1734 tablet_bootstrap.cc:492] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:21.481455  1760 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.491854  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:21.492810  1618 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.496682  1733 tablet_bootstrap.cc:654] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.498692  1760 raft_consensus.cc:697] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:21.499747  1760 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.491966  1738 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:21.489385  1734 tablet_bootstrap.cc:654] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.501151  1618 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:21.513840  1732 tablet_bootstrap.cc:492] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:21.513922  1734 tablet_bootstrap.cc:492] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:21.514572  1732 ts_tablet_manager.cc:1403] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.085s	user 0.036s	sys 0.044s
I20260709 12:59:21.514649  1734 ts_tablet_manager.cc:1403] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.068s	user 0.012s	sys 0.006s
I20260709 12:59:21.518486  1732 raft_consensus.cc:359] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.518815  1734 raft_consensus.cc:359] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.519467  1732 raft_consensus.cc:385] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.519738  1734 raft_consensus.cc:385] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.519888  1732 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.520205  1734 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.521334  1734 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.522157  1732 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.524272  1733 tablet_bootstrap.cc:492] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:21.528057  1733 ts_tablet_manager.cc:1403] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.003s
I20260709 12:59:21.530784  1733 raft_consensus.cc:359] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.531577  1733 raft_consensus.cc:385] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.530874  1439 catalog_manager.cc:5697] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:21.531667  1760 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.531888  1733 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.532226  1760 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.532799  1733 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.524298  1732 ts_tablet_manager.cc:1434] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:59:21.526785  1763 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.534667  1763 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.535745  1733 ts_tablet_manager.cc:1434] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:59:21.525801  1734 ts_tablet_manager.cc:1434] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 12:59:21.536446  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.537803  1763 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 12:59:21.537943  1658 leader_election.cc:343] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.538178  1734 tablet_bootstrap.cc:492] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:21.538970  1732 tablet_bootstrap.cc:492] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:21.540454  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:21.541075  1760 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:21.542153  1733 tablet_bootstrap.cc:492] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:21.542277  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:21.541085  1692 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
W20260709 12:59:21.545707  1658 leader_election.cc:343] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.546166  1658 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:21.547065  1732 tablet_bootstrap.cc:654] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.547252  1760 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:21.549561  1740 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.550490  1734 tablet_bootstrap.cc:654] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.550220  1740 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.538779  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.549260  1582 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:21.556844  1733 tablet_bootstrap.cc:654] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.557669  1763 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.558511  1763 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.558852  1763 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.558972  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 12:59:21.560953  1509 leader_election.cc:343] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.554072  1542 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:21.572059  1739 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:21.572094  1740 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.577948  1760 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.578917  1760 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.577930  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:21.580871  1692 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:21.582237  1760 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:21.584484  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.585098  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:21.585391  1542 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:21.585999  1618 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:21.587460  1507 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:21.587755  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:21.589154  1760 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.589543  1740 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.589797  1760 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.590173  1740 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.590540  1760 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.590734  1740 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.596108  1763 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.600428  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:21.601275  1542 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.601405  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:21.602300  1692 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.611912  1542 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:21.614997  1732 tablet_bootstrap.cc:492] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:21.615578  1732 ts_tablet_manager.cc:1403] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.077s	user 0.018s	sys 0.005s
I20260709 12:59:21.616343  1738 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.616830  1738 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.618297  1732 raft_consensus.cc:359] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.618963  1732 raft_consensus.cc:385] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.619334  1732 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.618896  1738 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:21.620087  1732 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.620779  1760 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.620975  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.622229  1732 ts_tablet_manager.cc:1434] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 12:59:21.623134  1732 tablet_bootstrap.cc:492] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:21.623713  1734 tablet_bootstrap.cc:492] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:21.624154  1734 ts_tablet_manager.cc:1403] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.086s	user 0.019s	sys 0.035s
I20260709 12:59:21.625170  1584 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:21.626114  1739 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.626574  1739 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:21.626876  1734 raft_consensus.cc:359] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.627537  1734 raft_consensus.cc:385] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.627808  1734 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.627362  1739 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.628157  1740 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:21.629110  1658 leader_election.cc:343] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.629956  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:21.630398  1732 tablet_bootstrap.cc:654] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.630571  1618 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:21.630717  1763 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.630852  1734 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.631724  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 12:59:21.633571  1509 leader_election.cc:343] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.635937  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:21.637355  1760 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:21.638132  1760 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.638435  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:21.638584  1760 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.638890  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:21.639086  1543 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.639566  1617 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.655793  1543 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:21.657471  1777 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:21.658994  1740 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.634989  1692 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:21.661366  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:21.663760  1738 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.664354  1738 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:21.665385  1738 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.662587  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:21.669350  1690 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.639083  1760 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.686178  1760 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.686841  1732 tablet_bootstrap.cc:492] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:21.687314  1732 ts_tablet_manager.cc:1403] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.064s	user 0.014s	sys 0.016s
I20260709 12:59:21.689951  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 12:59:21.692221  1658 leader_election.cc:343] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.690001  1732 raft_consensus.cc:359] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.694974  1732 raft_consensus.cc:385] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.695257  1732 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.697731  1732 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.698525  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:21.699090  1618 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.699913  1732 ts_tablet_manager.cc:1434] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:59:21.700846  1732 tablet_bootstrap.cc:492] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:21.706898  1439 catalog_manager.cc:5697] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:21.708787  1618 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:21.709632  1760 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:21.711745  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:21.713721  1760 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.714203  1760 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:21.714735  1732 tablet_bootstrap.cc:654] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.715066  1760 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.718152  1777 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.718907  1777 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.723170  1740 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.723822  1740 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 12:59:21.726855  1732 tablet_bootstrap.cc:492] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:21.727015  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.727365  1732 ts_tablet_manager.cc:1403] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.002s
I20260709 12:59:21.728281  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 12:59:21.728533  1584 leader_election.cc:343] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.729125  1691 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 12:59:21.729534  1740 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 12:59:21.728643  1489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:21.730093  1732 raft_consensus.cc:359] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.731009  1732 raft_consensus.cc:385] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.731439  1732 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.730849  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:21.732313  1732 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.733891  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:21.735584  1740 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.739542  1732 ts_tablet_manager.cc:1434] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:59:21.740871  1732 tablet_bootstrap.cc:492] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:21.750258  1732 tablet_bootstrap.cc:654] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.752686  1690 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:21.754253  1507 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:21.754751  1732 tablet_bootstrap.cc:492] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:21.755231  1732 ts_tablet_manager.cc:1403] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.005s
I20260709 12:59:21.754448  1760 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.757766  1733 tablet_bootstrap.cc:492] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:21.758234  1733 ts_tablet_manager.cc:1403] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.216s	user 0.062s	sys 0.062s
I20260709 12:59:21.758559  1732 raft_consensus.cc:359] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.759245  1732 raft_consensus.cc:385] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.759523  1732 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.759380  1438 catalog_manager.cc:5697] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:21.760236  1732 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.760758  1733 raft_consensus.cc:359] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.761405  1733 raft_consensus.cc:385] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.761883  1733 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.762320  1732 ts_tablet_manager.cc:1434] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:59:21.762676  1733 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.763207  1732 tablet_bootstrap.cc:492] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:21.757289  1760 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.764853  1733 ts_tablet_manager.cc:1434] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:59:21.765826  1733 tablet_bootstrap.cc:492] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:21.768229  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:21.770372  1732 tablet_bootstrap.cc:654] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.771833  1617 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:21.772571  1777 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.774458  1740 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:21.787096  1740 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.787781  1740 raft_consensus.cc:697] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:21.765300  1760 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:21.780143  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:21.788934  1740 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.781121  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.785226  1732 tablet_bootstrap.cc:492] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:21.790809  1732 ts_tablet_manager.cc:1403] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.008s
I20260709 12:59:21.791011  1542 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 12:59:21.792698  1582 leader_election.cc:343] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.785923  1733 tablet_bootstrap.cc:654] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.793988  1732 raft_consensus.cc:359] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.794755  1732 raft_consensus.cc:385] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.794411  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:21.795043  1732 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.795518  1760 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.795739  1732 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.793344  1582 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:21.797796  1732 ts_tablet_manager.cc:1434] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:59:21.797852  1763 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:21.798777  1732 tablet_bootstrap.cc:492] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:21.803318  1760 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:21.804633  1777 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.805107  1777 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.809188  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 12:59:21.811265  1584 leader_election.cc:343] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.812626  1732 tablet_bootstrap.cc:654] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.811820  1439 catalog_manager.cc:5697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:21.814132  1777 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.814759  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 12:59:21.818662  1582 leader_election.cc:343] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.819146  1582 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:21.820699  1777 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:21.821033  1733 tablet_bootstrap.cc:492] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:21.821607  1733 ts_tablet_manager.cc:1403] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.056s	user 0.012s	sys 0.002s
I20260709 12:59:21.824599  1733 raft_consensus.cc:359] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.825291  1733 raft_consensus.cc:385] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.825593  1733 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.826515  1733 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.829612  1733 ts_tablet_manager.cc:1434] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:59:21.830835  1733 tablet_bootstrap.cc:492] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:21.831137  1734 ts_tablet_manager.cc:1434] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.207s	user 0.055s	sys 0.081s
I20260709 12:59:21.832095  1734 tablet_bootstrap.cc:492] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:21.838773  1733 tablet_bootstrap.cc:654] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.840384  1732 tablet_bootstrap.cc:492] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:21.840847  1732 ts_tablet_manager.cc:1403] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.042s	user 0.008s	sys 0.014s
I20260709 12:59:21.842020  1734 tablet_bootstrap.cc:654] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.846771  1732 raft_consensus.cc:359] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.847505  1732 raft_consensus.cc:385] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.847784  1732 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.848587  1732 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.850951  1732 ts_tablet_manager.cc:1434] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260709 12:59:21.851902  1732 tablet_bootstrap.cc:492] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:21.853734  1812 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.854238  1812 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.857717  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:21.858342  1617 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:21.858666  1732 tablet_bootstrap.cc:654] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.860224  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:21.861223  1800 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.861812  1800 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.862253  1800 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.863600  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:21.864244  1690 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:21.864506  1763 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:21.866719  1812 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.869292  1763 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:21.872730  1734 tablet_bootstrap.cc:492] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:21.873219  1734 ts_tablet_manager.cc:1403] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.019s
I20260709 12:59:21.874609  1800 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.875919  1734 raft_consensus.cc:359] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.876631  1734 raft_consensus.cc:385] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:21.876113  1441 catalog_manager.cc:5697] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:21.876904  1734 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.877552  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:21.878108  1617 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.878587  1734 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.881204  1734 ts_tablet_manager.cc:1434] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.001s
I20260709 12:59:21.882448  1734 tablet_bootstrap.cc:492] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:21.882172  1763 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:21.891878  1763 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:21.894505  1617 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:21.897033  1733 tablet_bootstrap.cc:492] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:21.897743  1733 ts_tablet_manager.cc:1403] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.067s	user 0.018s	sys 0.010s
I20260709 12:59:21.906198  1760 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.900520  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:21.908286  1812 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.910223  1800 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.913102  1812 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.915364  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:21.916000  1689 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.908511  1740 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:21.921651  1777 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:21.925612  1812 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.925879  1777 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.928903  1777 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.907891  1760 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.930217  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.931200  1760 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:21.931401  1734 tablet_bootstrap.cc:654] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:21.937640  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:21.938522  1617 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:21.939627  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:21.939945  1509 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:21.940629  1812 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.940513  1688 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
W20260709 12:59:21.936612  1584 leader_election.cc:343] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.941102  1812 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.933051  1740 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:21.941731  1812 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.942132  1582 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:21.942740  1740 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.943570  1777 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:21.948745  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:21.949225  1760 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:21.949575  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:21.949719  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:21.945043  1777 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:21.955134  1777 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:21.955653  1739 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:59:21.958251  1507 leader_election.cc:343] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:21.959861  1658 leader_election.cc:343] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:21.960772  1658 leader_election.cc:343] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:21.961287  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:21.970068  1733 raft_consensus.cc:359] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:21.970098  1732 tablet_bootstrap.cc:492] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:21.971758  1733 raft_consensus.cc:385] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.043838  1732 ts_tablet_manager.cc:1403] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.192s	user 0.021s	sys 0.011s
I20260709 12:59:22.043826  1733 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:21.990351  1763 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.050992  1777 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.051488  1763 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.052157  1733 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.051345  1778 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:22.063503  1733 ts_tablet_manager.cc:1434] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.165s	user 0.013s	sys 0.015s
I20260709 12:59:22.063975  1763 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:21.968652  1760 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.064055  1812 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.068493  1738 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.068740  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:22.069272  1617 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.068986  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:22.070225  1690 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 1: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 12:59:22.071357  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:22.011646  1739 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.076839  1584 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:21.974501  1689 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:22.079591  1761 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.080513  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 12:59:22.095235  1507 leader_election.cc:343] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.101306  1734 tablet_bootstrap.cc:492] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:22.101990  1734 ts_tablet_manager.cc:1403] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.220s	user 0.019s	sys 0.000s
I20260709 12:59:22.103823  1761 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.108110  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:22.109098  1692 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.078573  1739 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.086457  1812 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:22.117311  1733 tablet_bootstrap.cc:492] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:22.116097  1734 raft_consensus.cc:359] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.119832  1734 raft_consensus.cc:385] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.120292  1734 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.124030  1812 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:22.124290  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 47558.3us
I20260709 12:59:22.124644  1812 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.126248  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:22.128430  1734 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.113572  1777 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:22.135454  1732 raft_consensus.cc:359] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.136574  1732 raft_consensus.cc:385] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.137045  1732 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.138550  1734 ts_tablet_manager.cc:1434] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.004s
I20260709 12:59:22.140197  1732 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.143998  1734 tablet_bootstrap.cc:492] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
W20260709 12:59:22.145008  1584 leader_election.cc:343] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.148396  1692 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:22.150246  1732 ts_tablet_manager.cc:1434] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.106s	user 0.010s	sys 0.002s
I20260709 12:59:22.151628  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:22.162554  1616 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 12:59:22.164171  1739 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:22.165783  1733 tablet_bootstrap.cc:654] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.152371  1617 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:22.145011  1812 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:22.158617  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:22.168946  1738 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.169734  1738 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.171255  1582 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:22.172047  1738 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:22.172230  1739 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.173573  1732 tablet_bootstrap.cc:492] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:22.172721  1739 raft_consensus.cc:697] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:22.175119  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:22.176157  1739 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.177640  1812 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.193710  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:22.194573  1617 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:22.196336  1812 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.204685  1509 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 12:59:22.206156  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:22.206949  1812 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.207435  1812 raft_consensus.cc:697] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:22.208119  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:22.208297  1812 consensus_queue.cc:237] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.190148  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:22.208994  1760 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.212845  1760 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.213359  1760 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.201098  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 12:59:22.222318  1658 leader_election.cc:343] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.231101  1792 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:59:22.239447  1739 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.240430  1739 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.244722  1739 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:22.245679  1739 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.245858  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:22.246152  1739 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.246552  1692 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:22.248211  1739 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:22.249135  1441 catalog_manager.cc:5697] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:22.254871  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:22.255550  1691 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:22.258550  1582 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:22.259407  1739 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.259833  1739 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.260257  1739 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:22.256158  1584 leader_election.cc:343] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.259102  1760 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.262115  1733 tablet_bootstrap.cc:492] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:22.262639  1733 ts_tablet_manager.cc:1403] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.147s	user 0.015s	sys 0.017s
I20260709 12:59:22.267164  1689 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 1: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 12:59:22.270848  1828 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:22.274713  1733 raft_consensus.cc:359] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.275396  1733 raft_consensus.cc:385] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.275663  1733 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.276925  1733 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.276374  1738 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.278146  1792 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.278410  1738 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.284497  1732 tablet_bootstrap.cc:654] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.283536  1738 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:22.289258  1760 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:22.290406  1829 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:59:22.296002  1734 tablet_bootstrap.cc:654] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.291553  1738 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:59:22.279330  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:22.300163  1438 catalog_manager.cc:5697] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:22.310843  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:22.294380  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:22.316833  1777 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.318907  1777 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.319303  1777 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:22.321244  1584 leader_election.cc:343] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.318400  1778 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.323868  1739 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.325994  1733 ts_tablet_manager.cc:1434] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.063s	user 0.004s	sys 0.012s
I20260709 12:59:22.336459  1733 tablet_bootstrap.cc:492] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:22.335311  1812 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:59:22.354290  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:22.354509  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:22.357740  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:22.358434  1617 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:22.355346  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:22.364100  1739 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:22.364339  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:22.361696  1777 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.370779  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:22.377218  1691 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.409209  1540 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:22.437776  1740 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:22.382655  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:22.382884  1441 catalog_manager.cc:5697] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:22.446727  1584 leader_election.cc:343] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.452662  1733 tablet_bootstrap.cc:654] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.375717  1778 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:22.457602  1658 leader_election.cc:343] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:22.458295  1658 leader_election.cc:343] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.463016  1691 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:22.464990  1582 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:22.470685  1761 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.471222  1761 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:22.471839  1777 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:22.472062  1761 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.484488  1836 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:59:22.524184  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:22.554937  1691 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.510136  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:22.569561  1732 tablet_bootstrap.cc:492] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:22.569664  1836 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:22.570130  1836 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:22.570510  1836 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.570127  1732 ts_tablet_manager.cc:1403] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.397s	user 0.021s	sys 0.019s
W20260709 12:59:22.567891  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 97647.6us
I20260709 12:59:22.575547  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:22.576359  1540 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:22.571184  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 79251.1us
I20260709 12:59:22.586799  1812 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.589062  1792 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.588038  1732 raft_consensus.cc:359] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.590194  1732 raft_consensus.cc:385] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.590586  1732 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.591032  1792 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.591349  1732 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.594066  1732 ts_tablet_manager.cc:1434] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:59:22.594607  1792 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:22.601714  1732 tablet_bootstrap.cc:492] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:22.622059  1540 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:22.641728  1734 tablet_bootstrap.cc:492] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:22.642258  1734 ts_tablet_manager.cc:1403] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.498s	user 0.047s	sys 0.049s
I20260709 12:59:22.650957  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:22.652809  1691 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:22.663600  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:22.665738  1777 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.673645  1733 tablet_bootstrap.cc:492] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:22.674953  1733 ts_tablet_manager.cc:1403] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.339s	user 0.016s	sys 0.013s
I20260709 12:59:22.666770  1441 catalog_manager.cc:5697] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:22.678807  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:22.679951  1618 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 1: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 12:59:22.681131  1777 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:22.682389  1777 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.685767  1617 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:22.661003  1740 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:59:22.690080  1734 raft_consensus.cc:359] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.690666  1734 raft_consensus.cc:385] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.690896  1734 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.691450  1734 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.694561  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 12:59:22.694837  1584 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:22.696352  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:22.697934  1738 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.701932  1792 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.702627  1792 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:22.703806  1792 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.704174  1836 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.782037  1733 raft_consensus.cc:359] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.783017  1733 raft_consensus.cc:385] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.783298  1733 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.790475  1778 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.791194  1778 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.796962  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:22.799278  1732 tablet_bootstrap.cc:654] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:22.800586  1584 leader_election.cc:343] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.802021  1738 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.802654  1778 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:22.803501  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 12:59:22.805269  1582 leader_election.cc:343] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.805877  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:22.806891  1828 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.810863  1828 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:22.811126  1836 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:22.825167  1733 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:22.827136  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 130867us
I20260709 12:59:22.830417  1734 ts_tablet_manager.cc:1434] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.188s	user 0.052s	sys 0.069s
I20260709 12:59:22.830999  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:22.832899  1734 tablet_bootstrap.cc:492] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:22.833863  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:22.834679  1617 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:22.840724  1852 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.843961  1734 tablet_bootstrap.cc:654] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.849658  1617 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:22.849931  1733 ts_tablet_manager.cc:1434] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.175s	user 0.005s	sys 0.004s
W20260709 12:59:22.850492  1658 leader_election.cc:343] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:22.851387  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:22.854450  1733 tablet_bootstrap.cc:492] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:22.860687  1732 tablet_bootstrap.cc:492] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:22.860541  1852 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:22.861202  1732 ts_tablet_manager.cc:1403] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.260s	user 0.011s	sys 0.006s
I20260709 12:59:22.861227  1852 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.866137  1836 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:22.866688  1836 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:22.866786  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:22.867547  1836 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.869520  1852 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:22.871032  1852 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:22.878139  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:22.886754  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:22.887964  1852 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:22.910796  1732 raft_consensus.cc:359] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.911538  1732 raft_consensus.cc:385] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.911847  1732 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.912585  1732 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.918280  1734 tablet_bootstrap.cc:492] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:22.920050  1777 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:22.923372  1734 ts_tablet_manager.cc:1403] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.091s	user 0.022s	sys 0.025s
W20260709 12:59:22.922524  1858 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:59:22.927263  1732 ts_tablet_manager.cc:1434] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.001s
I20260709 12:59:22.934206  1732 tablet_bootstrap.cc:492] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:22.946075  1734 raft_consensus.cc:359] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.947459  1734 raft_consensus.cc:385] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.947860  1734 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.948725  1734 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:22.960387  1489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:22.960510  1734 ts_tablet_manager.cc:1434] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 12:59:22.961724  1734 tablet_bootstrap.cc:492] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:22.953784  1438 catalog_manager.cc:5697] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:22.970129  1733 tablet_bootstrap.cc:654] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:22.979885  1584 consensus_peers.cc:597] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:22.985857  1732 tablet_bootstrap.cc:654] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:22.992015  1733 tablet_bootstrap.cc:492] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:22.993742  1733 ts_tablet_manager.cc:1403] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.139s	user 0.014s	sys 0.015s
I20260709 12:59:22.986150  1439 catalog_manager.cc:5697] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:22.996285  1733 raft_consensus.cc:359] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.996995  1733 raft_consensus.cc:385] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:22.997299  1733 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:22.998080  1733 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:22.999889  1733 ts_tablet_manager.cc:1434] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:59:22.992888  1734 tablet_bootstrap.cc:654] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.000669  1733 tablet_bootstrap.cc:492] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:23.003993  1732 tablet_bootstrap.cc:492] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:23.004552  1732 ts_tablet_manager.cc:1403] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.001s
I20260709 12:59:23.007486  1733 tablet_bootstrap.cc:654] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.008324  1732 raft_consensus.cc:359] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.009323  1732 raft_consensus.cc:385] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.009706  1732 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.020375  1732 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.023366  1732 ts_tablet_manager.cc:1434] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:59:23.025655  1732 tablet_bootstrap.cc:492] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:23.025732  1439 catalog_manager.cc:5697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:23.026748  1734 tablet_bootstrap.cc:492] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:23.027235  1734 ts_tablet_manager.cc:1403] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.066s	user 0.011s	sys 0.019s
I20260709 12:59:23.030426  1734 raft_consensus.cc:359] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.031112  1734 raft_consensus.cc:385] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.031366  1734 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.031996  1734 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.036639  1733 tablet_bootstrap.cc:492] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:23.037138  1733 ts_tablet_manager.cc:1403] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.037s	user 0.010s	sys 0.006s
I20260709 12:59:23.038631  1732 tablet_bootstrap.cc:654] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.040850  1734 ts_tablet_manager.cc:1434] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:59:23.041790  1734 tablet_bootstrap.cc:492] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:23.042207  1733 raft_consensus.cc:359] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.042917  1733 raft_consensus.cc:385] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.043219  1733 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.044041  1733 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.046685  1733 ts_tablet_manager.cc:1434] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:59:23.047714  1733 tablet_bootstrap.cc:492] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:23.049397  1734 tablet_bootstrap.cc:654] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.053874  1732 tablet_bootstrap.cc:492] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:23.054345  1732 ts_tablet_manager.cc:1403] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.008s
I20260709 12:59:23.063747  1732 raft_consensus.cc:359] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.064472  1732 raft_consensus.cc:385] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.064764  1732 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.065379  1733 tablet_bootstrap.cc:654] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.065623  1732 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.068091  1732 ts_tablet_manager.cc:1434] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.004s
I20260709 12:59:23.069018  1732 tablet_bootstrap.cc:492] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:23.079234  1829 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.088078  1740 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.088564  1733 tablet_bootstrap.cc:492] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:23.088601  1740 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.089534  1733 ts_tablet_manager.cc:1403] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.042s	user 0.008s	sys 0.014s
I20260709 12:59:23.091315  1732 tablet_bootstrap.cc:654] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.095577  1733 raft_consensus.cc:359] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.096274  1733 raft_consensus.cc:385] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.096519  1733 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.100010  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:23.107649  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:23.108768  1740 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.109103  1507 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:23.110432  1740 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.113381  1733 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.114594  1734 tablet_bootstrap.cc:492] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:23.115054  1734 ts_tablet_manager.cc:1403] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.073s	user 0.025s	sys 0.042s
I20260709 12:59:23.115634  1733 ts_tablet_manager.cc:1434] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.026s	user 0.000s	sys 0.004s
I20260709 12:59:23.116458  1828 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.117866  1732 tablet_bootstrap.cc:492] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:23.117524  1734 raft_consensus.cc:359] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.118278  1733 tablet_bootstrap.cc:492] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:23.118321  1732 ts_tablet_manager.cc:1403] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.049s	user 0.007s	sys 0.003s
I20260709 12:59:23.118243  1734 raft_consensus.cc:385] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.118829  1734 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.119603  1734 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.120918  1732 raft_consensus.cc:359] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.121624  1732 raft_consensus.cc:385] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.121896  1732 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.122509  1732 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.125450  1733 tablet_bootstrap.cc:654] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.126008  1734 ts_tablet_manager.cc:1434] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:59:23.127424  1734 tablet_bootstrap.cc:492] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:23.127938  1732 ts_tablet_manager.cc:1434] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:59:23.128882  1732 tablet_bootstrap.cc:492] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:23.135818  1734 tablet_bootstrap.cc:654] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.135941  1732 tablet_bootstrap.cc:654] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.148299  1540 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.151976  1836 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.152499  1836 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.154997  1836 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:23.156908  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:23.157689  1618 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:23.158936  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:23.156802  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:23.160075  1539 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:23.161330  1836 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.161828  1836 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.162165  1836 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.170173  1836 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.172773  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:23.173377  1539 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.182943  1539 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:23.184110  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:23.185612  1792 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.186101  1792 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:23.187124  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:23.186919  1792 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.187664  1618 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.196313  1618 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:23.198091  1836 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 12:59:23.215474  1658 consensus_peers.cc:597] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:23.216995  1440 catalog_manager.cc:5697] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:23.242906  1733 tablet_bootstrap.cc:492] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:23.243451  1733 ts_tablet_manager.cc:1403] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.125s	user 0.034s	sys 0.067s
I20260709 12:59:23.244102  1734 tablet_bootstrap.cc:492] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:23.244563  1734 ts_tablet_manager.cc:1403] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.117s	user 0.049s	sys 0.055s
I20260709 12:59:23.244777  1732 tablet_bootstrap.cc:492] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:23.245680  1732 ts_tablet_manager.cc:1403] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.117s	user 0.017s	sys 0.028s
I20260709 12:59:23.246114  1733 raft_consensus.cc:359] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.246801  1733 raft_consensus.cc:385] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.247083  1733 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.246994  1734 raft_consensus.cc:359] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.247663  1734 raft_consensus.cc:385] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.247988  1734 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.247797  1733 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.247815  1732 raft_consensus.cc:359] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.248667  1732 raft_consensus.cc:385] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.248968  1732 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.248840  1734 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.249886  1732 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.251103  1734 ts_tablet_manager.cc:1434] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:59:23.252296  1734 tablet_bootstrap.cc:492] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:23.252890  1733 ts_tablet_manager.cc:1434] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:59:23.253808  1733 tablet_bootstrap.cc:492] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:23.255784  1732 ts_tablet_manager.cc:1434] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.002s
I20260709 12:59:23.256690  1732 tablet_bootstrap.cc:492] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:23.258404  1734 tablet_bootstrap.cc:654] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.259825  1733 tablet_bootstrap.cc:654] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.264171  1732 tablet_bootstrap.cc:654] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.271085  1733 tablet_bootstrap.cc:492] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:23.271561  1733 ts_tablet_manager.cc:1403] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.002s
I20260709 12:59:23.275032  1734 tablet_bootstrap.cc:492] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:23.275439  1732 tablet_bootstrap.cc:492] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:23.276497  1732 ts_tablet_manager.cc:1403] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 12:59:23.277891  1733 raft_consensus.cc:359] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.278599  1733 raft_consensus.cc:385] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.278861  1733 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.279114  1732 raft_consensus.cc:359] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.279807  1732 raft_consensus.cc:385] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.279670  1733 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.280121  1732 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.275491  1734 ts_tablet_manager.cc:1403] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.004s
I20260709 12:59:23.282151  1733 ts_tablet_manager.cc:1434] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:59:23.283069  1733 tablet_bootstrap.cc:492] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:23.283258  1734 raft_consensus.cc:359] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.283720  1734 raft_consensus.cc:385] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.283999  1734 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.284461  1734 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.286098  1734 ts_tablet_manager.cc:1434] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:59:23.286875  1734 tablet_bootstrap.cc:492] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:23.289928  1733 tablet_bootstrap.cc:654] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.289919  1732 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.291452  1734 tablet_bootstrap.cc:654] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.292208  1732 ts_tablet_manager.cc:1434] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 12:59:23.297948  1732 tablet_bootstrap.cc:492] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:23.306325  1740 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.307060  1740 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.307415  1732 tablet_bootstrap.cc:654] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.310274  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:23.311684  1740 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.313439  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:23.314185  1618 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 12:59:23.316459  1509 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:23.318061  1740 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.324671  1732 tablet_bootstrap.cc:492] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:23.325311  1732 ts_tablet_manager.cc:1403] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.011s
I20260709 12:59:23.328909  1732 raft_consensus.cc:359] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.329937  1732 raft_consensus.cc:385] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.330288  1732 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.330591  1828 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.331085  1828 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.331374  1732 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.334941  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:23.336360  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:23.337250  1689 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:23.338846  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:23.336232  1828 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.342259  1828 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.342701  1828 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.343119  1828 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:23.340844  1584 leader_election.cc:343] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.348347  1732 ts_tablet_manager.cc:1434] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.001s
I20260709 12:59:23.349738  1732 tablet_bootstrap.cc:492] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:23.358153  1732 tablet_bootstrap.cc:654] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.359510  1792 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.360662  1792 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.364555  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:23.367851  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:23.368767  1618 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
W20260709 12:59:23.371829  1658 leader_election.cc:343] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.372771  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:23.373978  1836 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.374965  1836 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.375419  1836 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.378646  1853 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.385229  1792 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:23.389405  1739 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.389951  1739 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.390705  1853 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.391992  1739 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.394269  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:23.394565  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:23.394918  1689 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:23.395159  1540 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:23.396140  1582 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:23.397217  1739 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.397703  1739 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.398130  1739 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.402985  1777 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.403544  1777 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.405601  1777 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.408037  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:23.408195  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 12:59:23.412907  1582 leader_election.cc:343] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:23.421669  1584 leader_election.cc:343] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.422358  1584 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:23.423653  1829 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.426661  1864 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.427397  1864 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.432142  1540 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.432404  1864 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.432629  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:23.436637  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:23.438699  1507 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:23.439766  1732 tablet_bootstrap.cc:492] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:23.440243  1732 ts_tablet_manager.cc:1403] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.091s	user 0.015s	sys 0.008s
I20260709 12:59:23.443409  1732 raft_consensus.cc:359] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.444088  1732 raft_consensus.cc:385] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.444357  1732 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.448861  1732 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.452849  1732 ts_tablet_manager.cc:1434] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:59:23.453486  1740 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.458659  1836 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.461625  1836 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:23.463207  1828 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.453890  1732 tablet_bootstrap.cc:492] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:23.469262  1733 tablet_bootstrap.cc:492] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:23.469698  1733 ts_tablet_manager.cc:1403] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.187s	user 0.084s	sys 0.080s
I20260709 12:59:23.467362  1828 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.476435  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:23.478470  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:23.481781  1689 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.479905  1739 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:23.484828  1658 leader_election.cc:343] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.485471  1864 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.486280  1739 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.487470  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:23.488070  1691 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.491477  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:23.492280  1539 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.493640  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:23.502132  1734 tablet_bootstrap.cc:492] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:23.502983  1734 ts_tablet_manager.cc:1403] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.216s	user 0.060s	sys 0.102s
I20260709 12:59:23.507649  1689 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:23.509050  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:23.506573  1691 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:23.514788  1739 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.515323  1739 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:23.515862  1582 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:23.516196  1739 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.516752  1828 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.517203  1828 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:23.517998  1828 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.519474  1733 raft_consensus.cc:359] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.520112  1733 raft_consensus.cc:385] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.520358  1733 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.520828  1539 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:23.514477  1792 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:23.521871  1733 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.528259  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:23.529536  1617 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:23.530746  1584 leader_election.cc:343] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.535810  1540 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.538540  1732 tablet_bootstrap.cc:654] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.540979  1733 ts_tablet_manager.cc:1434] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.071s	user 0.007s	sys 0.000s
I20260709 12:59:23.542022  1733 tablet_bootstrap.cc:492] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:23.542114  1617 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:23.543952  1890 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.545985  1890 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.550138  1890 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 12:59:23.551492  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45773.5us
I20260709 12:59:23.555186  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:23.568523  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:23.569197  1617 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:23.569641  1777 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.570322  1777 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.572778  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:23.577782  1890 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.579892  1734 raft_consensus.cc:359] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.596024  1734 raft_consensus.cc:385] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.596344  1734 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.573083  1777 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.596920  1734 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.602509  1438 catalog_manager.cc:5697] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:23.597612  1890 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:23.610051  1777 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.653016  1733 tablet_bootstrap.cc:654] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.655964  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:23.656852  1734 ts_tablet_manager.cc:1434] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.154s	user 0.000s	sys 0.016s
I20260709 12:59:23.657836  1734 tablet_bootstrap.cc:492] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:23.652851  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:23.661947  1890 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.635520  1777 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.670647  1777 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.680622  1732 tablet_bootstrap.cc:492] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:23.681097  1732 ts_tablet_manager.cc:1403] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.227s	user 0.012s	sys 0.019s
W20260709 12:59:23.669194  1582 leader_election.cc:343] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.661240  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 12:59:23.682651  1584 leader_election.cc:343] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.684904  1584 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:23.686690  1898 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.687644  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:23.688438  1732 raft_consensus.cc:359] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:23.688973  1584 leader_election.cc:343] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.689191  1732 raft_consensus.cc:385] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.689543  1732 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.689743  1853 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.690138  1853 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.690541  1853 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.690207  1732 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:23.691496  1658 leader_election.cc:343] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.695484  1895 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.696985  1853 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.698172  1895 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.699115  1894 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:23.699069  1895 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.700666  1732 ts_tablet_manager.cc:1434] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 12:59:23.702008  1732 tablet_bootstrap.cc:492] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:23.705700  1733 tablet_bootstrap.cc:492] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:23.706348  1733 ts_tablet_manager.cc:1403] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.165s	user 0.016s	sys 0.026s
I20260709 12:59:23.709810  1895 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:23.710841  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:23.711221  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:23.713714  1853 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:23.716770  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:23.717454  1828 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.721695  1828 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.724102  1733 raft_consensus.cc:359] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:23.717861  1658 leader_election.cc:343] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.725176  1733 raft_consensus.cc:385] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.725579  1733 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
W20260709 12:59:23.726202  1658 leader_election.cc:343] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.726629  1732 tablet_bootstrap.cc:654] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.726758  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:23.727391  1616 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.726321  1733 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.731161  1733 ts_tablet_manager.cc:1434] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:59:23.732198  1733 tablet_bootstrap.cc:492] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:23.733790  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 12:59:23.735237  1584 leader_election.cc:343] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:23.719877  1582 leader_election.cc:343] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.720860  1734 tablet_bootstrap.cc:654] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.724511  1828 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.717473  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 12:59:23.742144  1658 leader_election.cc:343] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.742641  1658 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:23.743748  1853 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.745393  1616 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:23.739118  1582 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:23.747448  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:23.747812  1828 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.748535  1853 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.749042  1853 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:23.757884  1733 tablet_bootstrap.cc:654] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.749878  1853 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.785158  1732 tablet_bootstrap.cc:492] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:23.785904  1732 ts_tablet_manager.cc:1403] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.084s	user 0.018s	sys 0.004s
I20260709 12:59:23.788102  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 12:59:23.798044  1582 leader_election.cc:343] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.798525  1582 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:23.799342  1828 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.800285  1440 catalog_manager.cc:5697] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:23.805913  1732 raft_consensus.cc:359] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.806677  1732 raft_consensus.cc:385] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.806980  1732 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.807674  1732 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.810617  1732 ts_tablet_manager.cc:1434] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 12:59:23.811582  1732 tablet_bootstrap.cc:492] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:23.814599  1853 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.815320  1853 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.819657  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:23.819983  1732 tablet_bootstrap.cc:654] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.825228  1734 tablet_bootstrap.cc:492] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:23.825752  1734 ts_tablet_manager.cc:1403] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.168s	user 0.016s	sys 0.011s
I20260709 12:59:23.826162  1853 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:23.826247  1777 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.826777  1777 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.828264  1734 raft_consensus.cc:359] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.828814  1777 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.828941  1734 raft_consensus.cc:385] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.829265  1734 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.830044  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:23.829954  1734 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:23.830363  1658 leader_election.cc:343] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.830435  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:23.820010  1437 catalog_manager.cc:5697] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:23.831722  1584 leader_election.cc:343] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.832700  1734 ts_tablet_manager.cc:1434] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:59:23.833732  1734 tablet_bootstrap.cc:492] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
W20260709 12:59:23.833613  1582 leader_election.cc:343] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.834110  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:23.835297  1777 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.836448  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:23.837291  1617 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:23.842774  1733 tablet_bootstrap.cc:492] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:23.842796  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:23.843485  1733 ts_tablet_manager.cc:1403] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.112s	user 0.011s	sys 0.016s
I20260709 12:59:23.843778  1836 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.844177  1836 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.844501  1836 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.846707  1733 raft_consensus.cc:359] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.847415  1733 raft_consensus.cc:385] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.847679  1733 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.848223  1734 tablet_bootstrap.cc:654] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.850397  1733 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.851292  1836 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.855409  1733 ts_tablet_manager.cc:1434] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 12:59:23.856673  1733 tablet_bootstrap.cc:492] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:23.857398  1864 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.858095  1864 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.858836  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:23.859781  1740 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:23.860728  1658 leader_election.cc:343] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.860718  1740 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.861884  1836 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:23.862300  1864 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.862936  1740 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.863818  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:23.863936  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:23.864650  1618 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.863945  1440 catalog_manager.cc:5697] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:23.865757  1732 tablet_bootstrap.cc:492] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:23.866220  1732 ts_tablet_manager.cc:1403] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.055s	user 0.007s	sys 0.008s
I20260709 12:59:23.867750  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:23.868549  1691 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 12:59:23.868721  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:23.869481  1617 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 12:59:23.873731  1507 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:23.875393  1740 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.874327  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:23.877532  1507 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:23.878363  1864 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.878952  1618 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:23.888762  1740 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.891484  1864 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:23.892310  1732 raft_consensus.cc:359] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.893507  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:23.893929  1733 tablet_bootstrap.cc:654] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.893527  1732 raft_consensus.cc:385] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.895720  1732 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.896490  1836 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:23.895727  1734 tablet_bootstrap.cc:492] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:23.897714  1734 ts_tablet_manager.cc:1403] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.064s	user 0.008s	sys 0.016s
I20260709 12:59:23.897343  1732 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.899742  1836 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:23.900584  1734 raft_consensus.cc:359] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.901129  1734 raft_consensus.cc:385] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.901474  1734 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.900944  1836 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.902344  1853 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.902652  1777 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:23.902796  1853 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.903378  1777 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.904448  1734 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.905280  1853 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:23.907130  1734 ts_tablet_manager.cc:1434] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 12:59:23.908190  1734 tablet_bootstrap.cc:492] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:23.908545  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:23.909175  1691 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:23.910290  1582 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:23.910251  1733 tablet_bootstrap.cc:492] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:23.911322  1733 ts_tablet_manager.cc:1403] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.055s	user 0.011s	sys 0.005s
I20260709 12:59:23.913285  1828 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:23.914161  1828 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:23.914503  1828 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.914628  1732 ts_tablet_manager.cc:1434] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 12:59:23.915721  1732 tablet_bootstrap.cc:492] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:23.912765  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:23.912462  1777 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.922037  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:23.932976  1732 tablet_bootstrap.cc:654] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.935619  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 12:59:23.937919  1584 leader_election.cc:343] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:23.941201  1658 leader_election.cc:343] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.940733  1733 raft_consensus.cc:359] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.941844  1733 raft_consensus.cc:385] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.942245  1733 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.938434  1828 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.943316  1733 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.943971  1898 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.945060  1828 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:23.946635  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:23.947233  1691 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:23.947734  1733 ts_tablet_manager.cc:1434] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.001s
I20260709 12:59:23.949342  1733 tablet_bootstrap.cc:492] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:23.949509  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 12:59:23.958734  1658 leader_election.cc:343] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.959936  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:23.962255  1734 tablet_bootstrap.cc:654] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:23.969241  1853 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:23.973773  1584 leader_election.cc:343] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:23.981036  1732 tablet_bootstrap.cc:492] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:23.981652  1732 ts_tablet_manager.cc:1403] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.066s	user 0.029s	sys 0.017s
I20260709 12:59:23.995337  1732 raft_consensus.cc:359] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:23.996138  1732 raft_consensus.cc:385] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:23.996511  1732 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:23.997439  1829 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:23.998198  1732 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.000350  1732 ts_tablet_manager.cc:1434] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.003s
I20260709 12:59:24.007102  1732 tablet_bootstrap.cc:492] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:24.022159  1733 tablet_bootstrap.cc:654] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.024272  1828 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:24.027056  1828 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.027544  1828 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.037113  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 12:59:24.038771  1582 leader_election.cc:343] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.039781  1691 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:24.042418  1732 tablet_bootstrap.cc:654] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:24.041204  1489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:24.035980  1828 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:24.047437  1440 catalog_manager.cc:5697] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:24.049904  1582 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:24.051635  1828 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.053797  1828 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:24.054971  1828 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.057207  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 12:59:24.065912  1584 leader_election.cc:343] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.070416  1734 tablet_bootstrap.cc:492] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:24.070849  1584 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:24.071175  1734 ts_tablet_manager.cc:1403] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.163s	user 0.012s	sys 0.023s
I20260709 12:59:24.074142  1898 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:24.089663  1734 raft_consensus.cc:359] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.090703  1734 raft_consensus.cc:385] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.091097  1734 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.092926  1828 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:24.094179  1734 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.094141  1732 tablet_bootstrap.cc:492] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:24.095839  1732 ts_tablet_manager.cc:1403] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.095s	user 0.019s	sys 0.006s
I20260709 12:59:24.096863  1734 ts_tablet_manager.cc:1434] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.003s
I20260709 12:59:24.098830  1734 tablet_bootstrap.cc:492] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:24.101815  1439 catalog_manager.cc:5697] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:24.107379  1732 raft_consensus.cc:359] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.108363  1732 raft_consensus.cc:385] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.108704  1732 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.109561  1732 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.113904  1732 ts_tablet_manager.cc:1434] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:59:24.114837  1732 tablet_bootstrap.cc:492] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:24.116228  1733 tablet_bootstrap.cc:492] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:24.116991  1733 ts_tablet_manager.cc:1403] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.168s	user 0.014s	sys 0.019s
I20260709 12:59:24.119432  1733 raft_consensus.cc:359] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.120092  1733 raft_consensus.cc:385] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.120373  1733 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.121349  1733 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.123329  1732 tablet_bootstrap.cc:654] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.124300  1733 ts_tablet_manager.cc:1434] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:59:24.126060  1733 tablet_bootstrap.cc:492] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:24.128880  1734 tablet_bootstrap.cc:654] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.132283  1733 tablet_bootstrap.cc:654] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.143172  1890 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.165843  1732 tablet_bootstrap.cc:492] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:24.166340  1732 ts_tablet_manager.cc:1403] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.052s	user 0.008s	sys 0.002s
I20260709 12:59:24.166540  1734 tablet_bootstrap.cc:492] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:24.166990  1734 ts_tablet_manager.cc:1403] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.068s	user 0.019s	sys 0.019s
I20260709 12:59:24.171083  1732 raft_consensus.cc:359] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.171839  1732 raft_consensus.cc:385] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.172132  1732 raft_consensus.cc:740] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.172827  1732 consensus_queue.cc:260] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.174793  1733 tablet_bootstrap.cc:492] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:24.174791  1777 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.175420  1733 ts_tablet_manager.cc:1403] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.050s	user 0.027s	sys 0.015s
I20260709 12:59:24.175382  1777 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.178041  1734 raft_consensus.cc:359] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.178561  1733 raft_consensus.cc:359] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.179467  1734 raft_consensus.cc:385] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.179562  1733 raft_consensus.cc:385] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.179800  1734 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.179934  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:24.180305  1836 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.180548  1734 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.180757  1836 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.182777  1836 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:24.182775  1734 ts_tablet_manager.cc:1434] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260709 12:59:24.179906  1733 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.183815  1734 tablet_bootstrap.cc:492] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:24.183914  1733 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:24.184501  1582 leader_election.cc:343] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.185047  1777 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:24.185786  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:24.186156  1733 ts_tablet_manager.cc:1434] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.004s
I20260709 12:59:24.187052  1733 tablet_bootstrap.cc:492] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:24.187775  1732 ts_tablet_manager.cc:1434] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.003s
I20260709 12:59:24.189145  1732 tablet_bootstrap.cc:492] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:24.190264  1734 tablet_bootstrap.cc:654] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.192934  1733 tablet_bootstrap.cc:654] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.196719  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 12:59:24.197322  1584 leader_election.cc:343] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.197803  1584 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:24.198232  1890 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.199790  1777 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:24.201037  1658 leader_election.cc:343] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.203086  1777 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.203739  1777 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.207357  1732 tablet_bootstrap.cc:654] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:24.213260  1658 consensus_peers.cc:597] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:24.214514  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:24.215356  1618 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:24.216452  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:24.217692  1733 tablet_bootstrap.cc:492] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:24.218158  1733 ts_tablet_manager.cc:1403] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.002s
W20260709 12:59:24.218324  1582 leader_election.cc:343] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.219408  1777 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:24.219656  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:24.220778  1733 raft_consensus.cc:359] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.221493  1733 raft_consensus.cc:385] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.221768  1733 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
W20260709 12:59:24.222102  1584 leader_election.cc:343] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.222644  1733 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.223264  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:24.222731  1658 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:24.224632  1777 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:24.225361  1734 tablet_bootstrap.cc:492] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:24.225850  1734 ts_tablet_manager.cc:1403] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.042s	user 0.021s	sys 0.019s
I20260709 12:59:24.226776  1890 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.227151  1890 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.227470  1890 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.228729  1733 ts_tablet_manager.cc:1434] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:59:24.228513  1734 raft_consensus.cc:359] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.229208  1734 raft_consensus.cc:385] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.229508  1734 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.229702  1733 tablet_bootstrap.cc:492] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:24.232637  1732 tablet_bootstrap.cc:492] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:24.233098  1732 ts_tablet_manager.cc:1403] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.044s	user 0.007s	sys 0.004s
I20260709 12:59:24.235670  1732 raft_consensus.cc:359] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.236315  1732 raft_consensus.cc:385] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.236598  1732 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.238123  1734 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.237144  1733 tablet_bootstrap.cc:654] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.240271  1734 ts_tablet_manager.cc:1434] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.003s
I20260709 12:59:24.240849  1732 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.243005  1732 ts_tablet_manager.cc:1434] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:59:24.244074  1734 tablet_bootstrap.cc:492] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:24.247478  1864 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.248435  1864 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.250576  1734 tablet_bootstrap.cc:654] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.251344  1733 tablet_bootstrap.cc:492] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:24.251396  1864 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:24.251883  1733 ts_tablet_manager.cc:1403] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.008s
I20260709 12:59:24.253851  1732 tablet_bootstrap.cc:492] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:24.254781  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:24.255432  1618 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:24.258998  1509 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:24.259281  1895 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.260002  1864 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.259804  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:24.259793  1895 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.263660  1895 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:24.264154  1691 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:24.260499  1864 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.267923  1864 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.272789  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:24.273485  1618 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:24.275497  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:24.276650  1895 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.277163  1895 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.277661  1895 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.278260  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:24.315722  1853 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:24.325515  1890 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.328400  1733 raft_consensus.cc:359] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.334255  1733 raft_consensus.cc:385] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.334604  1733 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.335670  1733 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.340837  1733 ts_tablet_manager.cc:1434] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.089s	user 0.018s	sys 0.000s
I20260709 12:59:24.341881  1733 tablet_bootstrap.cc:492] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:24.340806  1864 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.345557  1540 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 1: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 12:59:24.345794  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 12:59:24.347071  1658 leader_election.cc:343] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.350494  1732 tablet_bootstrap.cc:654] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.351996  1734 tablet_bootstrap.cc:492] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:24.352566  1734 ts_tablet_manager.cc:1403] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.109s	user 0.035s	sys 0.047s
I20260709 12:59:24.354836  1890 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:24.356060  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:24.356688  1617 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.357347  1864 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:24.358789  1734 raft_consensus.cc:359] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.359470  1734 raft_consensus.cc:385] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.359743  1734 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.360386  1734 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.361654  1733 tablet_bootstrap.cc:654] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.361711  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:24.362309  1616 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.362653  1734 ts_tablet_manager.cc:1434] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:59:24.366787  1734 tablet_bootstrap.cc:492] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:24.370074  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:24.374771  1617 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:24.382401  1616 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:24.383292  1895 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.386891  1895 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:24.388475  1658 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:24.387389  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:24.389249  1890 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.374389  1829 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.392931  1539 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 1: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 12:59:24.395872  1689 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 12:59:24.395927  1733 tablet_bootstrap.cc:492] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:24.396853  1733 ts_tablet_manager.cc:1403] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.008s
I20260709 12:59:24.396989  1734 tablet_bootstrap.cc:654] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.392628  1829 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.392633  1890 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:24.402642  1890 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.403642  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:24.404428  1617 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 1: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 12:59:24.392714  1509 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:24.405663  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:24.405941  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:24.406241  1864 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.406824  1864 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 12:59:24.407028  1584 leader_election.cc:343] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.406380  1829 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:24.408174  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:24.407979  1864 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.408792  1689 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:24.410652  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:24.414125  1739 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.415160  1739 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.415526  1739 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.414323  1895 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:24.422237  1734 tablet_bootstrap.cc:492] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:24.425740  1734 ts_tablet_manager.cc:1403] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.008s
I20260709 12:59:24.424916  1733 raft_consensus.cc:359] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.427275  1733 raft_consensus.cc:385] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.427562  1733 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.431098  1733 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.432536  1734 raft_consensus.cc:359] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.434394  1734 raft_consensus.cc:385] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.434672  1734 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.435315  1734 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.435847  1739 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.437883  1734 ts_tablet_manager.cc:1434] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:59:24.439100  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:24.439785  1733 ts_tablet_manager.cc:1434] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.043s	user 0.004s	sys 0.003s
I20260709 12:59:24.440744  1734 tablet_bootstrap.cc:492] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:24.441447  1733 tablet_bootstrap.cc:492] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
W20260709 12:59:24.442600  1584 leader_election.cc:343] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.444339  1739 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:24.445633  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:24.446341  1689 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.448220  1732 tablet_bootstrap.cc:492] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:24.448696  1732 ts_tablet_manager.cc:1403] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.195s	user 0.004s	sys 0.014s
I20260709 12:59:24.453969  1440 catalog_manager.cc:5697] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:24.462929  1689 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:24.462594  1732 raft_consensus.cc:359] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.465219  1732 raft_consensus.cc:385] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.464562  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:24.465481  1733 tablet_bootstrap.cc:654] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.465534  1732 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.466712  1829 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.466439  1732 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.467231  1829 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:24.468132  1829 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.464684  1734 tablet_bootstrap.cc:654] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.471154  1732 ts_tablet_manager.cc:1434] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.006s
I20260709 12:59:24.472419  1732 tablet_bootstrap.cc:492] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:24.483954  1732 tablet_bootstrap.cc:654] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.485463  1829 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.485852  1890 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.486372  1890 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.487744  1739 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.488691  1890 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:24.488409  1739 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.489796  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:24.490362  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:24.490476  1616 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
W20260709 12:59:24.492336  1658 leader_election.cc:343] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.492506  1739 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:24.492385  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:24.493134  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:24.493273  1539 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:24.493606  1733 tablet_bootstrap.cc:492] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:24.494045  1733 ts_tablet_manager.cc:1403] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.053s	user 0.017s	sys 0.002s
I20260709 12:59:24.494138  1890 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.494549  1890 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.494894  1890 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.495605  1439 catalog_manager.cc:5697] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:24.496399  1895 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.496868  1895 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.497480  1733 raft_consensus.cc:359] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.498158  1733 raft_consensus.cc:385] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.498436  1733 raft_consensus.cc:740] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.498277  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:24.498852  1895 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:24.499096  1733 consensus_queue.cc:260] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.486106  1829 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.500531  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:24.501199  1733 ts_tablet_manager.cc:1434] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:59:24.501145  1616 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:24.494313  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:24.502496  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:24.502727  1689 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:24.503230  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:24.503618  1895 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.503996  1895 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.504312  1895 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.500525  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:24.499532  1739 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.509145  1829 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:24.505808  1539 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
W20260709 12:59:24.505823  1584 leader_election.cc:343] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.508142  1440 catalog_manager.cc:5697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:24.509460  1739 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.509814  1890 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.511945  1895 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.512950  1890 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:24.513473  1739 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.514660  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:24.514856  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:24.515291  1540 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.515935  1616 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.516338  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:24.520166  1732 tablet_bootstrap.cc:492] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:24.520646  1732 ts_tablet_manager.cc:1403] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.048s	user 0.011s	sys 0.006s
I20260709 12:59:24.523727  1732 raft_consensus.cc:359] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.524418  1732 raft_consensus.cc:385] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.524672  1732 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.525529  1732 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.527817  1732 ts_tablet_manager.cc:1434] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:59:24.528062  1739 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.529203  1732 tablet_bootstrap.cc:492] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:24.530279  1739 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 12:59:24.531349  1658 leader_election.cc:343] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.533581  1734 tablet_bootstrap.cc:492] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:24.534034  1734 ts_tablet_manager.cc:1403] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.094s	user 0.012s	sys 0.002s
I20260709 12:59:24.535475  1540 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:24.536213  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:24.536646  1734 raft_consensus.cc:359] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.537354  1734 raft_consensus.cc:385] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.537634  1734 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.536795  1543 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.538352  1734 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.539340  1733 tablet_bootstrap.cc:492] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:24.540791  1734 ts_tablet_manager.cc:1434] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:24.541582  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:24.542167  1734 tablet_bootstrap.cc:492] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:24.542281  1691 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:24.543306  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:24.544310  1689 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.546382  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:24.547099  1895 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:24.547298  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:24.547835  1733 tablet_bootstrap.cc:654] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.547956  1617 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.549674  1734 tablet_bootstrap.cc:654] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.555238  1829 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.555740  1829 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.556108  1829 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.557749  1734 tablet_bootstrap.cc:492] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:24.558233  1734 ts_tablet_manager.cc:1403] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.008s
I20260709 12:59:24.560964  1734 raft_consensus.cc:359] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.561681  1734 raft_consensus.cc:385] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.561976  1734 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.562626  1734 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.563143  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:24.564306  1890 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.564694  1734 ts_tablet_manager.cc:1434] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:24.564914  1890 raft_consensus.cc:697] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:24.566017  1890 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.566792  1734 tablet_bootstrap.cc:492] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:24.568022  1733 tablet_bootstrap.cc:492] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:24.568463  1733 ts_tablet_manager.cc:1403] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.029s	user 0.004s	sys 0.020s
I20260709 12:59:24.568683  1617 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:24.571281  1540 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.571604  1740 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.572062  1740 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.574257  1740 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:24.576644  1734 tablet_bootstrap.cc:654] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.577874  1733 raft_consensus.cc:359] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.578646  1733 raft_consensus.cc:385] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.578711  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:24.578979  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:24.579203  1733 raft_consensus.cc:740] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.579668  1691 raft_consensus.cc:2393] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 12:59:24.580659  1895 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.581640  1836 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.581845  1895 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.584344  1895 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:24.585295  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:24.586640  1740 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.580055  1733 consensus_queue.cc:260] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.603181  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:24.604516  1733 ts_tablet_manager.cc:1434] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.003s
I20260709 12:59:24.607329  1733 tablet_bootstrap.cc:492] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:24.599025  1689 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:24.608759  1617 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:24.595854  1543 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:24.610800  1582 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:24.611526  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:24.612200  1539 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:24.612769  1732 tablet_bootstrap.cc:654] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.607235  1740 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:24.616797  1739 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.617367  1739 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:24.618759  1739 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.598312  1616 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:24.624614  1777 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.628409  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:24.629405  1853 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.629894  1853 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.630298  1853 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.610813  1829 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.631387  1439 catalog_manager.cc:5697] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:24.634668  1890 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.635200  1890 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.636933  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:24.637504  1890 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 12:59:24.638366  1584 leader_election.cc:343] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.633266  1829 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:24.643965  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:24.644622  1890 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.644863  1895 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.645545  1895 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:24.646611  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:24.646442  1895 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.647222  1691 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.650256  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:24.655706  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:24.656463  1618 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:24.659464  1691 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:24.663936  1853 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:24.664902  1658 leader_election.cc:343] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.666095  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:24.666945  1439 catalog_manager.cc:5697] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:24.670984  1734 tablet_bootstrap.cc:492] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:24.671449  1734 ts_tablet_manager.cc:1403] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.106s	user 0.009s	sys 0.004s
I20260709 12:59:24.675191  1890 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:24.686748  1890 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:24.699559  1890 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.708021  1733 tablet_bootstrap.cc:654] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.719178  1829 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:24.720638  1890 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.698283  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:24.722270  1853 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:24.723505  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:24.729744  1540 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.735505  1777 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.736311  1777 raft_consensus.cc:697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:24.737524  1777 consensus_queue.cc:237] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.745927  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:24.746572  1618 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.748418  1734 raft_consensus.cc:359] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.749511  1734 raft_consensus.cc:385] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.749907  1734 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.757869  1541 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:24.764294  1734 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.812603  1734 ts_tablet_manager.cc:1434] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.141s	user 0.011s	sys 0.000s
I20260709 12:59:24.814035  1734 tablet_bootstrap.cc:492] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
W20260709 12:59:24.818197  1948 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:59:24.830183  1836 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.851364  1732 tablet_bootstrap.cc:492] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:24.853703  1732 ts_tablet_manager.cc:1403] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.325s	user 0.010s	sys 0.061s
I20260709 12:59:24.863639  1732 raft_consensus.cc:359] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.864241  1732 raft_consensus.cc:385] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.864506  1732 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.865159  1732 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.877962  1618 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:24.843874  1836 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.883318  1732 ts_tablet_manager.cc:1434] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 12:59:24.883786  1734 tablet_bootstrap.cc:654] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:24.877965  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:24.888886  1733 tablet_bootstrap.cc:492] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:24.889309  1733 ts_tablet_manager.cc:1403] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.284s	user 0.012s	sys 0.022s
I20260709 12:59:24.895473  1836 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:24.871742  1890 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 12:59:24.900722  1953 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.082s	user 0.002s	sys 0.000s
I20260709 12:59:24.900883  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:24.901593  1615 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:24.907431  1658 leader_election.cc:343] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.909922  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:24.910284  1540 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:24.911453  1740 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:24.912155  1733 raft_consensus.cc:359] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.912902  1733 raft_consensus.cc:385] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:24.913192  1733 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:24.913406  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:24.918453  1949 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:24.919209  1949 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:24.920311  1949 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.925900  1440 catalog_manager.cc:5697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:24.938599  1960 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260709 12:59:24.953451  1658 leader_election.cc:343] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:24.962321  1615 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:24.967835  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:24.966192  1733 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:24.971668  1732 tablet_bootstrap.cc:492] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:24.969380  1740 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.988469  1733 ts_tablet_manager.cc:1434] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.099s	user 0.008s	sys 0.000s
I20260709 12:59:24.992610  1733 tablet_bootstrap.cc:492] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:24.991096  1828 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:24.999053  1828 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:24.994720  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44380.9us
I20260709 12:59:24.994789  1740 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:25.024747  1658 leader_election.cc:343] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.025425  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:25.049860  1959 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:25.088729  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64681.4us
I20260709 12:59:25.092531  1732 tablet_bootstrap.cc:654] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.094878  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:25.096769  1829 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.106181  1829 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.098246  1853 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:25.108469  1853 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:25.109300  1853 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.111887  1740 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:25.112566  1734 tablet_bootstrap.cc:492] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:25.112790  1828 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:25.113034  1734 ts_tablet_manager.cc:1403] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.299s	user 0.022s	sys 0.011s
I20260709 12:59:25.115789  1733 tablet_bootstrap.cc:654] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.118199  1864 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 12:59:25.118510  1734 raft_consensus.cc:359] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.120270  1734 raft_consensus.cc:385] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.120571  1734 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.118808  1864 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.121236  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:25.123463  1734 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.126771  1734 ts_tablet_manager.cc:1434] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:59:25.127563  1734 tablet_bootstrap.cc:492] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:25.125639  1963 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:59:25.131528  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:25.132969  1614 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 12:59:25.138336  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 12:59:25.143002  1970 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.132s	user 0.000s	sys 0.003s
I20260709 12:59:25.146834  1740 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.147557  1740 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.155435  1836 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.156275  1740 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:25.156009  1836 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.158145  1836 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 12:59:25.161247  1582 leader_election.cc:343] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.161401  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:25.164973  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:25.165838  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:25.163110  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:25.166215  1539 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.175729  1968 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.179563  1732 tablet_bootstrap.cc:492] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:25.180032  1732 ts_tablet_manager.cc:1403] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.209s	user 0.009s	sys 0.016s
I20260709 12:59:25.180593  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:25.192716  1733 tablet_bootstrap.cc:492] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:25.193189  1733 ts_tablet_manager.cc:1403] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.201s	user 0.019s	sys 0.000s
I20260709 12:59:25.194898  1864 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:25.189790  1829 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 12:59:25.199373  1658 leader_election.cc:343] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.196074  1437 catalog_manager.cc:5697] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:25.200762  1437 catalog_manager.cc:5697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:25.196656  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:25.210820  1732 raft_consensus.cc:359] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.210865  1734 tablet_bootstrap.cc:654] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.211923  1732 raft_consensus.cc:385] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.212503  1732 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.216118  1733 raft_consensus.cc:359] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.216854  1733 raft_consensus.cc:385] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.217116  1733 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.217986  1733 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.221974  1733 ts_tablet_manager.cc:1434] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 12:59:25.222987  1955 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:25.222970  1733 tablet_bootstrap.cc:492] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:25.222448  1732 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.240321  1955 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.241067  1732 ts_tablet_manager.cc:1434] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260709 12:59:25.240875  1955 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.243440  1732 tablet_bootstrap.cc:492] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:25.244009  1955 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:25.245157  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:25.246068  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 12:59:25.246759  1658 leader_election.cc:343] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.248361  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:25.248827  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:25.258347  1829 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:59:25.259176  1582 leader_election.cc:343] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.258836  1895 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:25.260113  1509 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:25.261215  1864 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:25.268581  1509 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:25.269804  1864 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:25.270150  1437 catalog_manager.cc:5697] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:25.277043  1733 tablet_bootstrap.cc:654] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.283923  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:25.289927  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:25.291918  1739 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:25.277632  1968 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:25.308197  1853 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:25.329841  1968 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.335216  1734 tablet_bootstrap.cc:492] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:25.335700  1734 ts_tablet_manager.cc:1403] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.208s	user 0.024s	sys 0.027s
I20260709 12:59:25.330480  1968 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:25.335095  1584 leader_election.cc:343] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.339244  1584 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 12:59:25.353016  1584 consensus_peers.cc:597] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:59:25.353556  1584 leader_election.cc:343] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.352195  1732 tablet_bootstrap.cc:654] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.353906  1964 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:25.353929  1584 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:25.358024  1964 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:25.358227  1658 leader_election.cc:343] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.358635  1658 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:25.360136  1960 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:25.361518  1890 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:25.366324  1949 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:59:25.368251  1658 leader_election.cc:343] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.368757  1658 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:25.369555  1890 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:25.369771  1829 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.370671  1829 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.376341  1829 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:25.385818  1733 tablet_bootstrap.cc:492] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:25.386389  1733 ts_tablet_manager.cc:1403] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.164s	user 0.012s	sys 0.006s
I20260709 12:59:25.388465  1734 raft_consensus.cc:359] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.390590  1734 raft_consensus.cc:385] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.390971  1734 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.395216  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:25.398340  1733 raft_consensus.cc:359] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.411851  1733 raft_consensus.cc:385] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.412456  1733 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
W20260709 12:59:25.403095  1584 consensus_peers.cc:597] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:25.413133  1734 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.414506  1733 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:25.419080  1584 leader_election.cc:343] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.423261  1733 ts_tablet_manager.cc:1434] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 12:59:25.424079  1733 tablet_bootstrap.cc:492] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:25.424750  1734 ts_tablet_manager.cc:1434] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.089s	user 0.007s	sys 0.000s
I20260709 12:59:25.425515  1734 tablet_bootstrap.cc:492] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:25.427054  1777 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:25.420305  1961 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:59:25.431408  1489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:25.438695  1955 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:25.439513  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:25.443244  1615 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:25.444279  1963 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.457562  1963 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.441100  1968 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:25.477687  1509 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:25.479445  1740 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:25.479839  1740 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:25.480222  1740 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.472672  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:25.484638  1691 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:25.493016  1963 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:25.494415  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:25.495360  1614 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:25.496728  1509 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:25.497400  1740 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.497964  1963 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:25.498476  1963 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:25.498871  1963 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.506306  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:25.511269  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:25.511878  1614 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.516276  1732 tablet_bootstrap.cc:492] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:25.516714  1732 ts_tablet_manager.cc:1403] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.274s	user 0.025s	sys 0.030s
I20260709 12:59:25.518469  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 12:59:25.519268  1507 leader_election.cc:343] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.509699  1740 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:25.522290  1691 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.526841  1733 tablet_bootstrap.cc:654] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.530156  1734 tablet_bootstrap.cc:654] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.530117  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:25.545367  1614 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:25.549233  1963 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.554756  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:25.555598  1615 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.559921  1955 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:25.549227  1732 raft_consensus.cc:359] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.562667  1732 raft_consensus.cc:385] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.562953  1732 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.565332  1509 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:25.566741  1968 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:25.568452  1968 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:25.567987  1732 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.569031  1691 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:25.569536  1968 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.571038  1732 ts_tablet_manager.cc:1434] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.001s
I20260709 12:59:25.572099  1732 tablet_bootstrap.cc:492] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:25.572295  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 12:59:25.573489  1507 leader_election.cc:343] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.571099  1963 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:25.611732  1733 tablet_bootstrap.cc:492] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:25.612154  1733 ts_tablet_manager.cc:1403] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.188s	user 0.012s	sys 0.009s
I20260709 12:59:25.621734  1955 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 12:59:25.623399  1582 leader_election.cc:343] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.623881  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:25.625485  1828 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:25.629364  1658 consensus_peers.cc:597] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:25.629899  1733 raft_consensus.cc:359] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.632284  1733 raft_consensus.cc:385] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.632575  1733 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.633876  1615 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:25.635169  1733 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.637317  1733 ts_tablet_manager.cc:1434] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 12:59:25.638305  1733 tablet_bootstrap.cc:492] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:25.681473  1509 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 12:59:25.689459  1739 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.691145  1734 tablet_bootstrap.cc:492] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:25.690596  1739 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.692726  1734 ts_tablet_manager.cc:1403] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.267s	user 0.025s	sys 0.004s
I20260709 12:59:25.699592  1968 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:25.700785  1968 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:25.702853  1968 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.708909  1733 tablet_bootstrap.cc:654] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.711287  1890 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.712195  1890 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.715510  1890 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:25.717533  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 12:59:25.722200  1658 leader_election.cc:343] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.721660  1732 tablet_bootstrap.cc:654] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.731666  1739 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:25.732271  1734 raft_consensus.cc:359] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.733626  1734 raft_consensus.cc:385] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.734000  1734 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.734417  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:25.734954  1734 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.740556  1734 ts_tablet_manager.cc:1434] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.004s
I20260709 12:59:25.743213  1734 tablet_bootstrap.cc:492] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:25.750375  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:25.753471  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 12:59:25.759013  1584 leader_election.cc:343] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:25.759644  1582 leader_election.cc:343] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.760031  1582 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:25.761034  1828 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:59:25.785497  1658 leader_election.cc:343] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.786059  1658 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:25.793570  1955 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:25.809921  1733 tablet_bootstrap.cc:492] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:25.810436  1734 tablet_bootstrap.cc:654] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.810822  1733 ts_tablet_manager.cc:1403] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.173s	user 0.027s	sys 0.023s
I20260709 12:59:25.814874  1733 raft_consensus.cc:359] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.815714  1733 raft_consensus.cc:385] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.816048  1733 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.816782  1733 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.818408  1733 ts_tablet_manager.cc:1434] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.006s
I20260709 12:59:25.822995  1733 tablet_bootstrap.cc:492] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:25.836935  1437 catalog_manager.cc:5697] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:25.840293  1437 catalog_manager.cc:5697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:25.841442  1732 tablet_bootstrap.cc:492] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:25.842116  1732 ts_tablet_manager.cc:1403] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.270s	user 0.014s	sys 0.010s
I20260709 12:59:25.845638  1732 raft_consensus.cc:359] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.846379  1732 raft_consensus.cc:385] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.846735  1732 raft_consensus.cc:740] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.849596  1734 tablet_bootstrap.cc:492] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:25.850576  1734 ts_tablet_manager.cc:1403] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.108s	user 0.017s	sys 0.005s
I20260709 12:59:25.852169  1733 tablet_bootstrap.cc:654] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.849411  1732 consensus_queue.cc:260] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.854943  1732 ts_tablet_manager.cc:1434] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:59:25.863860  1734 raft_consensus.cc:359] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.865002  1734 raft_consensus.cc:385] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.865280  1734 raft_consensus.cc:740] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.866055  1734 consensus_queue.cc:260] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.869007  1734 ts_tablet_manager.cc:1434] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:59:25.870550  1734 tablet_bootstrap.cc:492] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:25.874779  1732 tablet_bootstrap.cc:492] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:25.900713  1733 tablet_bootstrap.cc:492] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:25.901242  1733 ts_tablet_manager.cc:1403] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.080s	user 0.017s	sys 0.005s
I20260709 12:59:25.904428  1734 tablet_bootstrap.cc:654] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.904160  1733 raft_consensus.cc:359] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.905740  1733 raft_consensus.cc:385] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.906028  1733 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.908613  1733 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.913314  1732 tablet_bootstrap.cc:654] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.914482  1539 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.923847  1733 ts_tablet_manager.cc:1434] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 12:59:25.924811  1732 tablet_bootstrap.cc:492] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:25.925312  1732 ts_tablet_manager.cc:1403] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.051s	user 0.011s	sys 0.003s
I20260709 12:59:25.928304  1732 raft_consensus.cc:359] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.929142  1732 raft_consensus.cc:385] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.929454  1732 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.930516  1732 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.932271  1734 tablet_bootstrap.cc:492] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:25.932883  1734 ts_tablet_manager.cc:1403] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.063s	user 0.008s	sys 0.004s
I20260709 12:59:25.933101  1732 ts_tablet_manager.cc:1434] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:59:25.934110  1732 tablet_bootstrap.cc:492] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:25.935563  1734 raft_consensus.cc:359] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.936264  1734 raft_consensus.cc:385] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.936517  1734 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.940513  1732 tablet_bootstrap.cc:654] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.941602  1733 tablet_bootstrap.cc:492] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:25.942567  1968 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 12:59:25.943107  1968 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.948518  1968 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:25.949682  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:25.937156  1734 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.952217  1734 ts_tablet_manager.cc:1434] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.004s
I20260709 12:59:25.952217  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:25.955106  1949 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:25.955632  1949 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.956648  1732 tablet_bootstrap.cc:492] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:25.957103  1732 ts_tablet_manager.cc:1403] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.023s	user 0.003s	sys 0.016s
I20260709 12:59:25.958105  1949 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:25.958885  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:25.959460  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:25.960289  1732 raft_consensus.cc:359] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:25.960958  1732 raft_consensus.cc:385] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:25.961364  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:25.961383  1732 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:25.962224  1616 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:25.962445  1732 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:25.964071  1658 leader_election.cc:343] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:25.964828  1732 ts_tablet_manager.cc:1434] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:59:25.965723  1734 tablet_bootstrap.cc:492] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:25.966006  1732 tablet_bootstrap.cc:492] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:25.966645  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:25.968225  1836 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:25.968736  1836 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:25.969113  1836 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:25.960927  1968 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:25.972585  1732 tablet_bootstrap.cc:654] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.973940  1734 tablet_bootstrap.cc:654] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.957403  1733 tablet_bootstrap.cc:654] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:25.995108  1836 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.000931  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 12:59:26.003422  1658 leader_election.cc:343] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.002419  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:26.004503  1616 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.007462  1836 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:26.014247  1963 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.015007  1963 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.021088  1616 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:26.029788  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:26.030875  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:26.032461  1836 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:26.033147  1836 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:26.035117  1963 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.036275  1734 tablet_bootstrap.cc:492] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:26.036957  1828 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.037106  1734 ts_tablet_manager.cc:1403] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.072s	user 0.018s	sys 0.022s
I20260709 12:59:26.037511  1828 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.037618  1836 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.039733  1828 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.040630  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:26.040809  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 12:59:26.042229  1584 leader_election.cc:343] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.043063  1507 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:26.043290  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:26.043982  1687 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:26.045261  1582 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:26.046663  1828 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:26.047186  1828 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:26.047632  1828 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.044605  1963 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:26.051083  1734 raft_consensus.cc:359] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.052136  1734 raft_consensus.cc:385] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.052606  1734 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.036345  1732 tablet_bootstrap.cc:492] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:26.054167  1732 ts_tablet_manager.cc:1403] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.088s	user 0.035s	sys 0.028s
I20260709 12:59:26.053946  1734 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.058386  1734 ts_tablet_manager.cc:1434] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 12:59:26.061175  1734 tablet_bootstrap.cc:492] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:26.061866  1733 tablet_bootstrap.cc:492] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:26.062371  1733 ts_tablet_manager.cc:1403] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.121s	user 0.019s	sys 0.016s
I20260709 12:59:26.065467  1733 raft_consensus.cc:359] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.066231  1733 raft_consensus.cc:385] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.066567  1733 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.066907  1539 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.067391  1733 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.068933  1734 tablet_bootstrap.cc:654] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.069979  1733 ts_tablet_manager.cc:1434] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:59:26.071285  1733 tablet_bootstrap.cc:492] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:26.075675  1828 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.078369  1733 tablet_bootstrap.cc:654] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.077306  1732 raft_consensus.cc:359] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.079486  1732 raft_consensus.cc:385] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.079792  1732 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.080402  1732 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.081139  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:26.081835  1537 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.084262  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:26.084853  1687 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.087893  1732 ts_tablet_manager.cc:1434] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 12:59:26.088744  1961 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.089092  1732 tablet_bootstrap.cc:492] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:26.089296  1961 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.092988  1828 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.098423  1961 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.098631  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:26.100521  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:26.101639  1688 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
W20260709 12:59:26.102658  1584 leader_election.cc:343] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.103611  1582 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:26.105144  1961 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:26.105880  1961 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:26.106460  1961 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.107091  1537 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:26.108551  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:26.109587  1687 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:26.112581  1828 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:26.113445  1828 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:26.114259  1828 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.126303  1542 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.126644  1963 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:26.129024  1739 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.129511  1739 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.131398  1739 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.132999  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:26.133294  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 12:59:26.146443  1582 leader_election.cc:343] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:26.149802  1584 leader_election.cc:343] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.150271  1584 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:26.152122  1739 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:26.160104  1961 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.190388  1734 tablet_bootstrap.cc:492] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:26.191432  1734 ts_tablet_manager.cc:1403] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.131s	user 0.017s	sys 0.008s
I20260709 12:59:26.191973  1963 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:59:26.192443  1828 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:26.213181  1507 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:26.220956  1733 tablet_bootstrap.cc:492] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:26.202085  1828 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.224001  1828 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.223551  1439 catalog_manager.cc:5697] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:26.216820  1734 raft_consensus.cc:359] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.226260  1734 raft_consensus.cc:385] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.226560  1734 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.226778  1733 ts_tablet_manager.cc:1403] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.156s	user 0.046s	sys 0.050s
I20260709 12:59:26.235148  1734 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.252033  1961 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.260862  1734 ts_tablet_manager.cc:1434] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.069s	user 0.006s	sys 0.001s
I20260709 12:59:26.256304  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:26.256724  1836 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.259713  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:26.270464  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:26.271189  1692 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.271832  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:26.267302  1836 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.273376  1733 raft_consensus.cc:359] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.275841  1836 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:26.276196  1733 raft_consensus.cc:385] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.276505  1733 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.271292  1732 tablet_bootstrap.cc:654] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.268690  1687 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:26.277642  1733 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:26.278860  1584 leader_election.cc:343] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.279400  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:26.280115  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 12:59:26.281725  1584 leader_election.cc:343] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.280881  1616 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:26.287674  1733 ts_tablet_manager.cc:1434] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.056s	user 0.004s	sys 0.003s
I20260709 12:59:26.287074  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:26.288630  1734 tablet_bootstrap.cc:492] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:26.292593  1733 tablet_bootstrap.cc:492] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:26.294721  1440 catalog_manager.cc:5697] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:26.295820  1828 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:26.296234  1828 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:26.296609  1828 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:26.303107  1658 leader_election.cc:343] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.303210  1828 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.322005  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:26.328514  1864 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:26.333797  1949 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:26.341130  1949 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:26.359588  1949 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:26.360374  2020 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:59:26.361912  2017 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:59:26.361821  1692 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:26.366748  1582 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:26.367654  1989 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:26.368168  1989 raft_consensus.cc:697] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:26.370476  1828 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.372725  1989 consensus_queue.cc:237] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.384979  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:26.396620  1732 tablet_bootstrap.cc:492] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:26.397075  1732 ts_tablet_manager.cc:1403] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.308s	user 0.018s	sys 0.017s
I20260709 12:59:26.393285  1961 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:26.405054  1584 leader_election.cc:343] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.409643  1733 tablet_bootstrap.cc:654] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.411633  1968 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.412202  1968 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.413841  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:26.414366  1968 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.414481  1687 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.415102  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:26.418860  1890 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.419392  1890 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.421212  1687 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:26.437577  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:26.442474  1688 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:26.438383  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:26.407334  1961 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.443928  1615 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:26.445000  1961 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.446578  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:26.447121  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:26.448050  1541 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:26.448550  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:26.449723  1739 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:26.450310  1739 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:26.449635  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:26.451041  1739 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.447346  1688 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:26.446650  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:26.453074  1864 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:26.453130  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:26.453667  1864 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:26.454140  1864 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.454638  1962 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:26.455019  1962 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:26.455336  1962 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.421350  1890 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:26.475432  1949 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:26.476265  1658 leader_election.cc:343] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.487386  1949 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:26.484808  1864 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.489300  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:26.489569  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:26.485105  1962 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.497134  1618 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.498428  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:26.499372  1615 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 12:59:26.491166  1658 consensus_peers.cc:597] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:26.501415  1732 raft_consensus.cc:359] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.502640  1732 raft_consensus.cc:385] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.502978  1732 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.495021  1734 tablet_bootstrap.cc:654] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.505512  1733 tablet_bootstrap.cc:492] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
W20260709 12:59:26.505446  1658 leader_election.cc:343] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.506099  1733 ts_tablet_manager.cc:1403] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.214s	user 0.010s	sys 0.008s
I20260709 12:59:26.506022  1658 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:26.506948  1732 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.512014  2027 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:26.514993  1732 ts_tablet_manager.cc:1434] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.118s	user 0.007s	sys 0.001s
I20260709 12:59:26.511446  1949 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:59:26.517108  1732 tablet_bootstrap.cc:492] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:26.515486  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:26.518064  1543 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 1: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 12:59:26.511791  1658 leader_election.cc:343] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.527276  1739 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:26.520645  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:26.528971  1687 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.530305  1618 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:26.531736  1440 catalog_manager.cc:5697] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:26.508288  1864 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.533356  1733 raft_consensus.cc:359] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.535789  1733 raft_consensus.cc:385] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.536056  1733 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.540114  1733 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.543220  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:26.545190  1733 ts_tablet_manager.cc:1434] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.003s
I20260709 12:59:26.547049  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:26.548281  2026 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:26.545248  1962 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.559598  1733 tablet_bootstrap.cc:492] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:26.558475  2026 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:26.564169  1582 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:26.565222  1828 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:26.567802  2026 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.573257  1739 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:26.573611  1687 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:26.577934  1734 tablet_bootstrap.cc:492] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:26.579228  1734 ts_tablet_manager.cc:1403] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.310s	user 0.006s	sys 0.022s
I20260709 12:59:26.586503  1507 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:26.590087  2030 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:26.590808  2030 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:26.591163  1734 raft_consensus.cc:359] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.591996  1734 raft_consensus.cc:385] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.592321  1734 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.597136  1734 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.598578  1692 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.601150  1732 tablet_bootstrap.cc:654] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.601316  1437 catalog_manager.cc:5697] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:26.607074  1734 ts_tablet_manager.cc:1434] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 12:59:26.591666  2030 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.612959  1733 tablet_bootstrap.cc:654] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.618726  1439 catalog_manager.cc:5697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:26.628715  1734 tablet_bootstrap.cc:492] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:26.666414  1733 tablet_bootstrap.cc:492] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:26.666944  1733 ts_tablet_manager.cc:1403] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.114s	user 0.011s	sys 0.005s
I20260709 12:59:26.668412  1734 tablet_bootstrap.cc:654] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.680377  1439 catalog_manager.cc:5697] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:26.684005  1733 raft_consensus.cc:359] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.684834  1733 raft_consensus.cc:385] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.685163  1733 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.685995  1733 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.688145  1733 ts_tablet_manager.cc:1434] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.021s	user 0.003s	sys 0.002s
W20260709 12:59:26.696789  1489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:26.697767  1734 tablet_bootstrap.cc:492] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:26.698289  1734 ts_tablet_manager.cc:1403] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.077s	user 0.015s	sys 0.008s
I20260709 12:59:26.698338  1733 tablet_bootstrap.cc:492] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:26.700390  1734 raft_consensus.cc:359] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.700927  1734 raft_consensus.cc:385] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.701185  1734 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.701862  1734 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.704108  1734 ts_tablet_manager.cc:1434] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:59:26.705189  1734 tablet_bootstrap.cc:492] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:26.708742  1733 tablet_bootstrap.cc:654] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.710404  1734 tablet_bootstrap.cc:654] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.715682  1732 tablet_bootstrap.cc:492] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:26.716445  1732 ts_tablet_manager.cc:1403] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.200s	user 0.023s	sys 0.003s
I20260709 12:59:26.723126  1732 raft_consensus.cc:359] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.724191  1732 raft_consensus.cc:385] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.724643  1732 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.725656  1732 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.762517  1732 ts_tablet_manager.cc:1434] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.046s	user 0.004s	sys 0.004s
I20260709 12:59:26.763650  1732 tablet_bootstrap.cc:492] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:26.764365  1734 tablet_bootstrap.cc:492] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:26.764825  1734 ts_tablet_manager.cc:1403] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.060s	user 0.022s	sys 0.035s
I20260709 12:59:26.767484  1734 raft_consensus.cc:359] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.768151  1734 raft_consensus.cc:385] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.768405  1734 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.769168  1733 tablet_bootstrap.cc:492] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:26.769055  1734 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.769685  1733 ts_tablet_manager.cc:1403] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.072s	user 0.023s	sys 0.020s
I20260709 12:59:26.771160  1734 ts_tablet_manager.cc:1434] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20260709 12:59:26.772358  1734 tablet_bootstrap.cc:492] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:26.772331  1733 raft_consensus.cc:359] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.773231  1733 raft_consensus.cc:385] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.773522  1733 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.774192  1733 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.772895  1732 tablet_bootstrap.cc:654] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.779260  1734 tablet_bootstrap.cc:654] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.779913  1733 ts_tablet_manager.cc:1434] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:59:26.783404  1733 tablet_bootstrap.cc:492] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:26.789024  1733 tablet_bootstrap.cc:654] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.791486  1955 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.792172  1955 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.795598  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:26.796253  1539 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:26.797581  1955 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:26.798563  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:26.799466  1616 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:26.801247  1732 tablet_bootstrap.cc:492] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:26.801738  1732 ts_tablet_manager.cc:1403] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.006s
I20260709 12:59:26.802568  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:26.803918  1955 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:26.804294  1955 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:26.804617  1955 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.805018  1732 raft_consensus.cc:359] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.805763  1732 raft_consensus.cc:385] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.806038  1732 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.806849  1732 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.808969  1732 ts_tablet_manager.cc:1434] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 12:59:26.810088  1732 tablet_bootstrap.cc:492] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:26.810411  1962 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.810974  1962 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.811684  1955 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.815382  1962 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.816668  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:26.817176  1955 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:26.817353  1692 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:26.818719  1582 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:26.816704  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 12:59:26.821458  1584 leader_election.cc:343] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.821952  1733 tablet_bootstrap.cc:492] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:26.822391  1733 ts_tablet_manager.cc:1403] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.001s
I20260709 12:59:26.824880  1732 tablet_bootstrap.cc:654] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.825879  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:26.826459  1539 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.831249  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:26.831836  1615 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.833900  1733 raft_consensus.cc:359] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.840834  1733 raft_consensus.cc:385] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.835930  2029 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:26.845657  1733 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.846496  1733 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.823006  1962 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:26.847957  1962 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:26.848392  1962 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.849334  1733 ts_tablet_manager.cc:1434] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.001s
I20260709 12:59:26.851887  1615 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:26.853123  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:26.853319  1733 tablet_bootstrap.cc:492] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:26.853818  1734 tablet_bootstrap.cc:492] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:26.853912  1955 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:26.854350  1734 ts_tablet_manager.cc:1403] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.082s	user 0.036s	sys 0.023s
I20260709 12:59:26.854452  1955 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:26.855535  1955 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.856009  1539 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:26.857324  1734 raft_consensus.cc:359] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.858507  1734 raft_consensus.cc:385] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.858781  1734 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.859608  1734 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.862347  1734 ts_tablet_manager.cc:1434] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 12:59:26.865556  1734 tablet_bootstrap.cc:492] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:26.871479  1733 tablet_bootstrap.cc:654] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.875834  1732 tablet_bootstrap.cc:492] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:26.876341  1732 ts_tablet_manager.cc:1403] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.066s	user 0.010s	sys 0.005s
I20260709 12:59:26.880415  1732 raft_consensus.cc:359] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.881203  1732 raft_consensus.cc:385] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.881603  1732 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.888284  1734 tablet_bootstrap.cc:654] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.892247  1732 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.895401  1439 catalog_manager.cc:5697] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:26.897862  1962 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.902019  1962 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:26.904760  1732 ts_tablet_manager.cc:1434] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 12:59:26.905871  1732 tablet_bootstrap.cc:492] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:26.907193  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:26.907811  1691 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.920513  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 12:59:26.923427  1584 leader_election.cc:343] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.926976  1733 tablet_bootstrap.cc:492] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:26.927928  1733 ts_tablet_manager.cc:1403] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.075s	user 0.013s	sys 0.007s
I20260709 12:59:26.928735  1732 tablet_bootstrap.cc:654] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:26.929513  1584 consensus_peers.cc:597] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:26.930523  1733 raft_consensus.cc:359] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.931176  1733 raft_consensus.cc:385] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.931222  1734 tablet_bootstrap.cc:492] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:26.931478  1733 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.932060  1734 ts_tablet_manager.cc:1403] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.067s	user 0.014s	sys 0.009s
I20260709 12:59:26.932286  1733 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.934778  1733 ts_tablet_manager.cc:1434] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:59:26.935798  1733 tablet_bootstrap.cc:492] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:26.936120  1734 raft_consensus.cc:359] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.937180  1734 raft_consensus.cc:385] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.937644  1734 raft_consensus.cc:740] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.938748  1734 consensus_queue.cc:260] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.942806  1733 tablet_bootstrap.cc:654] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.949664  1691 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:26.952450  1582 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:26.953648  1739 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:26.954087  1739 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:26.954803  1739 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.959266  1732 tablet_bootstrap.cc:492] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:26.959841  1732 ts_tablet_manager.cc:1403] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.014s
I20260709 12:59:26.962709  1732 raft_consensus.cc:359] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.963192  1955 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.963443  1732 raft_consensus.cc:385] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:26.963795  1732 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:26.963826  1955 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.964519  1732 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.966531  1734 ts_tablet_manager.cc:1434] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:59:26.968205  1734 tablet_bootstrap.cc:492] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:26.970528  1955 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:26.971352  1732 ts_tablet_manager.cc:1434] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:59:26.972155  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:26.974956  1541 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:26.974972  1828 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:59:26.979020  1658 leader_election.cc:343] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.978456  1437 catalog_manager.cc:5697] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:26.979730  1732 tablet_bootstrap.cc:492] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:26.984033  1734 tablet_bootstrap.cc:654] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.986259  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:26.988190  1968 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:26.988703  1968 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:26.990942  1968 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 12:59:26.992323  1658 leader_election.cc:343] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:26.992336  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:26.993036  1658 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:27.004535  1732 tablet_bootstrap.cc:654] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:26.997699  1955 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.013841  1543 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.018210  1955 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.018730  1955 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.026371  1733 tablet_bootstrap.cc:492] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:27.026849  1733 ts_tablet_manager.cc:1403] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.091s	user 0.020s	sys 0.029s
I20260709 12:59:27.029654  2030 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:27.030287  1955 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:27.019925  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:27.030776  1539 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.035559  1739 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.037509  1828 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.037704  1509 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:27.037961  1828 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.038130  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:27.039767  1828 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.036127  1739 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:27.044463  1658 leader_election.cc:343] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.055830  1733 raft_consensus.cc:359] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.056564  1733 raft_consensus.cc:385] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.059633  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:27.060333  1616 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:27.036945  1955 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.061861  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:27.062489  1687 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:27.063287  1734 tablet_bootstrap.cc:492] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:27.063925  1734 ts_tablet_manager.cc:1403] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.096s	user 0.014s	sys 0.008s
I20260709 12:59:27.064334  1739 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.066399  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:27.061035  1955 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.069299  1955 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:27.074783  1963 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:27.077358  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:27.056823  1733 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.080348  1582 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:27.084769  1828 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.085127  1828 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.082635  2027 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.082572  1864 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.085704  2027 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.087857  2027 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:27.090513  1828 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.090791  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:27.091439  1616 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:27.092168  1955 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.092562  1955 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.092867  1955 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.096854  1828 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:27.084055  1584 leader_election.cc:343] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.102428  1739 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:27.103965  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:27.108820  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:27.109516  1618 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:27.105856  1734 raft_consensus.cc:359] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.113021  1734 raft_consensus.cc:385] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.113874  1734 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.115445  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:27.116024  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:27.116534  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:27.116665  1955 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.112774  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 12:59:27.120080  1584 leader_election.cc:343] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.117560  1828 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.120499  1836 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.120626  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:27.120939  1836 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.121331  1836 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.121785  1955 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:27.122351  1955 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.122704  1955 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.123031  1955 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.113158  1733 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.123788  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:27.125411  1616 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 12:59:27.127573  1828 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.128193  1828 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.128916  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:27.131348  1955 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.131659  1828 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.132257  1732 tablet_bootstrap.cc:492] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:27.132730  1732 ts_tablet_manager.cc:1403] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.154s	user 0.030s	sys 0.056s
I20260709 12:59:27.136412  1732 raft_consensus.cc:359] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.137604  1732 raft_consensus.cc:385] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.137962  1955 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:27.137962  1732 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.139219  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:27.139393  1732 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:27.141659  1658 leader_election.cc:343] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.143358  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:27.144435  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:27.144867  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:27.145244  1691 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 1: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 12:59:27.145435  1616 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:27.148123  1658 leader_election.cc:343] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:27.149134  1658 leader_election.cc:343] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.149828  1732 ts_tablet_manager.cc:1434] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.005s
W20260709 12:59:27.157346  1582 leader_election.cc:343] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.157986  1582 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:27.154901  1836 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.160107  1836 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:27.161381  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:27.150241  1949 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.162010  1618 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.103770  2027 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.045s
I20260709 12:59:27.164847  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:27.161767  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:27.165244  1828 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.171639  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 12:59:27.177101  1582 leader_election.cc:343] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.181550  1732 tablet_bootstrap.cc:492] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:27.173471  2030 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.182932  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:27.175331  2026 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
W20260709 12:59:27.188918  1584 leader_election.cc:343] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.191829  1584 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:27.190511  1734 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.195333  1618 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:27.195830  1989 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.196614  1733 ts_tablet_manager.cc:1434] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.169s	user 0.021s	sys 0.026s
I20260709 12:59:27.196887  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:27.197953  2062 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:27.198325  1542 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:27.198451  2062 raft_consensus.cc:697] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:27.199790  2062 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.187255  2030 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.203377  2030 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.211045  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:27.211737  1614 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:27.213944  1734 ts_tablet_manager.cc:1434] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.150s	user 0.009s	sys 0.002s
I20260709 12:59:27.214988  1734 tablet_bootstrap.cc:492] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
W20260709 12:59:27.218699  1584 leader_election.cc:343] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.219129  1584 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:27.230829  1733 tablet_bootstrap.cc:492] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:27.222816  1989 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.243803  1733 tablet_bootstrap.cc:654] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.226218  1616 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:27.224216  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:27.261291  2065 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:27.261765  2065 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.262125  2065 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:27.236121  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:27.263346  1732 tablet_bootstrap.cc:654] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.264935  1734 tablet_bootstrap.cc:654] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.263530  2026 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.299247  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:27.300756  1949 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:27.301445  1949 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:27.302548  1949 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.338049  1440 catalog_manager.cc:5697] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:27.343173  2069 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.343986  2069 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.354980  2069 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.356863  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:27.360780  1734 tablet_bootstrap.cc:492] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:27.361253  1734 ts_tablet_manager.cc:1403] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.147s	user 0.011s	sys 0.005s
I20260709 12:59:27.312006  2026 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.364262  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:27.370388  1734 raft_consensus.cc:359] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.371207  1734 raft_consensus.cc:385] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.371593  1734 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.374001  1732 tablet_bootstrap.cc:492] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:27.374439  1732 ts_tablet_manager.cc:1403] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.196s	user 0.019s	sys 0.018s
I20260709 12:59:27.375057  1734 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.377472  2026 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:27.378093  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:27.325345  2065 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.379172  1543 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:59:27.379503  1734 ts_tablet_manager.cc:1434] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.018s	user 0.000s	sys 0.006s
W20260709 12:59:27.381042  1658 leader_election.cc:343] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.382130  1734 tablet_bootstrap.cc:492] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:27.385476  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:27.388797  1949 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
W20260709 12:59:27.391660  1658 leader_election.cc:343] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.392114  1658 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:27.388904  1732 raft_consensus.cc:359] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.392763  1732 raft_consensus.cc:385] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.393075  1732 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.393756  1732 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.394071  2063 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.401307  2065 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.402128  1509 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:27.402951  2069 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.407173  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:27.408383  1732 ts_tablet_manager.cc:1434] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.003s
I20260709 12:59:27.409456  1732 tablet_bootstrap.cc:492] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:27.410835  1733 tablet_bootstrap.cc:492] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:27.411471  1733 ts_tablet_manager.cc:1403] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.181s	user 0.011s	sys 0.004s
I20260709 12:59:27.410873  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:27.413352  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:27.414153  2030 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:59:27.415107  1949 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:27.415441  1949 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:27.416251  1949 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.417392  1949 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:27.417824  1733 raft_consensus.cc:359] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.418462  1733 raft_consensus.cc:385] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.418751  1733 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.419384  1733 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.421186  1733 ts_tablet_manager.cc:1434] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.000s
I20260709 12:59:27.422163  1733 tablet_bootstrap.cc:492] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:27.424355  1734 tablet_bootstrap.cc:654] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.424949  1732 tablet_bootstrap.cc:654] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.425503  1439 catalog_manager.cc:5697] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:27.447502  1733 tablet_bootstrap.cc:654] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.458665  1732 tablet_bootstrap.cc:492] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:27.459133  1732 ts_tablet_manager.cc:1403] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.019s
I20260709 12:59:27.460230  1733 tablet_bootstrap.cc:492] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:27.459151  1734 tablet_bootstrap.cc:492] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:27.460681  1733 ts_tablet_manager.cc:1403] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.001s
I20260709 12:59:27.461200  1734 ts_tablet_manager.cc:1403] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.079s	user 0.014s	sys 0.018s
I20260709 12:59:27.463186  1733 raft_consensus.cc:359] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.463749  1733 raft_consensus.cc:385] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.463763  1734 raft_consensus.cc:359] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.464248  1733 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.464439  1734 raft_consensus.cc:385] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.464756  1734 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.464906  1733 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.465384  1734 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.466365  2027 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:27.467020  1733 ts_tablet_manager.cc:1434] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:59:27.467536  1734 ts_tablet_manager.cc:1434] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:27.468005  1733 tablet_bootstrap.cc:492] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:27.468515  1734 tablet_bootstrap.cc:492] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:27.473016  1732 raft_consensus.cc:359] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.473722  1732 raft_consensus.cc:385] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.473982  1732 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.474592  1732 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.475220  1734 tablet_bootstrap.cc:654] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.476522  1732 ts_tablet_manager.cc:1434] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.000s
I20260709 12:59:27.477581  1732 tablet_bootstrap.cc:492] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:27.483938  1733 tablet_bootstrap.cc:654] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.483688  1949 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:27.486115  1734 tablet_bootstrap.cc:492] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:27.486583  1734 ts_tablet_manager.cc:1403] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.001s
I20260709 12:59:27.488143  1732 tablet_bootstrap.cc:654] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.489197  1734 raft_consensus.cc:359] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.489926  1734 raft_consensus.cc:385] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.490190  1734 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.490840  1734 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.495126  1734 ts_tablet_manager.cc:1434] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
W20260709 12:59:27.495707  1584 consensus_peers.cc:597] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:27.496088  1734 tablet_bootstrap.cc:492] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:27.498582  1828 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.499163  1828 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.500627  1541 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.503538  1828 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.504999  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:27.505822  1691 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:27.506973  1582 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:27.507182  1732 tablet_bootstrap.cc:492] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:27.508014  1732 ts_tablet_manager.cc:1403] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.004s
I20260709 12:59:27.505044  1734 tablet_bootstrap.cc:654] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.510078  1828 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.510421  1828 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.510685  1828 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.515920  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 12:59:27.522535  1584 leader_election.cc:343] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.523643  1733 tablet_bootstrap.cc:492] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:27.524574  1733 ts_tablet_manager.cc:1403] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.008s
I20260709 12:59:27.530323  1828 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.534703  1732 raft_consensus.cc:359] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.535216  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:27.537294  1828 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.535416  1732 raft_consensus.cc:385] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:59:27.537940  1584 leader_election.cc:343] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.538304  1732 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.539073  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:27.538970  1732 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.539585  1691 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.541026  1732 ts_tablet_manager.cc:1434] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.033s	user 0.003s	sys 0.002s
I20260709 12:59:27.545276  1734 tablet_bootstrap.cc:492] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:27.545783  1734 ts_tablet_manager.cc:1403] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.014s
I20260709 12:59:27.548251  1734 raft_consensus.cc:359] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.548913  1734 raft_consensus.cc:385] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.549170  1734 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.549803  1734 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.551077  2069 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.551571  2069 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.551826  1734 ts_tablet_manager.cc:1434] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:27.551779  1733 raft_consensus.cc:359] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.552403  1691 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:27.553623  2069 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.551049  1732 tablet_bootstrap.cc:492] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:27.554332  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:27.554723  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:27.554934  1615 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:27.552595  1733 raft_consensus.cc:385] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.556121  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:27.556401  1733 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.552934  1734 tablet_bootstrap.cc:492] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
W20260709 12:59:27.556658  1507 leader_election.cc:343] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.556926  2069 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.557291  2069 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.557632  2069 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.558175  1733 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.559844  1949 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.560447  1949 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.561142  1962 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.561663  1962 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.559986  1732 tablet_bootstrap.cc:654] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.563477  1949 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:27.563771  1962 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.565088  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:27.565646  1733 ts_tablet_manager.cc:1434] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.041s	user 0.005s	sys 0.001s
I20260709 12:59:27.565842  1537 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:27.566574  1733 tablet_bootstrap.cc:492] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:27.568053  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:27.568670  1537 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:27.568089  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:27.570447  2061 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.570879  2061 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.571271  2061 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.573927  1828 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:27.576421  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:27.579111  1734 tablet_bootstrap.cc:654] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.583498  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:27.584859  1733 tablet_bootstrap.cc:654] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.585462  1739 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.585819  1739 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.586161  1739 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.587080  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:27.593781  1739 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.594959  1615 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 12:59:27.595304  2069 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.598088  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:27.598694  1615 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.599014  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:27.606729  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:27.607383  1541 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.612361  2027 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.613399  2027 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.614174  1582 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:27.614521  1541 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:27.616001  1543 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.616258  1962 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:27.597690  1739 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.617010  1962 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:27.620134  2069 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.619933  1962 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.615979  2027 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:27.626981  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 12:59:27.629472  1658 leader_election.cc:343] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.629320  2027 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:27.634753  1733 tablet_bootstrap.cc:492] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:27.635708  1733 ts_tablet_manager.cc:1403] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.004s
I20260709 12:59:27.637123  1615 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:27.641901  2061 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.642812  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:27.643683  2069 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:27.644233  2069 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:27.645080  2069 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.643736  1692 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 1: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 12:59:27.646536  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:27.648588  1543 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 12:59:27.650869  1734 tablet_bootstrap.cc:492] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:27.651353  1734 ts_tablet_manager.cc:1403] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.099s	user 0.012s	sys 0.008s
I20260709 12:59:27.651196  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 12:59:27.652588  1507 leader_election.cc:343] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.653993  1733 raft_consensus.cc:359] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.659847  1733 raft_consensus.cc:385] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.660162  1733 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.660815  1733 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.664014  2061 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:27.664523  1733 ts_tablet_manager.cc:1434] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.024s	user 0.002s	sys 0.003s
I20260709 12:59:27.665248  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:27.665602  1733 tablet_bootstrap.cc:492] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:27.662357  1734 raft_consensus.cc:359] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.669727  1734 raft_consensus.cc:385] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.670035  1734 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.670787  1734 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.671566  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:27.672333  1617 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 12:59:27.673249  1734 ts_tablet_manager.cc:1434] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:59:27.673470  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 12:59:27.674283  1734 tablet_bootstrap.cc:492] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:27.675410  1739 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:27.675872  1739 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:27.676569  1739 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.677083  1733 tablet_bootstrap.cc:654] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.679840  1440 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:27.687295  1734 tablet_bootstrap.cc:654] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.694599  1733 tablet_bootstrap.cc:492] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:27.695142  1733 ts_tablet_manager.cc:1403] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.011s
I20260709 12:59:27.695343  1734 tablet_bootstrap.cc:492] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:27.696476  1734 ts_tablet_manager.cc:1403] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.003s
W20260709 12:59:27.697829  1658 leader_election.cc:343] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.698311  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:27.698053  1733 raft_consensus.cc:359] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.700062  1733 raft_consensus.cc:385] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.700371  2061 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.700397  1733 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.700914  1437 catalog_manager.cc:5697] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:27.701546  1733 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.704728  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:27.705478  2027 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.706384  1733 ts_tablet_manager.cc:1434] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:59:27.707444  1733 tablet_bootstrap.cc:492] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:27.707702  1734 raft_consensus.cc:359] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.708392  1734 raft_consensus.cc:385] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.708667  1734 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.709482  1734 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.711942  1734 ts_tablet_manager.cc:1434] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 12:59:27.713009  1734 tablet_bootstrap.cc:492] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:27.718091  1733 tablet_bootstrap.cc:654] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.719420  1734 tablet_bootstrap.cc:654] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.725652  2062 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:27.727523  1732 tablet_bootstrap.cc:492] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:27.728046  1732 ts_tablet_manager.cc:1403] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.177s	user 0.034s	sys 0.016s
I20260709 12:59:27.743738  1541 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.748067  1733 tablet_bootstrap.cc:492] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:27.750607  1437 catalog_manager.cc:5697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:27.748900  1733 ts_tablet_manager.cc:1403] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.005s
I20260709 12:59:27.766886  2062 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.767382  2062 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.768476  1734 tablet_bootstrap.cc:492] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:27.769006  1734 ts_tablet_manager.cc:1403] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.056s	user 0.018s	sys 0.022s
I20260709 12:59:27.769040  2026 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:27.771695  2062 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:27.771991  1732 raft_consensus.cc:359] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.772684  1732 raft_consensus.cc:385] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.772958  1732 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.773592  1732 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.778129  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:27.775592  1732 ts_tablet_manager.cc:1434] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.000s
I20260709 12:59:27.777804  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:27.779323  1615 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:27.779796  1732 tablet_bootstrap.cc:492] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
W20260709 12:59:27.780814  1658 leader_election.cc:343] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.781560  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:27.772181  1734 raft_consensus.cc:359] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.785079  1734 raft_consensus.cc:385] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.785470  1734 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.784667  1836 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.785985  1836 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.786316  1836 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.786276  1734 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.790973  1734 ts_tablet_manager.cc:1434] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 12:59:27.794478  1734 tablet_bootstrap.cc:492] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:27.794552  2026 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:59:27.802117  1734 tablet_bootstrap.cc:654] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:27.815349  1489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:27.815785  1962 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.816490  1962 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.816792  1836 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.819278  1962 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.820305  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:27.821872  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 12:59:27.822289  1658 leader_election.cc:343] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.815588  1733 raft_consensus.cc:359] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.825923  1733 raft_consensus.cc:385] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.826336  1733 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
W20260709 12:59:27.827009  1584 leader_election.cc:343] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.827471  1836 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:27.827023  1733 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.830231  1733 ts_tablet_manager.cc:1434] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.075s	user 0.006s	sys 0.003s
I20260709 12:59:27.830796  1732 tablet_bootstrap.cc:654] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.830467  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:27.831442  1614 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.832777  1739 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.833334  1739 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.835501  1739 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.842068  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 12:59:27.843674  1584 leader_election.cc:343] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.897137  1733 tablet_bootstrap.cc:492] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:27.901616  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:27.899515  1734 tablet_bootstrap.cc:492] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:27.915818  1691 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:27.915968  1734 ts_tablet_manager.cc:1403] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.122s	user 0.014s	sys 0.046s
I20260709 12:59:27.906301  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:27.915095  1732 tablet_bootstrap.cc:492] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:27.916893  1732 ts_tablet_manager.cc:1403] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.137s	user 0.010s	sys 0.009s
I20260709 12:59:27.918771  1734 raft_consensus.cc:359] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.919481  1734 raft_consensus.cc:385] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.919734  1734 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.920751  1734 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.922662  1614 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:27.922971  1734 ts_tablet_manager.cc:1434] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:27.924856  1734 tablet_bootstrap.cc:492] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:27.924856  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:27.926591  2061 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:27.927075  2061 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:27.927016  1732 raft_consensus.cc:359] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.927964  1732 raft_consensus.cc:385] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.928301  1732 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.928169  2061 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.930450  2069 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.930919  2069 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.929615  1732 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.932929  1734 tablet_bootstrap.cc:654] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.932896  2069 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.933734  1732 ts_tablet_manager.cc:1434] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 12:59:27.934758  1732 tablet_bootstrap.cc:492] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:27.938686  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:27.941213  1732 tablet_bootstrap.cc:654] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.943429  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:27.945613  1507 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:27.947417  2069 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.952420  2069 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.952886  2069 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.953655  1734 tablet_bootstrap.cc:492] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:27.954128  1734 ts_tablet_manager.cc:1403] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.002s
I20260709 12:59:27.956131  1582 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:27.956701  1732 tablet_bootstrap.cc:492] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:27.957258  1732 ts_tablet_manager.cc:1403] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.023s	user 0.005s	sys 0.012s
I20260709 12:59:27.956996  1734 raft_consensus.cc:359] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.957793  1734 raft_consensus.cc:385] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.957775  1739 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:27.958231  1734 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.958384  1739 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:27.958719  1739 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:27.958956  1734 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.961040  1734 ts_tablet_manager.cc:1434] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:59:27.960790  1732 raft_consensus.cc:359] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.961712  1732 raft_consensus.cc:385] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:27.962044  1732 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:27.962101  1734 tablet_bootstrap.cc:492] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:27.962853  1732 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:27.963979  1582 leader_election.cc:343] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.964411  1582 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:27.965113  1732 ts_tablet_manager.cc:1434] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:59:27.965094  1962 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:27.966176  2065 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.966641  2065 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.968415  1734 tablet_bootstrap.cc:654] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:27.968715  2065 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.972083  1732 tablet_bootstrap.cc:492] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:27.971957  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 12:59:27.973171  1507 leader_election.cc:343] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:27.973338  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:27.974037  1691 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 12:59:27.978694  1739 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.983982  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:27.984669  1617 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:27.969923  2069 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:27.990171  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 12:59:27.990492  2065 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:27.991832  2065 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:27.999603  1739 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.000656  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:28.001278  1691 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.008446  1732 tablet_bootstrap.cc:654] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.012770  1691 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:28.014281  1582 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:28.015345  1739 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:28.015839  1739 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:28.016811  1739 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.007311  2069 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.028826  1733 tablet_bootstrap.cc:654] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.027297  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.029498  2065 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.029983  1614 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 12:59:28.032922  1734 tablet_bootstrap.cc:492] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:28.033357  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:28.033404  1734 ts_tablet_manager.cc:1403] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.072s	user 0.010s	sys 0.048s
I20260709 12:59:28.036475  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:28.037104  1537 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.037300  1541 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.023876  2069 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.043473  2069 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.047111  2063 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:28.047336  1507 leader_election.cc:343] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.064705  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.116832  1509 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:28.117769  2065 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.122612  1439 catalog_manager.cc:5697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:28.126210  2063 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.127719  1617 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:28.128304  2063 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:28.131012  1537 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
W20260709 12:59:28.132058  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 72731.9us
I20260709 12:59:28.134002  1734 raft_consensus.cc:359] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.136097  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.136277  1734 raft_consensus.cc:385] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.140209  1734 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.140942  1734 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.139294  1437 catalog_manager.cc:5697] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 1, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:28.144201  2062 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.144698  2062 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.145187  1733 tablet_bootstrap.cc:492] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:28.145678  1733 ts_tablet_manager.cc:1403] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.249s	user 0.016s	sys 0.001s
W20260709 12:59:28.146317  1658 leader_election.cc:343] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.146663  2062 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:28.148288  1733 raft_consensus.cc:359] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.148927  1733 raft_consensus.cc:385] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.149168  1733 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.149782  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.149905  1733 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.151432  1733 ts_tablet_manager.cc:1434] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:59:28.152330  1733 tablet_bootstrap.cc:492] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:28.166216  2069 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:28.168131  1658 leader_election.cc:343] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.169984  1989 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:28.172066  1989 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:28.176468  1732 tablet_bootstrap.cc:492] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:28.176873  2065 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.176949  1732 ts_tablet_manager.cc:1403] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.205s	user 0.010s	sys 0.004s
I20260709 12:59:28.180241  1732 raft_consensus.cc:359] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.181022  1732 raft_consensus.cc:385] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.181295  1732 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.181941  1732 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.185185  1734 ts_tablet_manager.cc:1434] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.151s	user 0.011s	sys 0.007s
I20260709 12:59:28.186756  1734 tablet_bootstrap.cc:492] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:28.190600  2069 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.187789  2091 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.191321  2091 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.194486  2091 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.195102  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 12:59:28.196403  1507 leader_election.cc:343] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.196794  1732 ts_tablet_manager.cc:1434] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 12:59:28.197871  1732 tablet_bootstrap.cc:492] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
W20260709 12:59:28.204188  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 82173us
I20260709 12:59:28.205861  1509 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 12:59:28.206156  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:28.206434  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:28.206969  1617 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.207077  1688 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:28.208401  2112 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.208819  2112 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.209148  2112 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.209278  1582 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 12:59:28.211735  1507 consensus_peers.cc:597] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:28.215258  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.218607  1617 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:28.220064  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.220585  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 12:59:28.220775  1614 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 0.
I20260709 12:59:28.221796  2065 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:28.222478  2065 raft_consensus.cc:697] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:28.223560  1739 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.223357  2065 consensus_queue.cc:237] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.223951  1739 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.224298  1739 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.227025  1732 tablet_bootstrap.cc:654] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.230240  1734 tablet_bootstrap.cc:654] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.232753  1739 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.240794  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.241894  2063 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:59:28.243743  1658 leader_election.cc:343] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.247412  1542 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.252641  2107 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:28.260314  2063 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.262356  2063 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:28.264022  1739 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.252583  2112 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.259557  1658 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:28.277221  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:28.282608  1836 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.283111  1836 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.283521  1836 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.294327  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.297392  1733 tablet_bootstrap.cc:654] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:28.297688  1658 leader_election.cc:343] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.298554  1836 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.284721  2063 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.300783  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:28.304109  2110 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.322317  1732 tablet_bootstrap.cc:492] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:28.322592  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:28.322808  1732 ts_tablet_manager.cc:1403] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.125s	user 0.032s	sys 0.019s
I20260709 12:59:28.323360  1687 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 1: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 12:59:28.324522  1836 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:28.327836  2065 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.337182  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:28.326668  2110 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.337769  2065 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.318770  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.342119  2110 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 12:59:28.344373  1584 leader_election.cc:343] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:28.347433  1584 leader_election.cc:343] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.347288  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:28.347843  1584 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:28.348163  1734 tablet_bootstrap.cc:492] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:28.348317  1617 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 12:59:28.348651  1734 ts_tablet_manager.cc:1403] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.162s	user 0.021s	sys 0.020s
I20260709 12:59:28.351737  2065 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.354250  2069 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.354173  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:28.353540  1732 raft_consensus.cc:359] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.355494  1732 raft_consensus.cc:385] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.355795  1732 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
W20260709 12:59:28.353085  1658 leader_election.cc:343] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:59:28.357757  1658 leader_election.cc:343] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.357331  1732 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.358192  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:28.350548  2110 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.354852  2069 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.359865  1692 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:28.366801  1734 raft_consensus.cc:359] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.367548  1734 raft_consensus.cc:385] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.367794  2062 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.368232  2112 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.367828  1734 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.369226  1734 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.369827  1732 ts_tablet_manager.cc:1434] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 12:59:28.371012  1732 tablet_bootstrap.cc:492] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:28.371475  1734 ts_tablet_manager.cc:1434] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.002s
I20260709 12:59:28.372380  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.372624  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:28.373003  1616 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 0.
I20260709 12:59:28.373178  1618 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.374389  1734 tablet_bootstrap.cc:492] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:28.379208  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.380507  1439 catalog_manager.cc:5697] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:28.382553  1644 tablet.cc:2406] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:59:28.384091  1509 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:28.385780  2107 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.386149  2107 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.386468  2107 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.378358  2069 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.392115  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:28.394831  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.398138  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:28.400022  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:28.400529  2110 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:28.402138  1618 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:28.402806  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:28.403353  1509 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:28.404235  2112 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:28.404727  2112 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:28.404831  2107 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.405983  1949 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.405807  2112 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.407866  2107 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 12:59:28.408505  1584 leader_election.cc:343] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.409336  1734 tablet_bootstrap.cc:654] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.410398  1582 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:28.410792  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:28.411351  1618 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.411549  1962 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.411693  2091 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:28.411967  1962 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.412550  1962 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.420812  1618 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 12:59:28.422346  1509 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:28.423821  1733 tablet_bootstrap.cc:492] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:28.424315  1733 ts_tablet_manager.cc:1403] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.272s	user 0.009s	sys 0.013s
I20260709 12:59:28.425722  2062 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.426250  2062 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.426839  1733 raft_consensus.cc:359] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.427506  1733 raft_consensus.cc:385] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.427762  1733 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.428369  2062 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:28.428575  2112 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:28.429075  2112 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:28.429908  2112 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.428417  1733 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.437443  1733 ts_tablet_manager.cc:1434] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 12:59:28.438446  1733 tablet_bootstrap.cc:492] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:28.442939  1962 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.454943  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:28.456918  2110 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:59:28.458443  1584 leader_election.cc:343] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.444885  1732 tablet_bootstrap.cc:654] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
W20260709 12:59:28.463485  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 56214us
I20260709 12:59:28.452590  1733 tablet_bootstrap.cc:654] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.468236  1962 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.475732  1733 tablet_bootstrap.cc:492] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:28.476238  1733 ts_tablet_manager.cc:1403] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.038s	user 0.005s	sys 0.007s
I20260709 12:59:28.478792  1733 raft_consensus.cc:359] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.479445  1733 raft_consensus.cc:385] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.479725  1733 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.480320  1733 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 12:59:28.480858  1507 leader_election.cc:343] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.482203  1733 ts_tablet_manager.cc:1434] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:59:28.483527  1733 tablet_bootstrap.cc:492] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
W20260709 12:59:28.484683  1507 leader_election.cc:343] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.484305  1440 catalog_manager.cc:5697] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:28.486557  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:28.486155  1440 catalog_manager.cc:5697] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:28.487242  1687 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.490501  1732 tablet_bootstrap.cc:492] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:28.490959  1732 ts_tablet_manager.cc:1403] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.120s	user 0.007s	sys 0.003s
I20260709 12:59:28.491349  1507 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:28.491660  1733 tablet_bootstrap.cc:654] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.492647  2125 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:59:28.497663  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.499692  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.503844  1734 tablet_bootstrap.cc:492] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:28.507846  1734 ts_tablet_manager.cc:1403] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.134s	user 0.010s	sys 0.002s
I20260709 12:59:28.507946  1687 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:28.509547  1739 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.510020  1739 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.510701  1582 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:28.511458  2110 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:28.511889  1739 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.511898  2110 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:28.513270  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:28.514148  1687 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
W20260709 12:59:28.515342  1658 leader_election.cc:343] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.514361  2110 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.516038  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.517781  1732 raft_consensus.cc:359] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.518477  1732 raft_consensus.cc:385] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:59:28.519122  1584 leader_election.cc:343] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.518757  1732 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.521368  1582 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:28.527256  1962 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:59:28.527475  1658 leader_election.cc:343] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.527855  1962 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.528187  1658 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:28.528410  1962 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.529042  2062 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.520828  1734 raft_consensus.cc:359] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.530040  1734 raft_consensus.cc:385] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.530330  1734 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.520797  1732 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.533977  1732 ts_tablet_manager.cc:1434] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 12:59:28.535101  1732 tablet_bootstrap.cc:492] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:28.539408  1734 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.520756  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:28.551425  1734 ts_tablet_manager.cc:1434] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 12:59:28.552316  1734 tablet_bootstrap.cc:492] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
W20260709 12:59:28.556128  1658 consensus_peers.cc:597] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:28.559348  1739 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:28.569339  1437 catalog_manager.cc:5697] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:28.574400  1962 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.593384  1949 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:28.594336  1687 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:59:28.595624  1507 leader_election.cc:343] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.598129  1732 tablet_bootstrap.cc:654] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.621528  1733 tablet_bootstrap.cc:492] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:28.621316  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:28.621984  1733 ts_tablet_manager.cc:1403] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.139s	user 0.033s	sys 0.037s
W20260709 12:59:28.623575  1584 leader_election.cc:343] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.636467  1962 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.638775  1734 tablet_bootstrap.cc:654] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.643935  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:28.645671  1691 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.643877  1733 raft_consensus.cc:359] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.647601  1733 raft_consensus.cc:385] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.648003  1733 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
W20260709 12:59:28.650313  1507 consensus_peers.cc:597] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:28.657783  1733 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.659907  1733 ts_tablet_manager.cc:1434] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.001s
I20260709 12:59:28.664300  1734 tablet_bootstrap.cc:492] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:28.664795  1734 ts_tablet_manager.cc:1403] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.113s	user 0.010s	sys 0.003s
I20260709 12:59:28.666208  2110 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.666702  2110 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.667348  1734 raft_consensus.cc:359] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.667994  1734 raft_consensus.cc:385] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.668262  1734 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.668788  2110 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.669123  1734 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.670279  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.670953  1536 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:28.672734  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:28.673852  1733 tablet_bootstrap.cc:492] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:28.672926  1691 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:28.675092  1962 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.683280  1962 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.683671  1962 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.679766  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:28.680374  1734 ts_tablet_manager.cc:1434] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.003s
I20260709 12:59:28.682603  1733 tablet_bootstrap.cc:654] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.689872  1692 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:28.691044  1734 tablet_bootstrap.cc:492] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:28.693346  2123 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:28.694430  1962 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.698191  1962 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.698870  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:28.702448  1734 tablet_bootstrap.cc:654] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.702720  1538 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.710207  1538 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:28.716048  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:28.717783  1734 tablet_bootstrap.cc:492] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:28.718737  1734 ts_tablet_manager.cc:1403] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.003s
I20260709 12:59:28.718894  2110 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:28.719461  2110 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:28.720278  2110 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.721238  1734 raft_consensus.cc:359] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.721987  1734 raft_consensus.cc:385] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.722251  1734 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.722987  1734 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.723680  1582 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:28.723717  1732 tablet_bootstrap.cc:492] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:28.724714  1732 ts_tablet_manager.cc:1403] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.190s	user 0.007s	sys 0.013s
I20260709 12:59:28.725142  1962 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:28.725922  1962 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:28.727226  1962 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.732177  1734 ts_tablet_manager.cc:1434] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:59:28.733467  1734 tablet_bootstrap.cc:492] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:28.735304  2063 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.735445  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:28.735986  1691 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.735844  2063 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.738062  2063 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:28.739401  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.740331  1536 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:28.741611  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:28.742338  2063 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:28.742719  2063 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.743064  2063 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:28.743217  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.744150  1613 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:28.751243  2063 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.753242  2063 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:28.753813  1691 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:28.727759  1732 raft_consensus.cc:359] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.755255  1732 raft_consensus.cc:385] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.755380  1734 tablet_bootstrap.cc:654] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.755571  1732 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.756094  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:28.756943  1614 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:28.764118  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:28.765000  1536 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:28.767973  2062 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.768518  2062 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.770458  2062 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:28.781322  1732 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.794942  1733 tablet_bootstrap.cc:492] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:28.795647  1733 ts_tablet_manager.cc:1403] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.122s	user 0.017s	sys 0.009s
I20260709 12:59:28.797789  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.803224  1732 ts_tablet_manager.cc:1434] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.078s	user 0.008s	sys 0.000s
I20260709 12:59:28.815637  1949 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.816829  1949 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.805665  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.805938  2107 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.827790  2107 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.829981  2107 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.832849  1614 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 12:59:28.837793  1732 tablet_bootstrap.cc:492] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: Bootstrap starting.
I20260709 12:59:28.827795  1613 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 1: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 12:59:28.848583  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.839880  1733 raft_consensus.cc:359] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.851899  1733 raft_consensus.cc:385] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.852216  1733 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
W20260709 12:59:28.854096  1489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:28.838670  1949 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:28.859453  1733 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.861084  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:28.861577  1691 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.842499  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.863142  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:28.863884  1692 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 12:59:28.866693  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:28.871589  1733 ts_tablet_manager.cc:1434] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.076s	user 0.009s	sys 0.001s
I20260709 12:59:28.872905  1440 catalog_manager.cc:5697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:28.867537  2062 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:28.870924  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.869863  2110 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.871042  1949 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.884881  1536 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 12:59:28.877691  1613 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 1: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 12:59:28.878108  1733 tablet_bootstrap.cc:492] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Bootstrap starting.
I20260709 12:59:28.871628  2061 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.880256  2110 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.875810  1507 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:28.877045  2062 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:28.887831  2061 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.888696  1739 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 12:59:28.891275  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.887176  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:28.890614  2110 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.894456  2107 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.893520  2063 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.895355  2112 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:59:28.896554  1584 leader_election.cc:343] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:28.893311  2062 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.898285  1949 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.893818  2061 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:28.891762  1739 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.902350  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.904997  1733 tablet_bootstrap.cc:654] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.917309  1739 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.920637  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:28.921563  1688 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 12:59:28.893857  1732 tablet_bootstrap.cc:654] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.894729  1734 tablet_bootstrap.cc:492] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:28.922410  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.922842  1734 ts_tablet_manager.cc:1403] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.190s	user 0.036s	sys 0.035s
I20260709 12:59:28.926281  1734 raft_consensus.cc:359] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.928695  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:28.927455  1734 raft_consensus.cc:385] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.930322  1734 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.930941  1734 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.929405  1440 catalog_manager.cc:5697] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:28.938382  1732 tablet_bootstrap.cc:492] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: No bootstrap required, opened a new log
I20260709 12:59:28.938879  1732 ts_tablet_manager.cc:1403] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent bootstrapping tablet: real 0.124s	user 0.009s	sys 0.003s
I20260709 12:59:28.942768  1732 raft_consensus.cc:359] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.943444  1732 raft_consensus.cc:385] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.943712  1732 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.944320  1732 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.946611  2112 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.931933  1733 tablet_bootstrap.cc:492] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: No bootstrap required, opened a new log
I20260709 12:59:28.946748  1732 ts_tablet_manager.cc:1434] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:59:28.949009  1733 ts_tablet_manager.cc:1403] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Time spent bootstrapping tablet: real 0.072s	user 0.015s	sys 0.016s
I20260709 12:59:28.947103  2112 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.933121  1734 ts_tablet_manager.cc:1434] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 12:59:28.922461  2107 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.933748  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:28.951728  1542 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 2: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 12:59:28.951633  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:28.934427  1989 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:28.952354  1734 tablet_bootstrap.cc:492] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:28.952462  1614 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 12:59:28.964319  2107 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:28.966933  2107 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.948218  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:28.960857  2112 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.969063  2107 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.963560  1439 catalog_manager.cc:5697] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 2, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 2 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:28.970309  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 12:59:28.971932  1584 consensus_peers.cc:597] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:28.973011  1734 tablet_bootstrap.cc:654] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:28.948544  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:28.975888  1691 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:28.977108  1582 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:28.978349  2110 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:28.978742  2110 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:28.977883  2063 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:28.979218  2110 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.952571  1989 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 12:59:28.981009  1989 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:28.984961  1733 raft_consensus.cc:359] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.986227  1733 raft_consensus.cc:385] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:28.989334  1733 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Initialized, Role: FOLLOWER
I20260709 12:59:28.990170  1733 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.993376  2110 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.994139  1739 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:28.995530  2110 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.993335  1989 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:28.996773  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:28.997342  1692 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:28.998421  1989 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:28.999468  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:29.001762  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:29.002053  1614 raft_consensus.cc:2410] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:59:29.002097  1733 ts_tablet_manager.cc:1434] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Time spent starting tablet: real 0.053s	user 0.008s	sys 0.000s
I20260709 12:59:29.002590  1542 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 2: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 12:59:29.005610  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 12:59:29.006331  1584 leader_election.cc:343] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:59:29.004990  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:29.007735  1688 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 12:59:29.009843  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:29.011049  1507 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:29.011982  2123 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.012428  2112 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.013317  2123 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.016633  1536 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.018685  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:29.020188  2125 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:59:29.021950  1692 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:29.026530  1582 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:29.028343  1989 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:29.029004  1989 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:29.030050  1989 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:29.031720  1734 tablet_bootstrap.cc:492] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:29.032385  1734 ts_tablet_manager.cc:1403] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.080s	user 0.013s	sys 0.005s
I20260709 12:59:29.042637  2069 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:29.049767  1692 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.025241  1687 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 12:59:29.058320  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 12:59:29.059067  1989 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:29.059727  1989 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:29.060889  1989 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:29.068022  2123 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:29.076924  1734 raft_consensus.cc:359] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:29.078740  1734 raft_consensus.cc:385] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:29.079317  1739 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:29.079610  1734 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:29.080035  1739 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:29.082274  1739 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:29.081278  1734 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:29.085251  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:29.086509  1734 ts_tablet_manager.cc:1434] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.054s	user 0.013s	sys 0.000s
I20260709 12:59:29.088835  1734 tablet_bootstrap.cc:492] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Bootstrap starting.
I20260709 12:59:29.086634  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:29.094578  1687 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:29.086439  1538 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 0.
I20260709 12:59:29.098939  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:29.100199  2115 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:59:29.100667  2115 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:29.101001  2115 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.139909  2112 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:29.155145  2112 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:29.200973  1734 tablet_bootstrap.cc:654] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Neither blocks nor log segments found. Creating new log.
I20260709 12:59:29.201906  2112 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:29.214083  2115 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:29.217033  2115 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 12:59:29.241071  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 52101.2us
W20260709 12:59:29.241629  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 98010.4us
I20260709 12:59:29.324277  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:29.325079  1692 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.333976  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:29.340118  1692 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:29.330322  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:29.355930  1542 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.359869  2162 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.364681  1734 tablet_bootstrap.cc:492] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: No bootstrap required, opened a new log
I20260709 12:59:29.365216  1734 ts_tablet_manager.cc:1403] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Time spent bootstrapping tablet: real 0.277s	user 0.016s	sys 0.026s
I20260709 12:59:29.365964  2161 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.367013  1542 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 12:59:29.367825  1734 raft_consensus.cc:359] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:29.368542  1734 raft_consensus.cc:385] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:59:29.368801  1734 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Initialized, Role: FOLLOWER
I20260709 12:59:29.369573  1739 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.369410  1734 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:29.369735  1582 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:29.373955  1734 ts_tablet_manager.cc:1434] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:59:29.375666  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 12:59:29.377447  1692 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 1: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 12:59:29.384153  1542 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.394737  1543 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:59:29.398617  2115 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:59:29.399204  2115 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:29.400079  2115 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:29.405414  2163 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.423105  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 12:59:29.424021  2069 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:59:29.426445  1613 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:59:29.434198  2160 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:29.435359  1437 catalog_manager.cc:5697] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:29.438055  2069 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:59:29.438269  1437 catalog_manager.cc:5697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 1 to 2, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:29.452193  2115 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.505327  1739 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:29.528731  1739 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:59:29.558820  1687 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:29.566390  1437 catalog_manager.cc:5697] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:29.573823  2161 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:29.608728  1538 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:29.610577  2115 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:59:29.889720  1489 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:59:29.919148  2162 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:29.935101  1989 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:30.091773  2163 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:59:30.092868  2163 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:30.094784  2163 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:30.095453  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 12:59:30.095746  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 12:59:30.096107  1542 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:30.096302  1614 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 12:59:30.097735  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:30.098452  2160 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:59:30.098829  2160 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:59:30.099207  2160 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:30.104758  2160 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:30.106624  2160 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:30.107249  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:30.107420  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:30.107731  1542 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:30.107964  1614 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:30.112375  1542 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 12:59:30.112680  1614 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 12:59:30.113729  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:30.114446  2160 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:30.114907  2160 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:30.115710  2160 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:30.126801  1438 catalog_manager.cc:5697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 0 to 2, leader changed from <none> to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 2 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:30.598817  1542 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:59:30.600004  2160 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:30.612051  1614 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:59:30.614753  2163 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:59:31.613238   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:59:31.644429  2191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:59:31.644923  2192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:31.645556   431 server_base.cc:1061] running on GCE node
W20260709 12:59:31.646179  2194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:59:31.647365   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:59:31.647574   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:59:31.647715   431 hybrid_clock.cc:648] HybridClock initialized: now 1783601971647703 us; error 0 us; skew 500 ppm
I20260709 12:59:31.648254   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:59:31.651520   431 webserver.cc:533] Webserver started at http://127.0.107.196:39193/ using document root <none> and password file <none>
I20260709 12:59:31.652280   431 fs_manager.cc:362] Metadata directory not provided
I20260709 12:59:31.652541   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:59:31.652958   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:59:31.655202   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-3-root/instance:
uuid: "cecad4f86a114a7696e398b3e8cc511e"
format_stamp: "Formatted at 2026-07-09 12:59:31 on dist-test-slave-jk05"
I20260709 12:59:31.661563   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:59:31.665040  2199 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:59:31.666014   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:59:31.666296   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-3-root
uuid: "cecad4f86a114a7696e398b3e8cc511e"
format_stamp: "Formatted at 2026-07-09 12:59:31 on dist-test-slave-jk05"
I20260709 12:59:31.666553   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:59:31.700151   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:59:31.701754   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:59:31.705775   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:59:31.709342   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:59:31.709748   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:59:31.710088   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:59:31.710367   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:59:31.763090   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:39293
I20260709 12:59:31.763181  2261 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:39293 every 8 connection(s)
I20260709 12:59:31.769371   431 auto_leader_rebalancer-test.cc:384] add a tserver: cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:31.782938  2262 heartbeater.cc:344] Connected to a master server at 127.0.107.254:36375
I20260709 12:59:31.783385  2262 heartbeater.cc:461] Registering TS with master...
I20260709 12:59:31.784058  2262 heartbeater.cc:507] Master 127.0.107.254:36375 requested a full tablet report, sending...
I20260709 12:59:31.785871  1439 ts_manager.cc:194] Registered new tserver with Master: cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 12:59:31.787396  1439 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40172
I20260709 12:59:32.047079  1687 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:32.054992  1614 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:32.058395  1543 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:32.059278  2163 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:32.064253  2186 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:32.076929  1657 consensus_peers.cc:597] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e2c52c22aef94ec6b8226f64dc4baca4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:32.082463  2163 raft_consensus.cc:2955] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:32.084794  1543 raft_consensus.cc:2955] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:32.090437  1614 raft_consensus.cc:2955] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:32.097011  1439 catalog_manager.cc:5697] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:32.103226  1687 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:32.113673  1543 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:32.115409  1657 consensus_peers.cc:597] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ad1245d9a7bf4a21ac3d2291898dd976. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:32.116045  2163 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:32.117260  1614 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:32.119154  2163 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:32.126181  2160 raft_consensus.cc:2955] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:32.140763  1542 raft_consensus.cc:2955] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:32.149026  1614 raft_consensus.cc:2955] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:32.162806  1437 catalog_manager.cc:5697] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:32.179481  1542 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:32.194944  1691 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:32.196866  2190 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:32.198647  1613 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:32.199901  2190 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:32.206501  2183 raft_consensus.cc:2955] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
W20260709 12:59:32.210939  1508 consensus_peers.cc:597] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 214bc9e9284a4502844dbdc3395843fa. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:32.214073  1613 raft_consensus.cc:2955] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:32.214342  1691 raft_consensus.cc:2955] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:32.222608  1543 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:32.236905  1687 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:32.237255  1508 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7e984cd665a4768933ff09e8ba0e07b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:32.237277  1614 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:32.238680  2284 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:32.235791  1437 catalog_manager.cc:5697] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:32.241514  2190 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:32.252985  2183 raft_consensus.cc:2955] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:32.255406  1687 raft_consensus.cc:2955] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:32.284088  1437 catalog_manager.cc:5697] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:32.284408  1614 raft_consensus.cc:2955] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:32.566576  2291 ts_tablet_manager.cc:933] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:32.571573  2291 tablet_copy_client.cc:323] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 12:59:32.577015  2293 ts_tablet_manager.cc:933] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:32.594614  2293 tablet_copy_client.cc:323] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 12:59:32.600023  2297 ts_tablet_manager.cc:933] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
I20260709 12:59:32.600942  1702 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet ad1245d9a7bf4a21ac3d2291898dd976 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 12:59:32.601490  1701 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet e2c52c22aef94ec6b8226f64dc4baca4 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 12:59:32.601990  1702 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet ad1245d9a7bf4a21ac3d2291898dd976 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-ad1245d9a7bf4a21ac3d2291898dd976
I20260709 12:59:32.605720  1701 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet e2c52c22aef94ec6b8226f64dc4baca4 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-e2c52c22aef94ec6b8226f64dc4baca4
I20260709 12:59:32.620491  1701 tablet_copy_source_session.cc:215] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:32.626339  1702 tablet_copy_source_session.cc:215] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:32.631762  2293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2c52c22aef94ec6b8226f64dc4baca4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:32.634184  2297 tablet_copy_client.cc:323] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:43459
I20260709 12:59:32.635098  2291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad1245d9a7bf4a21ac3d2291898dd976. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:32.648017  1553 tablet_copy_service.cc:140] P 83f6f5810950417195f707ff308a8125: Received BeginTabletCopySession request for tablet 214bc9e9284a4502844dbdc3395843fa from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:33722)
I20260709 12:59:32.648698  1553 tablet_copy_service.cc:161] P 83f6f5810950417195f707ff308a8125: Beginning new tablet copy session on tablet 214bc9e9284a4502844dbdc3395843fa from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:33722: session id = cecad4f86a114a7696e398b3e8cc511e-214bc9e9284a4502844dbdc3395843fa
I20260709 12:59:32.652287  2291 tablet_copy_client.cc:806] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:32.653115  2291 tablet_copy_client.cc:670] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:32.661533  2291 tablet_copy_client.cc:538] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:32.661757  1553 tablet_copy_source_session.cc:215] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:32.665076  2297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 214bc9e9284a4502844dbdc3395843fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:32.676445  2291 tablet_bootstrap.cc:492] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:32.676611  2293 tablet_copy_client.cc:806] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:32.681181  2293 tablet_copy_client.cc:670] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:32.685144  2297 tablet_copy_client.cc:806] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:32.686404  2293 tablet_copy_client.cc:538] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:32.686558  2297 tablet_copy_client.cc:670] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:32.696523  2297 tablet_copy_client.cc:538] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:32.708930  2297 tablet_bootstrap.cc:492] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:32.710059  2293 tablet_bootstrap.cc:492] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:32.723142  2291 tablet_bootstrap.cc:492] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:32.725109  2291 tablet_bootstrap.cc:492] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:32.725931  2291 ts_tablet_manager.cc:1403] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.050s	user 0.024s	sys 0.006s
I20260709 12:59:32.729157  2291 raft_consensus.cc:359] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:32.731165  2291 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:32.731918  2291 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:32.755820  2297 tablet_bootstrap.cc:492] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:32.765668  2293 tablet_bootstrap.cc:492] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:32.766865  2293 tablet_bootstrap.cc:492] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:32.767621  2293 ts_tablet_manager.cc:1403] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.058s	user 0.014s	sys 0.012s
I20260709 12:59:32.770828  2293 raft_consensus.cc:359] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:32.772666  2293 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:32.772346  2297 tablet_bootstrap.cc:492] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:32.774528  2297 ts_tablet_manager.cc:1403] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.066s	user 0.019s	sys 0.008s
I20260709 12:59:32.778703  2297 raft_consensus.cc:359] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:32.779712  2297 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:32.773401  2293 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:32.782786  2297 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:32.789695  2262 heartbeater.cc:499] Master 127.0.107.254:36375 was elected leader, sending a full tablet report...
I20260709 12:59:32.812816  2293 ts_tablet_manager.cc:1434] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.045s	user 0.002s	sys 0.013s
I20260709 12:59:32.865796  1702 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-e2c52c22aef94ec6b8226f64dc4baca4 received from {username='slave'} at 127.0.0.1:40014
I20260709 12:59:32.866528  2297 ts_tablet_manager.cc:1434] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.091s	user 0.005s	sys 0.029s
I20260709 12:59:32.866674  1702 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-e2c52c22aef94ec6b8226f64dc4baca4 on tablet e2c52c22aef94ec6b8226f64dc4baca4 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:32.868906  1553 tablet_copy_service.cc:342] P 83f6f5810950417195f707ff308a8125: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-214bc9e9284a4502844dbdc3395843fa received from {username='slave'} at 127.0.0.1:33722
I20260709 12:59:32.869576  1553 tablet_copy_service.cc:434] P 83f6f5810950417195f707ff308a8125: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-214bc9e9284a4502844dbdc3395843fa on tablet 214bc9e9284a4502844dbdc3395843fa with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:32.891609  2305 ts_tablet_manager.cc:933] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
I20260709 12:59:32.898093  2305 tablet_copy_client.cc:323] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:43459
I20260709 12:59:32.901861  1553 tablet_copy_service.cc:140] P 83f6f5810950417195f707ff308a8125: Received BeginTabletCopySession request for tablet b7e984cd665a4768933ff09e8ba0e07b from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:33722)
I20260709 12:59:32.902449  1553 tablet_copy_service.cc:161] P 83f6f5810950417195f707ff308a8125: Beginning new tablet copy session on tablet b7e984cd665a4768933ff09e8ba0e07b from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:33722: session id = cecad4f86a114a7696e398b3e8cc511e-b7e984cd665a4768933ff09e8ba0e07b
I20260709 12:59:32.907436  2291 ts_tablet_manager.cc:1434] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.181s	user 0.022s	sys 0.085s
I20260709 12:59:32.909595  1702 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-ad1245d9a7bf4a21ac3d2291898dd976 received from {username='slave'} at 127.0.0.1:40014
I20260709 12:59:32.910037  1702 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-ad1245d9a7bf4a21ac3d2291898dd976 on tablet ad1245d9a7bf4a21ac3d2291898dd976 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:32.917850  1553 tablet_copy_source_session.cc:215] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:32.926697  2305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7e984cd665a4768933ff09e8ba0e07b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:32.944190  2305 tablet_copy_client.cc:806] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:32.944826  2305 tablet_copy_client.cc:670] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:32.948354  2305 tablet_copy_client.cc:538] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:32.959425  2305 tablet_bootstrap.cc:492] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:32.983495  2237 raft_consensus.cc:1217] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:32.988272  2305 tablet_bootstrap.cc:492] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:32.989317  2305 tablet_bootstrap.cc:492] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:32.990156  2305 ts_tablet_manager.cc:1403] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.012s
I20260709 12:59:32.994697  2305 raft_consensus.cc:359] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:32.996104  2305 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:32.996821  2305 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:33.000048  2305 ts_tablet_manager.cc:1434] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:59:33.002106  1553 tablet_copy_service.cc:342] P 83f6f5810950417195f707ff308a8125: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-b7e984cd665a4768933ff09e8ba0e07b received from {username='slave'} at 127.0.0.1:33722
I20260709 12:59:33.002602  1553 tablet_copy_service.cc:434] P 83f6f5810950417195f707ff308a8125: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-b7e984cd665a4768933ff09e8ba0e07b on tablet b7e984cd665a4768933ff09e8ba0e07b with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:33.004707  2237 raft_consensus.cc:1217] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:33.110271  2237 raft_consensus.cc:1217] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:33.373549  2237 raft_consensus.cc:1217] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:33.530076  2309 raft_consensus.cc:1064] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:33.532812  2309 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:33.539786  1687 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.539842  2237 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.541687  2284 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.542415  1614 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.543386  2309 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.544411  2284 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.562996  2163 raft_consensus.cc:1064] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:33.568738  2163 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:33.572492  2183 raft_consensus.cc:2955] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.580966  1613 raft_consensus.cc:2955] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.582851  2237 raft_consensus.cc:2955] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.586916  1691 raft_consensus.cc:2955] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.603581  2160 raft_consensus.cc:1064] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:33.613540  2160 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:33.606736  1439 catalog_manager.cc:5697] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:33.630060  1614 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.632754  2186 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.660769  1538 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.662451  2160 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.664016  2237 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.667497  2160 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.670842  2237 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.673141  1543 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.705909  2186 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:33.708097  1617 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.709463  2317 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:33.723326  2317 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:59:33.757643  1538 raft_consensus.cc:2955] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.755378  2186 raft_consensus.cc:2955] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.768774  2235 raft_consensus.cc:2955] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.834957  2315 raft_consensus.cc:2955] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.844312  1617 raft_consensus.cc:2955] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.838135  1541 raft_consensus.cc:2955] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.846704  1613 raft_consensus.cc:2955] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.869837  1543 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:33.871898  1438 catalog_manager.cc:5697] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.882933  2236 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.881897  2235 raft_consensus.cc:2955] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.886123  2309 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.889303  2284 raft_consensus.cc:1064] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:33.889544  1614 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.891690  2284 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:33.893939  2309 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.905346  2183 raft_consensus.cc:2955] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.912488  1613 raft_consensus.cc:2955] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.920264  1692 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:33.929358  2237 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:33.932462  2163 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:33.935575  1687 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:33.940260  1440 catalog_manager.cc:5697] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.947746  2237 raft_consensus.cc:2955] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:33.943533  2315 raft_consensus.cc:2955] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 12:59:33.978515  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 93042.9us
I20260709 12:59:33.985854  2335 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:59:34.000197  1440 catalog_manager.cc:5697] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:34.007020  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e2c52c22aef94ec6b8226f64dc4baca4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:34.009778  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 214bc9e9284a4502844dbdc3395843fa with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:34.011601  1692 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:34.024147  1439 catalog_manager.cc:5697] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:34.028982  2235 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:34.031594  2163 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:34.046104  2315 raft_consensus.cc:2955] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 12:59:34.063783  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 159427us
I20260709 12:59:34.068172  1542 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:34.082226  2237 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:34.083673  1614 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:34.092264  2183 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:34.095571  1528 tablet_service.cc:1558] Processing DeleteTablet for tablet e2c52c22aef94ec6b8226f64dc4baca4 with delete_type TABLET_DATA_TOMBSTONED (TS 83f6f5810950417195f707ff308a8125 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:59:34.111860  1614 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:34.116801  2336 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:59:34.119239  2237 raft_consensus.cc:2955] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 12:59:34.153366  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 57724.4us
I20260709 12:59:34.156222  2160 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:59:34.159132  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 59540.5us
I20260709 12:59:34.161924  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ad1245d9a7bf4a21ac3d2291898dd976 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:59:34.181743  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 107558us
I20260709 12:59:34.181164  2183 raft_consensus.cc:2955] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 12:59:34.251858  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64998.2us
I20260709 12:59:34.229267  2340 tablet_replica.cc:333] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 12:59:34.262684  1438 catalog_manager.cc:5697] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:34.264618  2340 raft_consensus.cc:2243] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:34.255213  1613 raft_consensus.cc:2955] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:34.226960  1679 tablet_service.cc:1558] Processing DeleteTablet for tablet 214bc9e9284a4502844dbdc3395843fa with delete_type TABLET_DATA_TOMBSTONED (TS 984c9089d9804652846f3caeb609d56e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40104
I20260709 12:59:34.277137  2235 raft_consensus.cc:2955] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:34.295055  2237 raft_consensus.cc:2955] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:34.265483  2340 raft_consensus.cc:2272] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:34.308429  1617 raft_consensus.cc:2955] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:34.314178  2340 ts_tablet_manager.cc:1916] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:34.326191  1439 catalog_manager.cc:5697] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:34.314931  2347 tablet_replica.cc:333] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 12:59:34.332094  2347 raft_consensus.cc:2243] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:34.334149  2347 raft_consensus.cc:2272] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:59:34.346480  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 136673us
I20260709 12:59:34.345291  1687 raft_consensus.cc:2955] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:34.347620  2347 ts_tablet_manager.cc:1916] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:34.349716  2339 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:59:34.381762  1606 tablet_service.cc:1558] Processing DeleteTablet for tablet ad1245d9a7bf4a21ac3d2291898dd976 with delete_type TABLET_DATA_TOMBSTONED (TS 8a6a5ae1b7f240408b3ddd30d8a18262 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43382
I20260709 12:59:34.399875  2340 ts_tablet_manager.cc:1929] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:34.403241  2340 log.cc:1199] T e2c52c22aef94ec6b8226f64dc4baca4 P 83f6f5810950417195f707ff308a8125: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/wals/e2c52c22aef94ec6b8226f64dc4baca4
I20260709 12:59:34.401845  1538 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:34.405414  2347 ts_tablet_manager.cc:1929] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:34.405920  2347 log.cc:1199] T 214bc9e9284a4502844dbdc3395843fa P 984c9089d9804652846f3caeb609d56e: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/wals/214bc9e9284a4502844dbdc3395843fa
I20260709 12:59:34.404347  1541 raft_consensus.cc:2955] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:34.407764  1428 catalog_manager.cc:5028] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): tablet e2c52c22aef94ec6b8226f64dc4baca4 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:34.414214  2235 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:34.415972  2183 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:34.421954  2348 tablet_replica.cc:333] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 12:59:34.428442  2334 raft_consensus.cc:2955] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:34.432828  2237 raft_consensus.cc:2955] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:34.435163  1426 catalog_manager.cc:5028] TS 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): tablet 214bc9e9284a4502844dbdc3395843fa (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:34.437985  1692 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:34.442170  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b7e984cd665a4768933ff09e8ba0e07b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:34.443701  2334 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:34.444519  2348 raft_consensus.cc:2243] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:34.449747  1687 raft_consensus.cc:2955] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:34.462307  2348 raft_consensus.cc:2272] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:34.473984  2348 ts_tablet_manager.cc:1916] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:34.467613  1438 catalog_manager.cc:5697] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:34.496181  2348 ts_tablet_manager.cc:1929] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:34.496683  2348 log.cc:1199] T ad1245d9a7bf4a21ac3d2291898dd976 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/wals/ad1245d9a7bf4a21ac3d2291898dd976
I20260709 12:59:34.498435  1428 catalog_manager.cc:5028] TS 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): tablet ad1245d9a7bf4a21ac3d2291898dd976 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:34.502657  1606 tablet_service.cc:1558] Processing DeleteTablet for tablet b7e984cd665a4768933ff09e8ba0e07b with delete_type TABLET_DATA_TOMBSTONED (TS 8a6a5ae1b7f240408b3ddd30d8a18262 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43382
I20260709 12:59:34.507436  2348 tablet_replica.cc:333] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 12:59:34.508507  2348 raft_consensus.cc:2243] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:34.509275  2348 raft_consensus.cc:2272] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:34.512080  2348 ts_tablet_manager.cc:1916] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:34.527462  2348 ts_tablet_manager.cc:1929] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:34.527973  2348 log.cc:1199] T b7e984cd665a4768933ff09e8ba0e07b P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/wals/b7e984cd665a4768933ff09e8ba0e07b
I20260709 12:59:34.529561  1428 catalog_manager.cc:5028] TS 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): tablet b7e984cd665a4768933ff09e8ba0e07b (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:35.573148  1687 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
W20260709 12:59:35.580101  1657 consensus_peers.cc:597] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c7154d65b3794a0e87f1ae971e939fec. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:35.580310  1537 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.580628  1613 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.581653  2160 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.582518  2351 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.594890  2163 raft_consensus.cc:2955] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:35.596623  1537 raft_consensus.cc:2955] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:35.596881  1613 raft_consensus.cc:2955] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:35.619462  1439 catalog_manager.cc:5697] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:35.649194  1613 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:35.659070  1537 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.659189  1687 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.660487  2349 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.661332  2345 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:35.671206  2346 raft_consensus.cc:2955] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:35.672691  1687 raft_consensus.cc:2955] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:35.678928  1537 raft_consensus.cc:2955] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
W20260709 12:59:35.689843  1583 consensus_peers.cc:597] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e1ef0a9c8bca48b6898df70834e97c4c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:35.693092  1438 catalog_manager.cc:5697] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:35.700881  1542 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
W20260709 12:59:35.710907  1508 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0695e4119591411b8adab45b5925e8eb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:35.713222  1687 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.716166  1613 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:35.718225  2334 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:35.728286  2367 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:35.736447  2334 raft_consensus.cc:2955] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:35.741551  1617 raft_consensus.cc:2955] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:35.745707  1687 raft_consensus.cc:2955] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:35.801244  1437 catalog_manager.cc:5697] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:35.865129  1687 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
W20260709 12:59:35.886817  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56456.5us
W20260709 12:59:35.996865  1657 consensus_peers.cc:597] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4d7cdb06308e4f21a67e6a685abdc179. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:36.000581  1537 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:36.016613  2375 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:36.025996  1618 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:36.033102  2163 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:36.043737  2374 raft_consensus.cc:2955] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:36.048170  1537 raft_consensus.cc:2955] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:36.054718  1615 raft_consensus.cc:2955] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:36.075713  1438 catalog_manager.cc:5697] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:36.145695  2381 ts_tablet_manager.cc:933] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:36.148322  2381 tablet_copy_client.cc:323] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 12:59:36.150478  1702 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet c7154d65b3794a0e87f1ae971e939fec from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 12:59:36.151129  1702 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet c7154d65b3794a0e87f1ae971e939fec from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-c7154d65b3794a0e87f1ae971e939fec
I20260709 12:59:36.159695  1702 tablet_copy_source_session.cc:215] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:36.163301  2381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7154d65b3794a0e87f1ae971e939fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:36.180167  2381 tablet_copy_client.cc:806] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:36.181082  2381 tablet_copy_client.cc:670] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:36.186007  2381 tablet_copy_client.cc:538] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:36.196507  2381 tablet_bootstrap.cc:492] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:36.216588  2381 tablet_bootstrap.cc:492] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:36.217480  2381 tablet_bootstrap.cc:492] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:36.218171  2381 ts_tablet_manager.cc:1403] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.022s	user 0.021s	sys 0.000s
I20260709 12:59:36.221325  2381 raft_consensus.cc:359] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:36.222024  2381 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:36.222648  2381 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:36.225591  2381 ts_tablet_manager.cc:1434] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:59:36.228550  1702 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-c7154d65b3794a0e87f1ae971e939fec received from {username='slave'} at 127.0.0.1:40014
I20260709 12:59:36.229044  1702 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-c7154d65b3794a0e87f1ae971e939fec on tablet c7154d65b3794a0e87f1ae971e939fec with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:36.252404  2381 ts_tablet_manager.cc:933] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
I20260709 12:59:36.257846  2381 tablet_copy_client.cc:323] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:43459
I20260709 12:59:36.258689  2385 ts_tablet_manager.cc:933] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:36.260075  1553 tablet_copy_service.cc:140] P 83f6f5810950417195f707ff308a8125: Received BeginTabletCopySession request for tablet 0695e4119591411b8adab45b5925e8eb from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:33722)
I20260709 12:59:36.260680  1553 tablet_copy_service.cc:161] P 83f6f5810950417195f707ff308a8125: Beginning new tablet copy session on tablet 0695e4119591411b8adab45b5925e8eb from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:33722: session id = cecad4f86a114a7696e398b3e8cc511e-0695e4119591411b8adab45b5925e8eb
I20260709 12:59:36.265239  2385 tablet_copy_client.cc:323] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43887
I20260709 12:59:36.270947  1553 tablet_copy_source_session.cc:215] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:36.288736  2381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0695e4119591411b8adab45b5925e8eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:36.299268  1628 tablet_copy_service.cc:140] P 8a6a5ae1b7f240408b3ddd30d8a18262: Received BeginTabletCopySession request for tablet e1ef0a9c8bca48b6898df70834e97c4c from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:45356)
I20260709 12:59:36.299897  1628 tablet_copy_service.cc:161] P 8a6a5ae1b7f240408b3ddd30d8a18262: Beginning new tablet copy session on tablet e1ef0a9c8bca48b6898df70834e97c4c from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:45356: session id = cecad4f86a114a7696e398b3e8cc511e-e1ef0a9c8bca48b6898df70834e97c4c
I20260709 12:59:36.306411  2381 tablet_copy_client.cc:806] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:36.307154  2381 tablet_copy_client.cc:670] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:36.308732  1628 tablet_copy_source_session.cc:215] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:36.311573  2381 tablet_copy_client.cc:538] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:36.314422  2385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ef0a9c8bca48b6898df70834e97c4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:36.322261  2381 tablet_bootstrap.cc:492] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:36.336306  2385 tablet_copy_client.cc:806] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:36.338567  2385 tablet_copy_client.cc:670] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:36.344097  2385 tablet_copy_client.cc:538] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:36.347586  2381 tablet_bootstrap.cc:492] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:36.348416  2381 tablet_bootstrap.cc:492] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:36.349012  2381 ts_tablet_manager.cc:1403] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.000s
I20260709 12:59:36.351761  2381 raft_consensus.cc:359] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:36.352746  2381 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:36.353731  2385 tablet_bootstrap.cc:492] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:36.353484  2381 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:36.362774  2381 ts_tablet_manager.cc:1434] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.004s
I20260709 12:59:36.364717  1553 tablet_copy_service.cc:342] P 83f6f5810950417195f707ff308a8125: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-0695e4119591411b8adab45b5925e8eb received from {username='slave'} at 127.0.0.1:33722
I20260709 12:59:36.365175  1553 tablet_copy_service.cc:434] P 83f6f5810950417195f707ff308a8125: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-0695e4119591411b8adab45b5925e8eb on tablet 0695e4119591411b8adab45b5925e8eb with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:36.387833  2385 tablet_bootstrap.cc:492] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:36.389030  2385 tablet_bootstrap.cc:492] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:36.389775  2385 ts_tablet_manager.cc:1403] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.036s	user 0.027s	sys 0.007s
I20260709 12:59:36.392781  2385 raft_consensus.cc:359] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:36.393671  2385 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:36.394341  2385 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:36.396880  2385 ts_tablet_manager.cc:1434] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:59:36.399608  1628 tablet_copy_service.cc:342] P 8a6a5ae1b7f240408b3ddd30d8a18262: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-e1ef0a9c8bca48b6898df70834e97c4c received from {username='slave'} at 127.0.0.1:45356
I20260709 12:59:36.400139  1628 tablet_copy_service.cc:434] P 8a6a5ae1b7f240408b3ddd30d8a18262: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-e1ef0a9c8bca48b6898df70834e97c4c on tablet e1ef0a9c8bca48b6898df70834e97c4c with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:36.417797  2385 ts_tablet_manager.cc:933] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:36.419240  2385 tablet_copy_client.cc:323] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 12:59:36.421155  1702 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet 4d7cdb06308e4f21a67e6a685abdc179 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 12:59:36.421702  1702 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet 4d7cdb06308e4f21a67e6a685abdc179 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-4d7cdb06308e4f21a67e6a685abdc179
I20260709 12:59:36.427507  1702 tablet_copy_source_session.cc:215] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:36.430334  2385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d7cdb06308e4f21a67e6a685abdc179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:36.443312  2385 tablet_copy_client.cc:806] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:36.443862  2385 tablet_copy_client.cc:670] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:36.447741  2385 tablet_copy_client.cc:538] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:36.456420  2385 tablet_bootstrap.cc:492] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:36.474692  2385 tablet_bootstrap.cc:492] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:36.475450  2385 tablet_bootstrap.cc:492] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:36.476078  2385 ts_tablet_manager.cc:1403] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 12:59:36.478847  2385 raft_consensus.cc:359] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:36.479521  2385 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:36.480018  2385 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:36.482036  2385 ts_tablet_manager.cc:1434] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:59:36.483784  1702 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-4d7cdb06308e4f21a67e6a685abdc179 received from {username='slave'} at 127.0.0.1:40014
I20260709 12:59:36.484160  1702 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-4d7cdb06308e4f21a67e6a685abdc179 on tablet 4d7cdb06308e4f21a67e6a685abdc179 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:36.650976  2237 raft_consensus.cc:1217] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:36.730901  2237 raft_consensus.cc:1217] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:36.830335  2237 raft_consensus.cc:1217] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:36.999430  2237 raft_consensus.cc:1217] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:37.140106  2376 raft_consensus.cc:1064] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:37.142818  2376 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:37.151455  2237 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.153266  2374 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:37.153615  1618 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.153308  1538 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.155275  2374 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:37.156565  2163 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:37.165709  2163 raft_consensus.cc:2955] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.177244  1618 raft_consensus.cc:2955] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.178979  1541 raft_consensus.cc:2955] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.181569  2235 raft_consensus.cc:2955] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.188800  1439 catalog_manager.cc:5697] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:37.230777  1688 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:37.236400  2235 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.236490  1618 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.237897  2376 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:37.238848  2163 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:37.246614  2365 raft_consensus.cc:1064] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:37.249154  2365 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:37.251582  2163 raft_consensus.cc:2955] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.252643  1618 raft_consensus.cc:2955] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.258217  2237 raft_consensus.cc:2955] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.280581  1439 catalog_manager.cc:5697] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.293648  1528 tablet_service.cc:1558] Processing DeleteTablet for tablet c7154d65b3794a0e87f1ae971e939fec with delete_type TABLET_DATA_TOMBSTONED (TS 83f6f5810950417195f707ff308a8125 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:59:37.303931  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c7154d65b3794a0e87f1ae971e939fec with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:37.304670  1687 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.305128  2237 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.306146  1541 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.306632  2365 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:37.308810  2406 tablet_replica.cc:333] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 12:59:37.310060  2406 raft_consensus.cc:2243] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:37.310818  2406 raft_consensus.cc:2272] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:37.311354  2395 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:37.314307  2365 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:37.328900  2378 raft_consensus.cc:1064] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:37.331523  2378 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:37.347649  1687 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.349040  1618 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.351388  2334 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:37.350184  2237 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.353929  2394 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:37.355216  2334 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:37.356979  2395 raft_consensus.cc:2955] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.361267  2237 raft_consensus.cc:2955] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.368302  2406 ts_tablet_manager.cc:1916] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:37.382838  1688 raft_consensus.cc:2955] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.405869  1537 raft_consensus.cc:2955] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.417125  1439 catalog_manager.cc:5697] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.424352  2378 raft_consensus.cc:2955] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.461766  1615 raft_consensus.cc:2955] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.468041  2163 raft_consensus.cc:1064] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:37.470885  2163 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:37.481969  1618 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:37.483034  1438 catalog_manager.cc:5697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:37.489876  2237 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.494200  2395 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:37.501255  1541 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.504370  2371 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:37.505189  2406 ts_tablet_manager.cc:1929] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:37.505725  2406 log.cc:1199] T c7154d65b3794a0e87f1ae971e939fec P 83f6f5810950417195f707ff308a8125: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/wals/c7154d65b3794a0e87f1ae971e939fec
I20260709 12:59:37.511181  1428 catalog_manager.cc:5028] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): tablet c7154d65b3794a0e87f1ae971e939fec (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:37.511885  2237 raft_consensus.cc:2955] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.519147  1688 raft_consensus.cc:2955] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.530195  2365 raft_consensus.cc:2955] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.532006  2235 raft_consensus.cc:2955] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.543314  1541 raft_consensus.cc:2955] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.548832  2236 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.550784  2404 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:37.542165  1537 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:37.564064  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e1ef0a9c8bca48b6898df70834e97c4c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:37.572966  1615 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.573415  1541 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:37.576973  2374 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:37.578047  1439 catalog_manager.cc:5697] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.580186  2374 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:37.584412  2235 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.588353  2415 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:37.595047  1688 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.596889  2378 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:37.597800  2412 raft_consensus.cc:2955] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.599303  2374 raft_consensus.cc:2955] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.602003  1688 raft_consensus.cc:2955] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.602435  2235 raft_consensus.cc:2955] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.602780  1537 raft_consensus.cc:2955] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.635418  1679 tablet_service.cc:1558] Processing DeleteTablet for tablet e1ef0a9c8bca48b6898df70834e97c4c with delete_type TABLET_DATA_TOMBSTONED (TS 984c9089d9804652846f3caeb609d56e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40104
I20260709 12:59:37.650120  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0695e4119591411b8adab45b5925e8eb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:37.650987  2421 tablet_replica.cc:333] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 12:59:37.657941  1437 catalog_manager.cc:5697] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.661981  2235 raft_consensus.cc:2955] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.666456  2421 raft_consensus.cc:2243] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:37.667393  2421 raft_consensus.cc:2272] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:37.670653  1615 raft_consensus.cc:2955] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.686664  1606 tablet_service.cc:1558] Processing DeleteTablet for tablet 0695e4119591411b8adab45b5925e8eb with delete_type TABLET_DATA_TOMBSTONED (TS 8a6a5ae1b7f240408b3ddd30d8a18262 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43382
I20260709 12:59:37.691866  1439 catalog_manager.cc:5697] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:37.711081  2421 ts_tablet_manager.cc:1916] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:37.713596  2422 tablet_replica.cc:333] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 12:59:37.720139  1687 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:37.727883  1618 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.729261  2236 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:37.731120  2404 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:37.733379  2404 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:37.740417  2374 raft_consensus.cc:2955] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.742493  2422 raft_consensus.cc:2243] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:37.742194  1618 raft_consensus.cc:2955] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.744268  2422 raft_consensus.cc:2272] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:37.748118  2422 ts_tablet_manager.cc:1916] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:37.750823  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4d7cdb06308e4f21a67e6a685abdc179 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:37.760434  2421 ts_tablet_manager.cc:1929] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:37.759321  2235 raft_consensus.cc:2955] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:37.765398  1439 catalog_manager.cc:5697] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:37.766935  2421 log.cc:1199] T e1ef0a9c8bca48b6898df70834e97c4c P 984c9089d9804652846f3caeb609d56e: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/wals/e1ef0a9c8bca48b6898df70834e97c4c
I20260709 12:59:37.770793  1426 catalog_manager.cc:5028] TS 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): tablet e1ef0a9c8bca48b6898df70834e97c4c (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:37.778919  2422 ts_tablet_manager.cc:1929] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:37.779503  2422 log.cc:1199] T 0695e4119591411b8adab45b5925e8eb P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/wals/0695e4119591411b8adab45b5925e8eb
I20260709 12:59:37.781633  1428 catalog_manager.cc:5028] TS 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): tablet 0695e4119591411b8adab45b5925e8eb (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:37.787329  1528 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d7cdb06308e4f21a67e6a685abdc179 with delete_type TABLET_DATA_TOMBSTONED (TS 83f6f5810950417195f707ff308a8125 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:59:37.788539  2406 tablet_replica.cc:333] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 12:59:37.789508  2406 raft_consensus.cc:2243] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:37.790208  2406 raft_consensus.cc:2272] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:37.793542  2406 ts_tablet_manager.cc:1916] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:37.810427  2406 ts_tablet_manager.cc:1929] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:37.810730  2406 log.cc:1199] T 4d7cdb06308e4f21a67e6a685abdc179 P 83f6f5810950417195f707ff308a8125: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/wals/4d7cdb06308e4f21a67e6a685abdc179
I20260709 12:59:37.811980  1428 catalog_manager.cc:5028] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): tablet 4d7cdb06308e4f21a67e6a685abdc179 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:38.878158  1613 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:38.884258  1688 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:38.884714  1583 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: afa53a120a654b379c23e60544c86b5e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:38.884889  1541 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.885637  2346 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.886395  2417 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.894881  2346 raft_consensus.cc:2955] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:38.898924  1688 raft_consensus.cc:2955] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:38.902720  1537 raft_consensus.cc:2955] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:38.916188  1687 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:38.924589  1438 catalog_manager.cc:5697] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:38.929085  1657 consensus_peers.cc:597] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a1244fbe0514ba59a0164883a315bbe. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:38.930325  1613 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.932337  1537 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:38.932838  2376 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.934374  2374 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.945998  2376 raft_consensus.cc:2955] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:38.948139  1613 raft_consensus.cc:2955] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:38.958302  1537 raft_consensus.cc:2955] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:38.968734  1687 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:38.971671  1437 catalog_manager.cc:5697] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:38.990908  1615 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:59:38.993237  1657 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 59d952214bc24204a149e6f1a9af3184. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:38.993754  1541 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:38.994156  2437 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:38.998677  2376 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:39.010744  1615 raft_consensus.cc:2955] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:39.009241  2437 raft_consensus.cc:2955] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:39.028951  1541 raft_consensus.cc:2955] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:39.029419  1438 catalog_manager.cc:5697] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:39.030824  1615 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
W20260709 12:59:39.044987  1583 consensus_peers.cc:597] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 12adfb7855f340329f110249c207f409. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:39.045982  1687 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:39.046069  1541 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:39.047652  2371 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:39.048581  2425 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:39.059010  2371 raft_consensus.cc:2955] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:39.060634  1537 raft_consensus.cc:2955] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:39.080535  1438 catalog_manager.cc:5697] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:39.090400  1687 raft_consensus.cc:2955] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:39.438043  2446 ts_tablet_manager.cc:933] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:39.440003  2446 tablet_copy_client.cc:323] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 12:59:39.441720  1702 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet 0a1244fbe0514ba59a0164883a315bbe from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 12:59:39.442268  1702 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet 0a1244fbe0514ba59a0164883a315bbe from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-0a1244fbe0514ba59a0164883a315bbe
I20260709 12:59:39.448495  1702 tablet_copy_source_session.cc:215] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:39.451171  2446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a1244fbe0514ba59a0164883a315bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:39.464313  2446 tablet_copy_client.cc:806] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:39.465040  2446 tablet_copy_client.cc:670] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:39.469162  2446 tablet_copy_client.cc:538] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:39.478511  2446 tablet_bootstrap.cc:492] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:39.509102  2450 ts_tablet_manager.cc:933] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:39.514189  2448 ts_tablet_manager.cc:933] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:39.518010  2448 tablet_copy_client.cc:323] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43887
I20260709 12:59:39.520522  1628 tablet_copy_service.cc:140] P 8a6a5ae1b7f240408b3ddd30d8a18262: Received BeginTabletCopySession request for tablet afa53a120a654b379c23e60544c86b5e from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:45356)
I20260709 12:59:39.521253  1628 tablet_copy_service.cc:161] P 8a6a5ae1b7f240408b3ddd30d8a18262: Beginning new tablet copy session on tablet afa53a120a654b379c23e60544c86b5e from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:45356: session id = cecad4f86a114a7696e398b3e8cc511e-afa53a120a654b379c23e60544c86b5e
I20260709 12:59:39.525904  2446 tablet_bootstrap.cc:492] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:39.525869  2450 tablet_copy_client.cc:323] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 12:59:39.529316  2446 tablet_bootstrap.cc:492] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:39.530141  2446 ts_tablet_manager.cc:1403] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.052s	user 0.030s	sys 0.018s
I20260709 12:59:39.530069  1702 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet 59d952214bc24204a149e6f1a9af3184 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 12:59:39.531015  1702 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet 59d952214bc24204a149e6f1a9af3184 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-59d952214bc24204a149e6f1a9af3184
I20260709 12:59:39.534139  2446 raft_consensus.cc:359] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:39.535112  2446 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:39.535791  2446 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:39.544816  1628 tablet_copy_source_session.cc:215] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:39.548344  1702 tablet_copy_source_session.cc:215] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:39.560513  2448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afa53a120a654b379c23e60544c86b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:39.578966  2446 ts_tablet_manager.cc:1434] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.019s
I20260709 12:59:39.582803  1701 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-0a1244fbe0514ba59a0164883a315bbe received from {username='slave'} at 127.0.0.1:40014
I20260709 12:59:39.583269  1701 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-0a1244fbe0514ba59a0164883a315bbe on tablet 0a1244fbe0514ba59a0164883a315bbe with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:39.583307  2448 tablet_copy_client.cc:806] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:39.584095  2450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59d952214bc24204a149e6f1a9af3184. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:39.584074  2448 tablet_copy_client.cc:670] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:39.609575  2448 tablet_copy_client.cc:538] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:39.618556  2450 tablet_copy_client.cc:806] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:39.619513  2450 tablet_copy_client.cc:670] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:39.630272  2448 tablet_bootstrap.cc:492] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:39.637984  2446 ts_tablet_manager.cc:933] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:39.639165  2450 tablet_copy_client.cc:538] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:39.650388  2446 tablet_copy_client.cc:323] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43887
I20260709 12:59:39.667464  2450 tablet_bootstrap.cc:492] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:39.668197  2448 tablet_bootstrap.cc:492] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:39.669309  2448 tablet_bootstrap.cc:492] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:39.670163  2448 ts_tablet_manager.cc:1403] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.040s	user 0.008s	sys 0.023s
I20260709 12:59:39.673781  1628 tablet_copy_service.cc:140] P 8a6a5ae1b7f240408b3ddd30d8a18262: Received BeginTabletCopySession request for tablet 12adfb7855f340329f110249c207f409 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:45356)
I20260709 12:59:39.674399  1628 tablet_copy_service.cc:161] P 8a6a5ae1b7f240408b3ddd30d8a18262: Beginning new tablet copy session on tablet 12adfb7855f340329f110249c207f409 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:45356: session id = cecad4f86a114a7696e398b3e8cc511e-12adfb7855f340329f110249c207f409
I20260709 12:59:39.676087  2448 raft_consensus.cc:359] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:39.678295  2448 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:39.679057  2448 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:39.681757  2448 ts_tablet_manager.cc:1434] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 12:59:39.700902  1628 tablet_copy_source_session.cc:215] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:39.702795  1627 tablet_copy_service.cc:342] P 8a6a5ae1b7f240408b3ddd30d8a18262: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-afa53a120a654b379c23e60544c86b5e received from {username='slave'} at 127.0.0.1:45356
I20260709 12:59:39.705241  1627 tablet_copy_service.cc:434] P 8a6a5ae1b7f240408b3ddd30d8a18262: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-afa53a120a654b379c23e60544c86b5e on tablet afa53a120a654b379c23e60544c86b5e with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:39.708343  2446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12adfb7855f340329f110249c207f409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:39.712212  2450 tablet_bootstrap.cc:492] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:39.713282  2450 tablet_bootstrap.cc:492] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:39.714648  2450 ts_tablet_manager.cc:1403] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.047s	user 0.024s	sys 0.000s
I20260709 12:59:39.718168  2450 raft_consensus.cc:359] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:39.719039  2450 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:39.719753  2450 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:39.725030  2450 ts_tablet_manager.cc:1434] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:59:39.727083  1701 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-59d952214bc24204a149e6f1a9af3184 received from {username='slave'} at 127.0.0.1:40014
I20260709 12:59:39.727540  1701 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-59d952214bc24204a149e6f1a9af3184 on tablet 59d952214bc24204a149e6f1a9af3184 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:39.730474  2446 tablet_copy_client.cc:806] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:39.731158  2446 tablet_copy_client.cc:670] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:39.739939  2446 tablet_copy_client.cc:538] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:39.747112  2446 tablet_bootstrap.cc:492] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:39.765359  2446 tablet_bootstrap.cc:492] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:39.766443  2446 tablet_bootstrap.cc:492] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:39.767202  2446 ts_tablet_manager.cc:1403] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.005s
I20260709 12:59:39.769944  2446 raft_consensus.cc:359] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:39.770574  2446 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:39.771101  2446 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:39.772992  2446 ts_tablet_manager.cc:1434] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:59:39.774856  1627 tablet_copy_service.cc:342] P 8a6a5ae1b7f240408b3ddd30d8a18262: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-12adfb7855f340329f110249c207f409 received from {username='slave'} at 127.0.0.1:45356
I20260709 12:59:39.775316  1627 tablet_copy_service.cc:434] P 8a6a5ae1b7f240408b3ddd30d8a18262: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-12adfb7855f340329f110249c207f409 on tablet 12adfb7855f340329f110249c207f409 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:39.886612  2235 raft_consensus.cc:1217] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:39.923224  2235 raft_consensus.cc:1217] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:39.992805  2235 raft_consensus.cc:1217] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:59:40.211390  2235 raft_consensus.cc:1217] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:40.387614  2376 raft_consensus.cc:1064] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:40.389704  2376 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:40.407608  2235 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.409077  2440 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.413491  1615 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.413887  1542 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.421495  2376 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.422729  2454 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.434021  2454 raft_consensus.cc:2955] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.436254  1615 raft_consensus.cc:2955] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.437163  1542 raft_consensus.cc:2955] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.486114  2371 raft_consensus.cc:1064] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:40.504199  2235 raft_consensus.cc:2955] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.504338  2371 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:40.520147  1437 catalog_manager.cc:5697] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.600113  2376 raft_consensus.cc:1064] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:40.602861  2235 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.604542  2468 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.604828  1541 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.607398  2346 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.608600  2376 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:40.621167  1688 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.624928  1537 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:40.627439  2469 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.627617  2235 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:40.629665  2469 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.638536  1618 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:40.641001  2470 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.641176  2468 raft_consensus.cc:2955] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.643903  1541 raft_consensus.cc:2955] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.651889  2472 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:59:40.660761  2236 raft_consensus.cc:2955] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.680518  2469 raft_consensus.cc:2955] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.713724  1691 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:40.718715  1541 raft_consensus.cc:2955] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.724030  2235 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.731002  2235 raft_consensus.cc:2955] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.738745  2454 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:40.751869  2371 raft_consensus.cc:1064] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:40.756983  2471 raft_consensus.cc:2955] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.763068  2233 raft_consensus.cc:2955] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.770859  1440 catalog_manager.cc:5697] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:40.736353  1439 catalog_manager.cc:5697] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.773967  2371 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:40.781195  1613 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.782732  1618 raft_consensus.cc:2955] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.786671  1688 raft_consensus.cc:2955] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.794703  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0a1244fbe0514ba59a0164883a315bbe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:40.805387  1615 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:40.808068  2469 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:59:40.808161  1687 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:40.817873  1613 raft_consensus.cc:2955] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.826710  1541 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:40.830453  2236 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:40.831787  1541 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.831897  2469 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:59:40.837242  2236 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.843820  2371 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.838708  2233 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:40.840229  1688 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.849568  2235 raft_consensus.cc:2955] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.854889  1538 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:40.889746  2454 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:40.848263  2471 raft_consensus.cc:2955] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.881194  2465 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:40.896564  2468 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:40.897974  1538 raft_consensus.cc:2955] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.899605  2468 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.907367  2472 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:40.914093  1439 catalog_manager.cc:5697] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:40.919365  2346 raft_consensus.cc:2955] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.930603  2235 raft_consensus.cc:2955] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.948046  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet afa53a120a654b379c23e60544c86b5e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:40.950572  1538 raft_consensus.cc:2955] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.957085  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 59d952214bc24204a149e6f1a9af3184 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:40.958448  1687 raft_consensus.cc:2955] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.963656  1542 raft_consensus.cc:2955] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.956604  2371 raft_consensus.cc:2955] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.986393  2235 raft_consensus.cc:2955] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.987725  1438 catalog_manager.cc:5697] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New cstate: current_term: 2 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:40.996177  1528 tablet_service.cc:1558] Processing DeleteTablet for tablet 0a1244fbe0514ba59a0164883a315bbe with delete_type TABLET_DATA_TOMBSTONED (TS 83f6f5810950417195f707ff308a8125 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:59:41.001194  2489 tablet_replica.cc:333] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 12:59:41.003466  2489 raft_consensus.cc:2243] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.004326  2489 raft_consensus.cc:2272] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.009742  1606 tablet_service.cc:1558] Processing DeleteTablet for tablet 59d952214bc24204a149e6f1a9af3184 with delete_type TABLET_DATA_TOMBSTONED (TS 8a6a5ae1b7f240408b3ddd30d8a18262 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43382
I20260709 12:59:41.012751  2489 ts_tablet_manager.cc:1916] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:41.023595  2490 tablet_replica.cc:333] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 12:59:41.025094  2490 raft_consensus.cc:2243] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.025934  2490 raft_consensus.cc:2272] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.029807  2490 ts_tablet_manager.cc:1916] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:41.030782  1437 catalog_manager.cc:5697] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:41.037873  2489 ts_tablet_manager.cc:1929] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:41.038429  2489 log.cc:1199] T 0a1244fbe0514ba59a0164883a315bbe P 83f6f5810950417195f707ff308a8125: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/wals/0a1244fbe0514ba59a0164883a315bbe
I20260709 12:59:41.040202  1428 catalog_manager.cc:5028] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): tablet 0a1244fbe0514ba59a0164883a315bbe (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:41.048005  2490 ts_tablet_manager.cc:1929] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:59:41.047374  1438 catalog_manager.cc:5697] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:41.048651  2490 log.cc:1199] T 59d952214bc24204a149e6f1a9af3184 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/wals/59d952214bc24204a149e6f1a9af3184
I20260709 12:59:41.050522  1428 catalog_manager.cc:5028] TS 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): tablet 59d952214bc24204a149e6f1a9af3184 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:41.069299  1613 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:41.074460  1679 tablet_service.cc:1558] Processing DeleteTablet for tablet afa53a120a654b379c23e60544c86b5e with delete_type TABLET_DATA_TOMBSTONED (TS 984c9089d9804652846f3caeb609d56e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40104
I20260709 12:59:41.075732  1542 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:41.076233  2235 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:41.077275  2472 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:41.080190  2465 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:41.079870  2491 tablet_replica.cc:333] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 12:59:41.083014  2491 raft_consensus.cc:2243] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.083808  2491 raft_consensus.cc:2272] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.089010  2465 raft_consensus.cc:2955] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:41.090139  2491 ts_tablet_manager.cc:1916] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:41.091794  2235 raft_consensus.cc:2955] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:41.096851  1542 raft_consensus.cc:2955] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:41.108717  1439 catalog_manager.cc:5697] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:41.115401  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 12adfb7855f340329f110249c207f409 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:41.128355  2491 ts_tablet_manager.cc:1929] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:41.129302  1679 tablet_service.cc:1558] Processing DeleteTablet for tablet 12adfb7855f340329f110249c207f409 with delete_type TABLET_DATA_TOMBSTONED (TS 984c9089d9804652846f3caeb609d56e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40104
I20260709 12:59:41.129351  2491 log.cc:1199] T afa53a120a654b379c23e60544c86b5e P 984c9089d9804652846f3caeb609d56e: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/wals/afa53a120a654b379c23e60544c86b5e
I20260709 12:59:41.133123  2491 tablet_replica.cc:333] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 12:59:41.134341  2491 raft_consensus.cc:2243] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:41.136540  2491 raft_consensus.cc:2272] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:41.134097  1426 catalog_manager.cc:5028] TS 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): tablet afa53a120a654b379c23e60544c86b5e (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:41.139822  2491 ts_tablet_manager.cc:1916] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:41.152643  2491 ts_tablet_manager.cc:1929] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:41.153072  2491 log.cc:1199] T 12adfb7855f340329f110249c207f409 P 984c9089d9804652846f3caeb609d56e: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/wals/12adfb7855f340329f110249c207f409
I20260709 12:59:41.154579  1426 catalog_manager.cc:5028] TS 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): tablet 12adfb7855f340329f110249c207f409 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:42.188144  1687 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:42.194365  1618 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:42.194522  1657 consensus_peers.cc:597] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ed741b21bc0447a0896904a04e535f8a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:42.196192  1538 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:42.197598  2471 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:42.199086  2466 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:42.208897  2454 raft_consensus.cc:2955] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.211946  1618 raft_consensus.cc:2955] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.213037  1538 raft_consensus.cc:2955] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.225842  1687 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:42.229017  1440 catalog_manager.cc:5697] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:42.246308  1618 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:42.246899  1657 consensus_peers.cc:597] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fb767ced66f24d579d3a9c116606ef2f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:42.248155  2495 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:42.252230  1542 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:42.253933  2495 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:42.263046  2454 raft_consensus.cc:2955] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.265331  1613 raft_consensus.cc:2955] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.267165  1538 raft_consensus.cc:2955] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.284394  1618 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:42.287413  1438 catalog_manager.cc:5697] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.304678  1687 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:42.305935  1583 consensus_peers.cc:597] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 969ec621630549bb9b288fdb50e46fbe. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:42.306351  2496 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:42.306107  1538 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:42.308430  2496 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:42.318202  2346 raft_consensus.cc:2955] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.323849  1542 raft_consensus.cc:2955] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.330511  1687 raft_consensus.cc:2955] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.334173  1691 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:42.340682  1438 catalog_manager.cc:5697] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:42.352161  1618 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:42.354185  1657 consensus_peers.cc:597] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 27181dd872d647748dc1fe8c2ac6c12b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:42.354485  1542 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:42.355196  2466 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:42.357079  2466 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:42.376211  2495 raft_consensus.cc:2955] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.379030  1542 raft_consensus.cc:2955] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.384677  1618 raft_consensus.cc:2955] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.402639  1437 catalog_manager.cc:5697] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:42.589972  2513 ts_tablet_manager.cc:933] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:42.591734  2513 tablet_copy_client.cc:323] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 12:59:42.593286  1701 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet ed741b21bc0447a0896904a04e535f8a from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 12:59:42.593809  1701 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet ed741b21bc0447a0896904a04e535f8a from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-ed741b21bc0447a0896904a04e535f8a
I20260709 12:59:42.600109  1701 tablet_copy_source_session.cc:215] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:42.603171  2513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed741b21bc0447a0896904a04e535f8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:42.616200  2513 tablet_copy_client.cc:806] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:42.616796  2513 tablet_copy_client.cc:670] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:42.622040  2513 tablet_copy_client.cc:538] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:42.631213  2513 tablet_bootstrap.cc:492] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:42.652954  2513 tablet_bootstrap.cc:492] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.653884  2513 tablet_bootstrap.cc:492] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:42.654553  2513 ts_tablet_manager.cc:1403] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.005s
I20260709 12:59:42.657495  2513 raft_consensus.cc:359] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:42.658347  2513 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:42.659041  2513 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:42.662348  2513 ts_tablet_manager.cc:1434] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:59:42.664112  1701 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-ed741b21bc0447a0896904a04e535f8a received from {username='slave'} at 127.0.0.1:40014
I20260709 12:59:42.664534  1701 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-ed741b21bc0447a0896904a04e535f8a on tablet ed741b21bc0447a0896904a04e535f8a with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:42.687855  2513 ts_tablet_manager.cc:933] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:42.689615  2513 tablet_copy_client.cc:323] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43887
I20260709 12:59:42.691320  1627 tablet_copy_service.cc:140] P 8a6a5ae1b7f240408b3ddd30d8a18262: Received BeginTabletCopySession request for tablet 969ec621630549bb9b288fdb50e46fbe from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:45356)
I20260709 12:59:42.691860  1627 tablet_copy_service.cc:161] P 8a6a5ae1b7f240408b3ddd30d8a18262: Beginning new tablet copy session on tablet 969ec621630549bb9b288fdb50e46fbe from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:45356: session id = cecad4f86a114a7696e398b3e8cc511e-969ec621630549bb9b288fdb50e46fbe
I20260709 12:59:42.704746  1627 tablet_copy_source_session.cc:215] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:42.712086  2513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 969ec621630549bb9b288fdb50e46fbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:42.728572  2513 tablet_copy_client.cc:806] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:42.729254  2513 tablet_copy_client.cc:670] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:42.732966  2513 tablet_copy_client.cc:538] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:42.739485  2513 tablet_bootstrap.cc:492] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:42.758404  2513 tablet_bootstrap.cc:492] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.759338  2513 tablet_bootstrap.cc:492] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:42.760010  2513 ts_tablet_manager.cc:1403] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.012s
I20260709 12:59:42.762928  2513 raft_consensus.cc:359] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:42.763774  2513 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:42.764447  2513 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:42.766767  2513 ts_tablet_manager.cc:1434] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:59:42.768770  1627 tablet_copy_service.cc:342] P 8a6a5ae1b7f240408b3ddd30d8a18262: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-969ec621630549bb9b288fdb50e46fbe received from {username='slave'} at 127.0.0.1:45356
I20260709 12:59:42.769212  1627 tablet_copy_service.cc:434] P 8a6a5ae1b7f240408b3ddd30d8a18262: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-969ec621630549bb9b288fdb50e46fbe on tablet 969ec621630549bb9b288fdb50e46fbe with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:42.774135  2513 ts_tablet_manager.cc:933] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:42.791512  2513 tablet_copy_client.cc:323] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 12:59:42.793447  1701 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet fb767ced66f24d579d3a9c116606ef2f from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 12:59:42.794018  1701 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet fb767ced66f24d579d3a9c116606ef2f from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-fb767ced66f24d579d3a9c116606ef2f
I20260709 12:59:42.801730  1701 tablet_copy_source_session.cc:215] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:42.805172  2513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb767ced66f24d579d3a9c116606ef2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:42.827333  2513 tablet_copy_client.cc:806] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:42.827943  2513 tablet_copy_client.cc:670] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:42.837312  2513 tablet_copy_client.cc:538] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:42.849126  2513 tablet_bootstrap.cc:492] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:42.851233  2519 ts_tablet_manager.cc:933] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:42.863024  2519 tablet_copy_client.cc:323] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 12:59:42.864790  1701 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet 27181dd872d647748dc1fe8c2ac6c12b from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 12:59:42.865283  1701 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet 27181dd872d647748dc1fe8c2ac6c12b from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-27181dd872d647748dc1fe8c2ac6c12b
I20260709 12:59:42.868848  2513 tablet_bootstrap.cc:492] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.869819  2513 tablet_bootstrap.cc:492] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:42.870473  2513 ts_tablet_manager.cc:1403] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.022s	user 0.022s	sys 0.000s
I20260709 12:59:42.873142  1701 tablet_copy_source_session.cc:215] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:42.873158  2513 raft_consensus.cc:359] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:42.874222  2513 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:42.874840  2513 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:42.876936  2513 ts_tablet_manager.cc:1434] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:59:42.878665  1701 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-fb767ced66f24d579d3a9c116606ef2f received from {username='slave'} at 127.0.0.1:40014
I20260709 12:59:42.879082  1701 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-fb767ced66f24d579d3a9c116606ef2f on tablet fb767ced66f24d579d3a9c116606ef2f with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:42.882840  2519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27181dd872d647748dc1fe8c2ac6c12b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:42.900058  2519 tablet_copy_client.cc:806] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:42.900637  2519 tablet_copy_client.cc:670] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:42.904100  2519 tablet_copy_client.cc:538] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:42.913100  2519 tablet_bootstrap.cc:492] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:42.937991  2519 tablet_bootstrap.cc:492] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:42.938952  2519 tablet_bootstrap.cc:492] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:42.939648  2519 ts_tablet_manager.cc:1403] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.006s
I20260709 12:59:42.942231  2519 raft_consensus.cc:359] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:42.943068  2519 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:42.943637  2519 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:42.946065  2519 ts_tablet_manager.cc:1434] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.004s
I20260709 12:59:42.947717  1701 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-27181dd872d647748dc1fe8c2ac6c12b received from {username='slave'} at 127.0.0.1:40014
I20260709 12:59:42.948143  1701 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-27181dd872d647748dc1fe8c2ac6c12b on tablet 27181dd872d647748dc1fe8c2ac6c12b with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:43.149570  2235 raft_consensus.cc:1217] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:43.171587  2235 raft_consensus.cc:1217] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:43.384567  2235 raft_consensus.cc:1217] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:43.470721  2235 raft_consensus.cc:1217] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:43.699535  2346 raft_consensus.cc:1064] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:43.701907  2346 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:43.708467  1691 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.710583  1542 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.711040  2507 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.716084  2235 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.717816  2512 raft_consensus.cc:1064] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:43.718457  2346 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.720499  2512 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:43.727705  2525 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:43.741706  2346 raft_consensus.cc:2955] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.751525  1687 raft_consensus.cc:2955] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.770790  2235 raft_consensus.cc:2955] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.785343  2233 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.787319  1437 catalog_manager.cc:5697] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:43.789305  2471 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:43.799703  1538 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.800995  2512 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:43.805167  1613 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:43.810968  1541 raft_consensus.cc:2955] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.810926  2495 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 12:59:43.839296  2471 raft_consensus.cc:2955] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.843106  2235 raft_consensus.cc:2955] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.854167  1613 raft_consensus.cc:2955] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.863212  1542 raft_consensus.cc:2955] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.873194  1440 catalog_manager.cc:5697] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.898772  1687 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:43.903014  1615 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:43.909400  1541 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.911727  1541 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.911727  2233 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.913378  2507 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:43.914269  2235 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:43.915277  2507 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:43.912402  2495 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:59:43.920792  2471 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:43.925141  2527 raft_consensus.cc:2955] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.931067  1537 raft_consensus.cc:2955] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.934872  2532 raft_consensus.cc:2955] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.943559  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 969ec621630549bb9b288fdb50e46fbe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:43.945251  2233 raft_consensus.cc:2955] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.945976  1542 raft_consensus.cc:2955] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.954630  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ed741b21bc0447a0896904a04e535f8a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:43.963662  1439 catalog_manager.cc:5697] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:43.969161  1438 catalog_manager.cc:5697] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:43.977633  1606 tablet_service.cc:1558] Processing DeleteTablet for tablet ed741b21bc0447a0896904a04e535f8a with delete_type TABLET_DATA_TOMBSTONED (TS 8a6a5ae1b7f240408b3ddd30d8a18262 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43382
I20260709 12:59:43.980996  2540 tablet_replica.cc:333] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 12:59:43.982474  2540 raft_consensus.cc:2243] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:43.983352  2540 raft_consensus.cc:2272] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:43.989185  2540 ts_tablet_manager.cc:1916] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:43.991508  2235 raft_consensus.cc:2955] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:43.994825  1679 tablet_service.cc:1558] Processing DeleteTablet for tablet 969ec621630549bb9b288fdb50e46fbe with delete_type TABLET_DATA_TOMBSTONED (TS 984c9089d9804652846f3caeb609d56e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40104
I20260709 12:59:44.007459  2471 raft_consensus.cc:1064] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:44.010146  2471 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:44.011852  2541 tablet_replica.cc:333] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 12:59:44.014215  2541 raft_consensus.cc:2243] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:44.014966  2541 raft_consensus.cc:2272] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:44.028664  2541 ts_tablet_manager.cc:1916] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:44.054687  2495 raft_consensus.cc:1064] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:44.056767  2540 ts_tablet_manager.cc:1929] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:44.057253  2540 log.cc:1199] T ed741b21bc0447a0896904a04e535f8a P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/wals/ed741b21bc0447a0896904a04e535f8a
I20260709 12:59:44.057294  2495 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:44.060225  2235 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:44.061899  2532 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:44.071967  1542 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:44.076236  2512 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:44.097743  2235 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:44.098240  1428 catalog_manager.cc:5028] TS 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): tablet ed741b21bc0447a0896904a04e535f8a (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:44.100556  2495 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:44.100929  1537 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:44.102027  1615 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:44.104002  2532 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:44.104628  2512 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:44.111857  1618 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:44.121080  2532 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:44.134626  2541 ts_tablet_manager.cc:1929] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:44.135123  2541 log.cc:1199] T 969ec621630549bb9b288fdb50e46fbe P 984c9089d9804652846f3caeb609d56e: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/wals/969ec621630549bb9b288fdb50e46fbe
I20260709 12:59:44.136902  1426 catalog_manager.cc:5028] TS 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): tablet 969ec621630549bb9b288fdb50e46fbe (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:44.140276  1542 raft_consensus.cc:2955] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.139236  2532 raft_consensus.cc:2955] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.152514  1613 raft_consensus.cc:2955] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.168090  1437 catalog_manager.cc:5697] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.177644  2233 raft_consensus.cc:2955] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.188113  2512 raft_consensus.cc:2955] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.209631  1618 raft_consensus.cc:2955] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.218580  2236 raft_consensus.cc:2955] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.235463  1437 catalog_manager.cc:5697] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.239967  1542 raft_consensus.cc:2955] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.246546  1688 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:44.253724  2236 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:44.254424  1618 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:44.259231  2532 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:44.265885  2512 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:44.278663  1688 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:44.279538  2512 raft_consensus.cc:2955] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.284090  2236 raft_consensus.cc:2955] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.285283  2233 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:44.287698  2543 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:44.295086  2495 raft_consensus.cc:2955] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.302078  1613 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:44.303453  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fb767ced66f24d579d3a9c116606ef2f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:44.309636  1618 raft_consensus.cc:2955] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.312250  2532 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:44.316828  1437 catalog_manager.cc:5697] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.318747  2236 raft_consensus.cc:2955] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.326594  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 27181dd872d647748dc1fe8c2ac6c12b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:44.324584  1613 raft_consensus.cc:2955] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.338739  1528 tablet_service.cc:1558] Processing DeleteTablet for tablet fb767ced66f24d579d3a9c116606ef2f with delete_type TABLET_DATA_TOMBSTONED (TS 83f6f5810950417195f707ff308a8125 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:59:44.354959  2551 tablet_replica.cc:333] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 12:59:44.356072  2551 raft_consensus.cc:2243] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:44.356873  2551 raft_consensus.cc:2272] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:44.355394  1439 catalog_manager.cc:5697] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:44.360879  2551 ts_tablet_manager.cc:1916] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:44.371444  1528 tablet_service.cc:1558] Processing DeleteTablet for tablet 27181dd872d647748dc1fe8c2ac6c12b with delete_type TABLET_DATA_TOMBSTONED (TS 83f6f5810950417195f707ff308a8125 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:59:44.379720  2551 ts_tablet_manager.cc:1929] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:44.380187  2551 log.cc:1199] T fb767ced66f24d579d3a9c116606ef2f P 83f6f5810950417195f707ff308a8125: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/wals/fb767ced66f24d579d3a9c116606ef2f
I20260709 12:59:44.381772  2551 tablet_replica.cc:333] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 12:59:44.381913  1428 catalog_manager.cc:5028] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): tablet fb767ced66f24d579d3a9c116606ef2f (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:44.397063  2551 raft_consensus.cc:2243] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:44.397779  2551 raft_consensus.cc:2272] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:44.400003  2551 ts_tablet_manager.cc:1916] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:44.412063  2551 ts_tablet_manager.cc:1929] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:44.412417  2551 log.cc:1199] T 27181dd872d647748dc1fe8c2ac6c12b P 83f6f5810950417195f707ff308a8125: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/wals/27181dd872d647748dc1fe8c2ac6c12b
I20260709 12:59:44.413887  1428 catalog_manager.cc:5028] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): tablet 27181dd872d647748dc1fe8c2ac6c12b (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:45.444146  1537 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
W20260709 12:59:45.452840  1508 consensus_peers.cc:597] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34b2ba71a23e4ba3becae445ee7598c5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:45.453720  1688 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.454193  1617 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.455363  2554 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.456342  2556 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.464645  2556 raft_consensus.cc:2955] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:45.466363  1617 raft_consensus.cc:2955] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:45.467815  1688 raft_consensus.cc:2955] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:45.483541  1618 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:45.488451  1437 catalog_manager.cc:5697] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:45.496901  1583 consensus_peers.cc:597] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1a69ce55a73e464a962e15e73fd1db2c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:45.497181  1687 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.499233  2553 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:45.501152  1537 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.503153  2553 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.520638  2527 raft_consensus.cc:2955] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:45.525017  1687 raft_consensus.cc:2955] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:45.537326  1537 raft_consensus.cc:2955] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:45.540237  1618 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:45.560858  1437 catalog_manager.cc:5697] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
W20260709 12:59:45.563681  1583 consensus_peers.cc:597] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8343e6d9eb5348b99ed8b5dc38a6087d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:45.564895  1687 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.566857  2525 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.572747  1542 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.574755  2525 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:45.579072  2553 raft_consensus.cc:2955] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:45.580755  1687 raft_consensus.cc:2955] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:45.586664  1542 raft_consensus.cc:2955] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:45.597877  1687 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:45.597863  1437 catalog_manager.cc:5697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:45.605275  1657 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81301f84b53f41349ac6bca78f1a77f4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:45.606355  1537 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.607754  1618 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:45.608014  2495 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.609776  2543 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:45.617511  2495 raft_consensus.cc:2955] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:45.624835  1542 raft_consensus.cc:2955] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:45.629727  1618 raft_consensus.cc:2955] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:45.643131  1437 catalog_manager.cc:5697] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:45.948954  2579 ts_tablet_manager.cc:933] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
I20260709 12:59:45.985695  2579 tablet_copy_client.cc:323] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:43459
I20260709 12:59:45.989439  1553 tablet_copy_service.cc:140] P 83f6f5810950417195f707ff308a8125: Received BeginTabletCopySession request for tablet 34b2ba71a23e4ba3becae445ee7598c5 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:33722)
I20260709 12:59:45.990298  1553 tablet_copy_service.cc:161] P 83f6f5810950417195f707ff308a8125: Beginning new tablet copy session on tablet 34b2ba71a23e4ba3becae445ee7598c5 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:33722: session id = cecad4f86a114a7696e398b3e8cc511e-34b2ba71a23e4ba3becae445ee7598c5
I20260709 12:59:46.021047  2581 ts_tablet_manager.cc:933] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:46.031536  1553 tablet_copy_source_session.cc:215] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:46.040689  2582 ts_tablet_manager.cc:933] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:46.052119  2579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34b2ba71a23e4ba3becae445ee7598c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:46.075153  2581 tablet_copy_client.cc:323] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43887
I20260709 12:59:46.082111  2582 tablet_copy_client.cc:323] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43887
I20260709 12:59:46.104472  1627 tablet_copy_service.cc:140] P 8a6a5ae1b7f240408b3ddd30d8a18262: Received BeginTabletCopySession request for tablet 1a69ce55a73e464a962e15e73fd1db2c from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:45356)
I20260709 12:59:46.105244  1627 tablet_copy_service.cc:161] P 8a6a5ae1b7f240408b3ddd30d8a18262: Beginning new tablet copy session on tablet 1a69ce55a73e464a962e15e73fd1db2c from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:45356: session id = cecad4f86a114a7696e398b3e8cc511e-1a69ce55a73e464a962e15e73fd1db2c
I20260709 12:59:46.111999  1628 tablet_copy_service.cc:140] P 8a6a5ae1b7f240408b3ddd30d8a18262: Received BeginTabletCopySession request for tablet 8343e6d9eb5348b99ed8b5dc38a6087d from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:45356)
I20260709 12:59:46.112672  1628 tablet_copy_service.cc:161] P 8a6a5ae1b7f240408b3ddd30d8a18262: Beginning new tablet copy session on tablet 8343e6d9eb5348b99ed8b5dc38a6087d from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:45356: session id = cecad4f86a114a7696e398b3e8cc511e-8343e6d9eb5348b99ed8b5dc38a6087d
I20260709 12:59:46.113636  2585 ts_tablet_manager.cc:933] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:46.138697  1628 tablet_copy_source_session.cc:215] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:46.139019  1627 tablet_copy_source_session.cc:215] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:46.155963  2582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8343e6d9eb5348b99ed8b5dc38a6087d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:46.168242  2581 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a69ce55a73e464a962e15e73fd1db2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:46.170992  2579 tablet_copy_client.cc:806] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:46.171701  2579 tablet_copy_client.cc:670] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:46.175910  2579 tablet_copy_client.cc:538] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:46.186790  2585 tablet_copy_client.cc:323] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 12:59:46.189015  2579 tablet_bootstrap.cc:492] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:46.189513  1701 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet 81301f84b53f41349ac6bca78f1a77f4 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 12:59:46.190125  1701 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet 81301f84b53f41349ac6bca78f1a77f4 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-81301f84b53f41349ac6bca78f1a77f4
I20260709 12:59:46.209985  1701 tablet_copy_source_session.cc:215] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:46.215832  2585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81301f84b53f41349ac6bca78f1a77f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:46.234664  2581 tablet_copy_client.cc:806] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:46.235342  2581 tablet_copy_client.cc:670] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:46.236544  2579 tablet_bootstrap.cc:492] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:46.237563  2579 tablet_bootstrap.cc:492] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:46.238289  2579 ts_tablet_manager.cc:1403] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.050s	user 0.018s	sys 0.011s
I20260709 12:59:46.238538  2582 tablet_copy_client.cc:806] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:46.239161  2582 tablet_copy_client.cc:670] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:46.240397  2581 tablet_copy_client.cc:538] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:46.243448  2582 tablet_copy_client.cc:538] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:46.246505  2579 raft_consensus.cc:359] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:46.247838  2579 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:46.250458  2585 tablet_copy_client.cc:806] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:46.251080  2585 tablet_copy_client.cc:670] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:46.251511  2581 tablet_bootstrap.cc:492] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:46.255028  2579 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:46.264525  2585 tablet_copy_client.cc:538] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:46.265020  2582 tablet_bootstrap.cc:492] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:46.275245  2585 tablet_bootstrap.cc:492] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:46.278611  2579 ts_tablet_manager.cc:1434] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.040s	user 0.010s	sys 0.004s
I20260709 12:59:46.280463  1553 tablet_copy_service.cc:342] P 83f6f5810950417195f707ff308a8125: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-34b2ba71a23e4ba3becae445ee7598c5 received from {username='slave'} at 127.0.0.1:33722
I20260709 12:59:46.281008  1553 tablet_copy_service.cc:434] P 83f6f5810950417195f707ff308a8125: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-34b2ba71a23e4ba3becae445ee7598c5 on tablet 34b2ba71a23e4ba3becae445ee7598c5 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:46.321468  2582 tablet_bootstrap.cc:492] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:46.322431  2582 tablet_bootstrap.cc:492] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:46.323119  2582 ts_tablet_manager.cc:1403] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.059s	user 0.011s	sys 0.020s
I20260709 12:59:46.326467  2582 raft_consensus.cc:359] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:46.334697  2582 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:46.335384  2582 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:46.341071  2582 ts_tablet_manager.cc:1434] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:59:46.338531  2585 tablet_bootstrap.cc:492] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:46.342284  2585 tablet_bootstrap.cc:492] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:46.342998  2585 ts_tablet_manager.cc:1403] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.068s	user 0.010s	sys 0.013s
I20260709 12:59:46.348292  2585 raft_consensus.cc:359] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:46.348881  1627 tablet_copy_service.cc:342] P 8a6a5ae1b7f240408b3ddd30d8a18262: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-8343e6d9eb5348b99ed8b5dc38a6087d received from {username='slave'} at 127.0.0.1:45356
I20260709 12:59:46.349217  2585 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:46.349633  1627 tablet_copy_service.cc:434] P 8a6a5ae1b7f240408b3ddd30d8a18262: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-8343e6d9eb5348b99ed8b5dc38a6087d on tablet 8343e6d9eb5348b99ed8b5dc38a6087d with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:46.349962  2585 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:46.354296  2585 ts_tablet_manager.cc:1434] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:59:46.355955  1701 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-81301f84b53f41349ac6bca78f1a77f4 received from {username='slave'} at 127.0.0.1:40014
I20260709 12:59:46.356443  1701 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-81301f84b53f41349ac6bca78f1a77f4 on tablet 81301f84b53f41349ac6bca78f1a77f4 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:46.373382  2581 tablet_bootstrap.cc:492] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:46.374431  2581 tablet_bootstrap.cc:492] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:46.375156  2581 ts_tablet_manager.cc:1403] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.124s	user 0.037s	sys 0.041s
I20260709 12:59:46.379968  2581 raft_consensus.cc:359] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:46.381001  2581 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:46.381970  2581 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:46.387636  2581 ts_tablet_manager.cc:1434] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 12:59:46.393208  1627 tablet_copy_service.cc:342] P 8a6a5ae1b7f240408b3ddd30d8a18262: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-1a69ce55a73e464a962e15e73fd1db2c received from {username='slave'} at 127.0.0.1:45356
I20260709 12:59:46.394096  1627 tablet_copy_service.cc:434] P 8a6a5ae1b7f240408b3ddd30d8a18262: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-1a69ce55a73e464a962e15e73fd1db2c on tablet 1a69ce55a73e464a962e15e73fd1db2c with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:46.472096  2236 raft_consensus.cc:1217] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:46.476276  2236 raft_consensus.cc:1217] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:46.586674  2236 raft_consensus.cc:1217] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:46.587073  2233 raft_consensus.cc:1217] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:46.868881  2576 raft_consensus.cc:1064] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:46.871608  2576 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:46.889809  2233 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.891974  1613 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.892982  2577 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:46.895067  2576 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:46.895315  1537 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.898662  2495 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:46.907328  2573 raft_consensus.cc:1064] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:46.939285  2573 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:46.980265  2236 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.981626  1688 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.983206  2573 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:46.983887  1618 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:46.985061  2594 raft_consensus.cc:2955] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:46.986866  2236 raft_consensus.cc:2955] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:46.991247  2574 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:46.993693  2592 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:46.999940  1541 raft_consensus.cc:2955] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.008550  1439 catalog_manager.cc:5697] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:47.016610  1613 raft_consensus.cc:2955] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.070652  2233 raft_consensus.cc:2955] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.086436  1617 raft_consensus.cc:2955] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.096968  2573 raft_consensus.cc:2955] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.116966  1691 raft_consensus.cc:2955] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.139448  2553 raft_consensus.cc:1064] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:47.136466  1437 catalog_manager.cc:5697] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.142844  2578 raft_consensus.cc:1064] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:47.145628  2578 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:47.150401  1687 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:47.157401  2233 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.156992  2553 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:47.159780  1541 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.162484  2606 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:47.163717  2577 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:47.176174  1541 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:47.185854  2576 raft_consensus.cc:2955] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.189935  2236 raft_consensus.cc:2955] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.246614  2233 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.248468  2573 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:47.257364  1542 raft_consensus.cc:2955] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.262861  1691 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.270081  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 81301f84b53f41349ac6bca78f1a77f4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:47.271150  2592 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:47.272680  2573 raft_consensus.cc:2955] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.276829  2233 raft_consensus.cc:2955] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.283283  1691 raft_consensus.cc:2955] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.290454  2236 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:47.291021  1687 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:47.292124  2553 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:47.292997  1541 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:47.293740  2527 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:47.304730  2553 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:47.312093  1438 catalog_manager.cc:5697] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.325788  2233 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:47.327225  2527 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:47.328029  1692 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:47.329934  2553 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:47.335878  1606 tablet_service.cc:1558] Processing DeleteTablet for tablet 81301f84b53f41349ac6bca78f1a77f4 with delete_type TABLET_DATA_TOMBSTONED (TS 8a6a5ae1b7f240408b3ddd30d8a18262 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43382
I20260709 12:59:47.338527  1537 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:47.341905  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 34b2ba71a23e4ba3becae445ee7598c5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:47.342012  1439 catalog_manager.cc:5697] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.357187  2575 raft_consensus.cc:2955] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.368144  1691 raft_consensus.cc:2955] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.371824  1601 tablet_service.cc:1558] Processing DeleteTablet for tablet 34b2ba71a23e4ba3becae445ee7598c5 with delete_type TABLET_DATA_TOMBSTONED (TS 8a6a5ae1b7f240408b3ddd30d8a18262 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43382
I20260709 12:59:47.377945  2613 tablet_replica.cc:333] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 12:59:47.379251  2613 raft_consensus.cc:2243] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:47.378159  2236 raft_consensus.cc:2955] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.380302  2613 raft_consensus.cc:2272] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:47.384446  2527 raft_consensus.cc:2955] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.386292  2233 raft_consensus.cc:2955] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.395592  2525 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:47.391779  2613 ts_tablet_manager.cc:1916] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:47.389328  1440 catalog_manager.cc:5697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:47.408674  1691 raft_consensus.cc:2955] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.420863  2613 ts_tablet_manager.cc:1929] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:47.421581  2613 log.cc:1199] T 81301f84b53f41349ac6bca78f1a77f4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/wals/81301f84b53f41349ac6bca78f1a77f4
I20260709 12:59:47.423489  1428 catalog_manager.cc:5028] TS 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): tablet 81301f84b53f41349ac6bca78f1a77f4 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:47.424105  2613 tablet_replica.cc:333] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 12:59:47.425133  2613 raft_consensus.cc:2243] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:47.425942  2613 raft_consensus.cc:2272] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:47.427151  1541 raft_consensus.cc:2955] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.429677  2613 ts_tablet_manager.cc:1916] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:47.433324  1617 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:47.435463  1441 catalog_manager.cc:5697] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.437001  1542 raft_consensus.cc:2955] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.448755  1691 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.450732  2233 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.452484  2575 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:47.453442  2525 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:47.463306  2525 raft_consensus.cc:2955] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.465018  2233 raft_consensus.cc:2955] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.475581  2613 ts_tablet_manager.cc:1929] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:47.476097  2613 log.cc:1199] T 34b2ba71a23e4ba3becae445ee7598c5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/wals/34b2ba71a23e4ba3becae445ee7598c5
I20260709 12:59:47.478003  1428 catalog_manager.cc:5028] TS 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): tablet 34b2ba71a23e4ba3becae445ee7598c5 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:47.479389  1688 raft_consensus.cc:2955] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.488633  1441 catalog_manager.cc:5697] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.513072  1528 tablet_service.cc:1558] Processing DeleteTablet for tablet 8343e6d9eb5348b99ed8b5dc38a6087d with delete_type TABLET_DATA_TOMBSTONED (TS 83f6f5810950417195f707ff308a8125 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:59:47.515540  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8343e6d9eb5348b99ed8b5dc38a6087d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:47.547405  2618 tablet_replica.cc:333] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 12:59:47.549196  2618 raft_consensus.cc:2243] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:47.552947  2618 raft_consensus.cc:2272] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:47.556865  2618 ts_tablet_manager.cc:1916] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:59:47.553699  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56532.9us
I20260709 12:59:47.594920  2618 ts_tablet_manager.cc:1929] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:47.595510  2618 log.cc:1199] T 8343e6d9eb5348b99ed8b5dc38a6087d P 83f6f5810950417195f707ff308a8125: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/wals/8343e6d9eb5348b99ed8b5dc38a6087d
I20260709 12:59:47.597941  1428 catalog_manager.cc:5028] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): tablet 8343e6d9eb5348b99ed8b5dc38a6087d (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:47.813843  1618 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:47.820060  2235 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.822434  2578 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:47.831053  2578 raft_consensus.cc:2955] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.832459  2235 raft_consensus.cc:2955] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.840792  1541 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:47.850829  2553 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:47.856595  1541 raft_consensus.cc:2955] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.857857  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1a69ce55a73e464a962e15e73fd1db2c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:47.860026  1437 catalog_manager.cc:5697] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:47.875411  1679 tablet_service.cc:1558] Processing DeleteTablet for tablet 1a69ce55a73e464a962e15e73fd1db2c with delete_type TABLET_DATA_TOMBSTONED (TS 984c9089d9804652846f3caeb609d56e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40104
I20260709 12:59:47.878795  2619 tablet_replica.cc:333] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 12:59:47.880054  2619 raft_consensus.cc:2243] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:47.880640  2619 raft_consensus.cc:2272] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:47.884963  2619 ts_tablet_manager.cc:1916] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:47.901481  2619 ts_tablet_manager.cc:1929] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:47.901851  2619 log.cc:1199] T 1a69ce55a73e464a962e15e73fd1db2c P 984c9089d9804652846f3caeb609d56e: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/wals/1a69ce55a73e464a962e15e73fd1db2c
I20260709 12:59:47.903299  1426 catalog_manager.cc:5028] TS 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): tablet 1a69ce55a73e464a962e15e73fd1db2c (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:48.950380  1542 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:48.956182  1687 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:48.957403  1508 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ab2e02ed3c849e9bda035a57add301d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:48.958055  1618 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:48.958279  2622 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:48.959589  2574 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:48.967108  2573 raft_consensus.cc:2955] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:48.968847  1687 raft_consensus.cc:2955] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:48.970619  1618 raft_consensus.cc:2955] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:48.982360  1617 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:48.982182  1441 catalog_manager.cc:5697] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:48.995297  1687 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:48.995939  1583 consensus_peers.cc:597] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eed65fd1add844e7821095e2215fb393. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:48.997884  2620 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:48.999869  1541 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:49.001233  2575 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:49.019393  2620 raft_consensus.cc:2955] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:49.021530  1687 raft_consensus.cc:2955] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:49.024204  1541 raft_consensus.cc:2955] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:49.038487  1437 catalog_manager.cc:5697] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:49.052460  1617 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
W20260709 12:59:49.061257  1583 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c601f0c0dfb54023907d2722495668cb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:49.061867  1541 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:49.063184  1687 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:49.063697  2553 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:49.064563  2575 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:49.074779  2620 raft_consensus.cc:2955] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:49.076262  1687 raft_consensus.cc:2955] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:49.077183  1541 raft_consensus.cc:2955] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:49.087548  1688 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:49.096591  1441 catalog_manager.cc:5697] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:49.100304  1617 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:49.100533  1542 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:49.101990  1657 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ae2bf844acee4ce5b28b92217256b084. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:49.102725  2577 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:49.103535  2576 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:49.114938  2577 raft_consensus.cc:2955] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:49.118347  1541 raft_consensus.cc:2955] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:49.135270  1615 raft_consensus.cc:2955] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:49.158892  1438 catalog_manager.cc:5697] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:49.380087  2640 ts_tablet_manager.cc:933] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
I20260709 12:59:49.382161  2640 tablet_copy_client.cc:323] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:43459
I20260709 12:59:49.384024  1553 tablet_copy_service.cc:140] P 83f6f5810950417195f707ff308a8125: Received BeginTabletCopySession request for tablet 3ab2e02ed3c849e9bda035a57add301d from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:33722)
I20260709 12:59:49.384696  1553 tablet_copy_service.cc:161] P 83f6f5810950417195f707ff308a8125: Beginning new tablet copy session on tablet 3ab2e02ed3c849e9bda035a57add301d from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:33722: session id = cecad4f86a114a7696e398b3e8cc511e-3ab2e02ed3c849e9bda035a57add301d
I20260709 12:59:49.392076  1553 tablet_copy_source_session.cc:215] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:49.395226  2640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ab2e02ed3c849e9bda035a57add301d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:49.410360  2642 ts_tablet_manager.cc:933] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:49.412842  2640 tablet_copy_client.cc:806] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:49.413668  2640 tablet_copy_client.cc:670] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:49.417908  2640 tablet_copy_client.cc:538] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:49.427785  2640 tablet_bootstrap.cc:492] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:49.455302  2640 tablet_bootstrap.cc:492] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:49.456351  2640 tablet_bootstrap.cc:492] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:49.457095  2640 ts_tablet_manager.cc:1403] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.030s	user 0.026s	sys 0.004s
I20260709 12:59:49.460048  2640 raft_consensus.cc:359] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:49.460979  2640 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:49.461694  2640 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:49.471774  2640 ts_tablet_manager.cc:1434] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.007s
I20260709 12:59:49.473979  1553 tablet_copy_service.cc:342] P 83f6f5810950417195f707ff308a8125: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-3ab2e02ed3c849e9bda035a57add301d received from {username='slave'} at 127.0.0.1:33722
I20260709 12:59:49.474505  1553 tablet_copy_service.cc:434] P 83f6f5810950417195f707ff308a8125: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-3ab2e02ed3c849e9bda035a57add301d on tablet 3ab2e02ed3c849e9bda035a57add301d with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:49.499660  2642 tablet_copy_client.cc:323] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43887
I20260709 12:59:49.501353  1627 tablet_copy_service.cc:140] P 8a6a5ae1b7f240408b3ddd30d8a18262: Received BeginTabletCopySession request for tablet eed65fd1add844e7821095e2215fb393 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:45356)
I20260709 12:59:49.501984  1627 tablet_copy_service.cc:161] P 8a6a5ae1b7f240408b3ddd30d8a18262: Beginning new tablet copy session on tablet eed65fd1add844e7821095e2215fb393 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:45356: session id = cecad4f86a114a7696e398b3e8cc511e-eed65fd1add844e7821095e2215fb393
I20260709 12:59:49.511955  1627 tablet_copy_source_session.cc:215] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:49.517390  2642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eed65fd1add844e7821095e2215fb393. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:49.532353  2642 tablet_copy_client.cc:806] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:49.532904  2642 tablet_copy_client.cc:670] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:49.537662  2642 tablet_copy_client.cc:538] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:49.548492  2642 tablet_bootstrap.cc:492] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:49.565605  2642 tablet_bootstrap.cc:492] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:49.566344  2642 tablet_bootstrap.cc:492] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:49.566913  2642 ts_tablet_manager.cc:1403] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.019s	user 0.019s	sys 0.000s
I20260709 12:59:49.569310  2642 raft_consensus.cc:359] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:49.569989  2642 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:49.570690  2642 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:49.574288  2642 ts_tablet_manager.cc:1434] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 12:59:49.575886  1627 tablet_copy_service.cc:342] P 8a6a5ae1b7f240408b3ddd30d8a18262: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-eed65fd1add844e7821095e2215fb393 received from {username='slave'} at 127.0.0.1:45356
I20260709 12:59:49.576331  1627 tablet_copy_service.cc:434] P 8a6a5ae1b7f240408b3ddd30d8a18262: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-eed65fd1add844e7821095e2215fb393 on tablet eed65fd1add844e7821095e2215fb393 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:49.648901  2642 ts_tablet_manager.cc:933] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:49.650590  2642 tablet_copy_client.cc:323] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43887
I20260709 12:59:49.651993  1627 tablet_copy_service.cc:140] P 8a6a5ae1b7f240408b3ddd30d8a18262: Received BeginTabletCopySession request for tablet c601f0c0dfb54023907d2722495668cb from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:45356)
I20260709 12:59:49.652498  1627 tablet_copy_service.cc:161] P 8a6a5ae1b7f240408b3ddd30d8a18262: Beginning new tablet copy session on tablet c601f0c0dfb54023907d2722495668cb from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:45356: session id = cecad4f86a114a7696e398b3e8cc511e-c601f0c0dfb54023907d2722495668cb
I20260709 12:59:49.659401  1627 tablet_copy_source_session.cc:215] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:49.662521  2642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c601f0c0dfb54023907d2722495668cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:49.676640  2642 tablet_copy_client.cc:806] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:49.677214  2642 tablet_copy_client.cc:670] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:49.680629  2642 tablet_copy_client.cc:538] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:49.686764  2640 ts_tablet_manager.cc:933] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:49.695585  2642 tablet_bootstrap.cc:492] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:49.707501  2640 tablet_copy_client.cc:323] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 12:59:49.709273  1701 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet ae2bf844acee4ce5b28b92217256b084 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 12:59:49.709841  1701 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet ae2bf844acee4ce5b28b92217256b084 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-ae2bf844acee4ce5b28b92217256b084
I20260709 12:59:49.737512  1701 tablet_copy_source_session.cc:215] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:49.743201  2640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae2bf844acee4ce5b28b92217256b084. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:49.773769  2640 tablet_copy_client.cc:806] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:49.774349  2640 tablet_copy_client.cc:670] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:49.778391  2640 tablet_copy_client.cc:538] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:49.791009  2642 tablet_bootstrap.cc:492] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:49.792001  2642 tablet_bootstrap.cc:492] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:49.792084  2640 tablet_bootstrap.cc:492] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:49.792785  2642 ts_tablet_manager.cc:1403] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.097s	user 0.034s	sys 0.008s
I20260709 12:59:49.795938  2642 raft_consensus.cc:359] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:49.796770  2642 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:49.798856  2642 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:49.801878  2642 ts_tablet_manager.cc:1434] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.003s
I20260709 12:59:49.804772  1627 tablet_copy_service.cc:342] P 8a6a5ae1b7f240408b3ddd30d8a18262: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-c601f0c0dfb54023907d2722495668cb received from {username='slave'} at 127.0.0.1:45356
I20260709 12:59:49.805202  1627 tablet_copy_service.cc:434] P 8a6a5ae1b7f240408b3ddd30d8a18262: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-c601f0c0dfb54023907d2722495668cb on tablet c601f0c0dfb54023907d2722495668cb with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:49.846194  2640 tablet_bootstrap.cc:492] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:49.847067  2640 tablet_bootstrap.cc:492] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:49.847697  2640 ts_tablet_manager.cc:1403] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.056s	user 0.023s	sys 0.000s
I20260709 12:59:49.850189  2640 raft_consensus.cc:359] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:49.850814  2640 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:49.851681  2640 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:49.853767  2640 ts_tablet_manager.cc:1434] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:59:49.856307  1701 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-ae2bf844acee4ce5b28b92217256b084 received from {username='slave'} at 127.0.0.1:40014
I20260709 12:59:49.856726  1701 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-ae2bf844acee4ce5b28b92217256b084 on tablet ae2bf844acee4ce5b28b92217256b084 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:49.953127  2236 raft_consensus.cc:1217] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:49.995844  2236 raft_consensus.cc:1217] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:50.080075  2236 raft_consensus.cc:1217] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:50.208369  2236 raft_consensus.cc:1217] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:50.376786  2654 raft_consensus.cc:1064] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:50.378823  2654 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:50.385764  2235 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.385802  1687 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.387521  2620 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:50.388028  1542 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.389525  2654 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:50.390401  2553 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:50.401378  2553 raft_consensus.cc:2955] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.404007  2235 raft_consensus.cc:2955] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.419489  1437 catalog_manager.cc:5697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.406001  1542 raft_consensus.cc:2955] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.421269  1687 raft_consensus.cc:2955] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.463384  1615 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:50.469030  2235 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.469468  1687 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.470999  2575 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.474323  2620 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.484895  2654 raft_consensus.cc:2955] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.487737  2235 raft_consensus.cc:2955] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.489902  1688 raft_consensus.cc:2955] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.495036  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet eed65fd1add844e7821095e2215fb393 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:50.506546  1441 catalog_manager.cc:5697] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:50.516340  2577 raft_consensus.cc:1064] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:50.518641  2577 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:50.520960  1528 tablet_service.cc:1558] Processing DeleteTablet for tablet eed65fd1add844e7821095e2215fb393 with delete_type TABLET_DATA_TOMBSTONED (TS 83f6f5810950417195f707ff308a8125 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:59:50.533644  2235 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.535658  2576 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:50.537971  2663 tablet_replica.cc:333] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 12:59:50.539273  2663 raft_consensus.cc:2243] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:50.540218  2663 raft_consensus.cc:2272] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:50.543921  1537 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.544384  2663 ts_tablet_manager.cc:1916] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:50.554378  1617 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.562645  2653 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:50.565805  2663 ts_tablet_manager.cc:1929] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:50.566380  2663 log.cc:1199] T eed65fd1add844e7821095e2215fb393 P 83f6f5810950417195f707ff308a8125: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/wals/eed65fd1add844e7821095e2215fb393
I20260709 12:59:50.568425  1428 catalog_manager.cc:5028] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): tablet eed65fd1add844e7821095e2215fb393 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:50.572964  2576 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:59:50.586331  2652 raft_consensus.cc:1064] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:50.608225  2620 raft_consensus.cc:1064] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:50.589198  2652 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:50.614492  2576 raft_consensus.cc:2955] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.618952  1537 raft_consensus.cc:2955] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.613101  2620 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:50.667397  2236 raft_consensus.cc:2955] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.672741  1437 catalog_manager.cc:5697] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.675175  1617 raft_consensus.cc:2955] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.683590  1687 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.684602  2233 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.683578  2235 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.686461  1542 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.687018  2669 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:50.688831  2575 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:50.689544  2651 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:50.693187  2654 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:50.694093  1618 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.694298  1688 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:50.696797  2651 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:50.706543  2575 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:50.750524  1691 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:50.764946  1537 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.764892  2233 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.766310  2653 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.775460  2653 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.787364  2576 raft_consensus.cc:2955] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.795804  1537 raft_consensus.cc:2955] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.798287  2651 raft_consensus.cc:2955] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.800477  1687 raft_consensus.cc:2955] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.811790  2234 raft_consensus.cc:2955] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.808277  2654 raft_consensus.cc:2955] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.823132  1437 catalog_manager.cc:5697] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:50.826287  1613 raft_consensus.cc:2955] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.822417  2235 raft_consensus.cc:2955] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.853453  1687 raft_consensus.cc:2955] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.862354  2233 raft_consensus.cc:2955] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.877960  1541 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:50.881042  1538 raft_consensus.cc:2955] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.916456  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ae2bf844acee4ce5b28b92217256b084 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:59:50.917237  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 72311.2us
I20260709 12:59:50.921046  2235 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.922500  2652 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.925122  1437 catalog_manager.cc:5697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.934101  2669 raft_consensus.cc:2955] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.948808  1613 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.950142  2233 raft_consensus.cc:2955] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.951687  2651 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.956277  1440 catalog_manager.cc:5697] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.957921  1618 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:50.957388  1613 raft_consensus.cc:2955] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.970808  2233 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.971280  1541 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:50.971369  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3ab2e02ed3c849e9bda035a57add301d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:50.972389  2678 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:50.974856  1606 tablet_service.cc:1558] Processing DeleteTablet for tablet ae2bf844acee4ce5b28b92217256b084 with delete_type TABLET_DATA_TOMBSTONED (TS 8a6a5ae1b7f240408b3ddd30d8a18262 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43382
I20260709 12:59:50.975908  2680 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:50.985321  2620 raft_consensus.cc:2955] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.995896  1537 raft_consensus.cc:2955] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:50.996883  1440 catalog_manager.cc:5697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:51.002216  2235 raft_consensus.cc:2955] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:51.003686  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c601f0c0dfb54023907d2722495668cb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:51.003608  2684 tablet_replica.cc:333] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 12:59:51.005481  2684 raft_consensus.cc:2243] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:51.006286  2684 raft_consensus.cc:2272] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:51.016980  2684 ts_tablet_manager.cc:1916] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:51.029768  1679 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ab2e02ed3c849e9bda035a57add301d with delete_type TABLET_DATA_TOMBSTONED (TS 984c9089d9804652846f3caeb609d56e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40104
I20260709 12:59:51.031759  2685 tablet_replica.cc:333] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 12:59:51.031950  2684 ts_tablet_manager.cc:1929] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:51.032486  2684 log.cc:1199] T ae2bf844acee4ce5b28b92217256b084 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/wals/ae2bf844acee4ce5b28b92217256b084
I20260709 12:59:51.034976  1428 catalog_manager.cc:5028] TS 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): tablet ae2bf844acee4ce5b28b92217256b084 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:51.035661  2685 raft_consensus.cc:2243] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:51.035267  1441 catalog_manager.cc:5697] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:51.037406  2685 raft_consensus.cc:2272] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:51.043684  2685 ts_tablet_manager.cc:1916] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:51.048966  1679 tablet_service.cc:1558] Processing DeleteTablet for tablet c601f0c0dfb54023907d2722495668cb with delete_type TABLET_DATA_TOMBSTONED (TS 984c9089d9804652846f3caeb609d56e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40104
I20260709 12:59:51.058107  2685 ts_tablet_manager.cc:1929] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:51.058593  2685 log.cc:1199] T 3ab2e02ed3c849e9bda035a57add301d P 984c9089d9804652846f3caeb609d56e: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/wals/3ab2e02ed3c849e9bda035a57add301d
I20260709 12:59:51.060794  2685 tablet_replica.cc:333] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 12:59:51.061668  1426 catalog_manager.cc:5028] TS 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): tablet 3ab2e02ed3c849e9bda035a57add301d (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:51.063216  2685 raft_consensus.cc:2243] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:51.063859  2685 raft_consensus.cc:2272] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:51.066265  2685 ts_tablet_manager.cc:1916] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:51.080081  2685 ts_tablet_manager.cc:1929] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:51.080458  2685 log.cc:1199] T c601f0c0dfb54023907d2722495668cb P 984c9089d9804652846f3caeb609d56e: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/wals/c601f0c0dfb54023907d2722495668cb
I20260709 12:59:51.081874  1426 catalog_manager.cc:5028] TS 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): tablet c601f0c0dfb54023907d2722495668cb (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:51.770916   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:51.824033  1618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13adc029302444abaaffd4bed11760a3"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:51.824738  1618 raft_consensus.cc:606] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:51.827060  1618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:51.827512  1618 raft_consensus.cc:606] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:51.829808  1618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:51.830314  1618 raft_consensus.cc:606] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:51.832616  1618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "474d3b57c33542aba459b97733122bf0"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:51.833191  1618 raft_consensus.cc:606] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:51.835778  1618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:51.836225  1618 raft_consensus.cc:606] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:51.848894  1691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:49352
I20260709 12:59:51.849400  1691 raft_consensus.cc:606] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:51.850613   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:59:51.851696   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:51.969633  2681 raft_consensus.cc:993] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:51.970168  2620 raft_consensus.cc:1081] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:51.971491  1537 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 12:59:51.972057  1537 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:51.972388  1537 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:51.977114  1537 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:51.979645  1537 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:51.980300  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:51.980952  1618 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:51.981258  1618 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:51.982056  1618 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:51.983284  1618 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:51.985800  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:51.986407  1691 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:51.990448  1618 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 12:59:51.992012  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:51.993561  2686 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:51.993942  1691 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 12:59:51.994092  2686 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 3 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:51.998387  2686 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:51.998840  2678 raft_consensus.cc:993] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:51.999493  2681 raft_consensus.cc:1081] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:52.001200  1541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 12:59:52.001715  1541 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:52.002110  1541 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.010645  1438 catalog_manager.cc:5697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 2 to 3, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:52.014561  1541 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:52.016252  1541 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:52.017051  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:52.017369  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:52.017782  1618 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:52.018229  1618 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:52.018280  1691 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.019102  1618 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:52.020326  1618 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.024789  1691 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:52.026356  1507 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:52.027100  2686 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:52.027585  2686 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:52.028523  2686 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:52.030746  2670 raft_consensus.cc:993] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: : Instructing follower cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:52.031261  2670 raft_consensus.cc:1081] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Signalling peer cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:52.032975  2233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:39452
I20260709 12:59:52.032920  2678 raft_consensus.cc:993] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:52.033614  2233 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:52.033972  2233 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.034086  2620 raft_consensus.cc:1081] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:52.036291  1541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "474d3b57c33542aba459b97733122bf0"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 12:59:52.036967  1541 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:52.037408  1541 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.051903  2233 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:52.056373  1618 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:52.056895  2233 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:52.058324  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:52.058943  1687 raft_consensus.cc:3055] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:52.059255  1687 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:52.058355  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:52.059872  1613 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.059756  1541 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:52.060024  1687 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:52.061151  1687 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.062358  1541 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:52.063311  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:52.070065  1692 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.074999  1438 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:52.078286  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:52.085865  1618 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:52.087399  1618 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:52.088173  1618 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:52.090108  1618 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.092252  1687 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:52.095005  1613 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:52.095734  2201 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:52.092924  1692 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:52.100713  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:52.102653  2686 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:52.103404  2686 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:52.104275  2686 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:52.105954  1618 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:52.106264  2696 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:52.114313  2696 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 12:59:52.115255  2696 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:52.118705  1440 catalog_manager.cc:5697] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:52.126113  1438 catalog_manager.cc:5697] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 2, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:52.132190  2694 raft_consensus.cc:993] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:52.132714  2694 raft_consensus.cc:1081] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:52.134344  1541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13adc029302444abaaffd4bed11760a3"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 12:59:52.134865  1541 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:52.135213  1541 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.141579  1541 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:52.144796  1541 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:52.144714  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:52.144270  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:52.145604  1692 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.145804  1618 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:52.146157  1618 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:52.147768  1618 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:52.148826  1618 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.151016  1692 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:52.152031  1507 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:52.152721  2686 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:52.153152  2686 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:52.154001  2686 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:52.155293  1618 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:52.165519  1438 catalog_manager.cc:5697] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:52.171028  1541 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:52.176901  1692 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:59:52.177080  1508 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 380b2dca2259476bbe1f22b2eb110fd0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:52.178121  2669 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.179275  1618 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:52.180718  2669 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.186744  2686 raft_consensus.cc:2955] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:52.188161  1692 raft_consensus.cc:2955] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:52.206350  1438 catalog_manager.cc:5697] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:52.212954  1617 raft_consensus.cc:2955] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:52.226603  1692 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
W20260709 12:59:52.244458  1657 consensus_peers.cc:597] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e4d8a1326e854e51b5539e7addc91fe5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:52.244942  1617 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:52.247201  1537 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:52.247064  2670 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:59:52.250376  2683 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.261519  2683 raft_consensus.cc:2955] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:52.263118  1537 raft_consensus.cc:2955] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:52.274876  1541 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:52.280804  1692 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:52.282229  1618 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:52.281877  1440 catalog_manager.cc:5697] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:52.282893  1508 consensus_peers.cc:597] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bb1171fd6323498ea16dfea3e5356fe0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:52.283491  2686 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.285948  1617 raft_consensus.cc:2955] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:52.286731  2669 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.323050  2686 raft_consensus.cc:2955] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:52.328895  2694 raft_consensus.cc:993] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:52.329483  2694 raft_consensus.cc:1081] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:52.332013  1537 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 12:59:52.332599  1537 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:52.332945  1537 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.336046  1692 raft_consensus.cc:2955] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:52.337635  1618 raft_consensus.cc:2955] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
W20260709 12:59:52.339362  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet 3ce476a5b0534bc5b6915606f6d37f30: Remote error: Service unavailable: leader transfer in progress
I20260709 12:59:52.343886  1537 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:52.351310  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:52.351873  1617 raft_consensus.cc:3055] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:52.352140  1617 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:52.352824  1617 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:52.353863  1617 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.356276  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:52.356871  1692 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:52.365556  1537 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:52.366487  1440 catalog_manager.cc:5697] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:52.367854  1692 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:52.368904  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:52.369680  2651 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:52.370131  2651 raft_consensus.cc:697] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:52.370958  2651 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:52.371959  1617 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:52.382110  1617 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:52.386873  2651 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:52.390390  1440 catalog_manager.cc:5697] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:52.405745  1692 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:52.410698  1617 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.412032  2669 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:52.412477  2686 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:52.444310  1691 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.446271  2669 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.538128  1691 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:52.539224  2696 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:52.556314  1617 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.558368  2669 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.573169  1618 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:52.574829  2696 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:52.581904  1613 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.583721  2669 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.614233  1691 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.616014  2669 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.635206  1691 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.636597  2651 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.744223  2726 ts_tablet_manager.cc:933] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:52.758431  2727 ts_tablet_manager.cc:933] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
I20260709 12:59:52.764034  2726 tablet_copy_client.cc:323] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 12:59:52.765794  1701 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet e4d8a1326e854e51b5539e7addc91fe5 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 12:59:52.766366  1701 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet e4d8a1326e854e51b5539e7addc91fe5 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-e4d8a1326e854e51b5539e7addc91fe5
I20260709 12:59:52.766584  2727 tablet_copy_client.cc:323] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:43459
I20260709 12:59:52.772360  1553 tablet_copy_service.cc:140] P 83f6f5810950417195f707ff308a8125: Received BeginTabletCopySession request for tablet 380b2dca2259476bbe1f22b2eb110fd0 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:33722)
I20260709 12:59:52.772972  1553 tablet_copy_service.cc:161] P 83f6f5810950417195f707ff308a8125: Beginning new tablet copy session on tablet 380b2dca2259476bbe1f22b2eb110fd0 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:33722: session id = cecad4f86a114a7696e398b3e8cc511e-380b2dca2259476bbe1f22b2eb110fd0
I20260709 12:59:52.776690  1701 tablet_copy_source_session.cc:215] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:52.780293  2726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4d8a1326e854e51b5539e7addc91fe5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:52.780189  1553 tablet_copy_source_session.cc:215] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:52.783481  2727 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 380b2dca2259476bbe1f22b2eb110fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:52.799059  2727 tablet_copy_client.cc:806] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:52.799656  2727 tablet_copy_client.cc:670] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:52.801231  2726 tablet_copy_client.cc:806] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:52.801800  2726 tablet_copy_client.cc:670] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:52.804670  2727 tablet_copy_client.cc:538] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:52.812229  2726 tablet_copy_client.cc:538] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:52.813771  2727 tablet_bootstrap.cc:492] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:52.818648  2726 tablet_bootstrap.cc:492] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:52.852895   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:52.867357  2727 tablet_bootstrap.cc:492] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:52.868234  2726 tablet_bootstrap.cc:492] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:52.869379  2726 tablet_bootstrap.cc:492] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:52.872365  2726 ts_tablet_manager.cc:1403] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.054s	user 0.021s	sys 0.019s
I20260709 12:59:52.868387  2727 tablet_bootstrap.cc:492] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:52.875100  2727 ts_tablet_manager.cc:1403] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.062s	user 0.026s	sys 0.012s
I20260709 12:59:52.875793  2726 raft_consensus.cc:359] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:52.876763  2726 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:52.877522  2726 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:52.878320  2727 raft_consensus.cc:359] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:52.879274  2727 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:52.879910  2727 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:52.881196  2726 ts_tablet_manager.cc:1434] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:59:52.883456  2727 ts_tablet_manager.cc:1434] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:59:52.884898  1701 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-e4d8a1326e854e51b5539e7addc91fe5 received from {username='slave'} at 127.0.0.1:40014
I20260709 12:59:52.885380  1701 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-e4d8a1326e854e51b5539e7addc91fe5 on tablet e4d8a1326e854e51b5539e7addc91fe5 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:52.885592  1553 tablet_copy_service.cc:342] P 83f6f5810950417195f707ff308a8125: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-380b2dca2259476bbe1f22b2eb110fd0 received from {username='slave'} at 127.0.0.1:33722
I20260709 12:59:52.886057  1553 tablet_copy_service.cc:434] P 83f6f5810950417195f707ff308a8125: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-380b2dca2259476bbe1f22b2eb110fd0 on tablet 380b2dca2259476bbe1f22b2eb110fd0 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:52.901598  2726 ts_tablet_manager.cc:933] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
I20260709 12:59:52.915828  2726 tablet_copy_client.cc:323] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:43459
I20260709 12:59:52.917686  1553 tablet_copy_service.cc:140] P 83f6f5810950417195f707ff308a8125: Received BeginTabletCopySession request for tablet bb1171fd6323498ea16dfea3e5356fe0 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:33722)
I20260709 12:59:52.918221  1553 tablet_copy_service.cc:161] P 83f6f5810950417195f707ff308a8125: Beginning new tablet copy session on tablet bb1171fd6323498ea16dfea3e5356fe0 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:33722: session id = cecad4f86a114a7696e398b3e8cc511e-bb1171fd6323498ea16dfea3e5356fe0
I20260709 12:59:52.928349  1618 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.930241  2651 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.933075  1553 tablet_copy_source_session.cc:215] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:52.936589  2726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb1171fd6323498ea16dfea3e5356fe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:52.946337   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:52.959362  2726 tablet_copy_client.cc:806] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:52.962450  1691 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:52.969973  2726 tablet_copy_client.cc:670] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:52.975847  2686 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:52.978183  2726 tablet_copy_client.cc:538] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:52.990681  2726 tablet_bootstrap.cc:492] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:53.025851  2726 tablet_bootstrap.cc:492] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:53.026885  2726 tablet_bootstrap.cc:492] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:53.027679  2726 ts_tablet_manager.cc:1403] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.037s	user 0.020s	sys 0.013s
I20260709 12:59:53.030929  2726 raft_consensus.cc:359] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:53.031853  2726 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:53.032541  2726 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:53.038532  2726 ts_tablet_manager.cc:1434] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.006s
I20260709 12:59:53.044425  1553 tablet_copy_service.cc:342] P 83f6f5810950417195f707ff308a8125: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-bb1171fd6323498ea16dfea3e5356fe0 received from {username='slave'} at 127.0.0.1:33722
I20260709 12:59:53.044919  1553 tablet_copy_service.cc:434] P 83f6f5810950417195f707ff308a8125: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-bb1171fd6323498ea16dfea3e5356fe0 on tablet bb1171fd6323498ea16dfea3e5356fe0 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:53.055685  1618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:53.089514  1618 raft_consensus.cc:606] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:53.142273  1537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:53670
I20260709 12:59:53.142935  1537 raft_consensus.cc:606] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:53.146414  1615 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:53.147051  1615 raft_consensus.cc:606] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:53.157407  1687 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 12:59:53.158030  1687 raft_consensus.cc:606] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:53.161098  1615 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:53.161705  1615 raft_consensus.cc:606] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:53.164179  1615 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eed65fd1add844e7821095e2215fb393"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:53.164723  1615 raft_consensus.cc:606] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:53.165763   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:59:53.167347   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:53.186239  2235 raft_consensus.cc:1217] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:53.241998  2669 raft_consensus.cc:993] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: : Instructing follower cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:53.242601  2686 raft_consensus.cc:1081] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Signalling peer cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:53.244244  2235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:39466
I20260709 12:59:53.244863  2235 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.245236  2235 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.264930  2235 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:53.268584  2235 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:53.269814  2234 raft_consensus.cc:1217] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:53.274134  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:53.274708  1537 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.275002  1537 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:53.275437  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:53.275983  1618 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.275673  1537 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:53.276746  1537 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.286924  1618 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:53.288059  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:53.288781  2696 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.289249  2696 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 12:59:53.290431  1537 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:53.290439  2696 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:53.303416  1440 catalog_manager.cc:5697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 2, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:53.311277  2681 raft_consensus.cc:993] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:53.311776  2678 raft_consensus.cc:1081] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:53.313726  2234 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:49846
I20260709 12:59:53.314312  2234 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.314795  2234 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.323738  2234 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:53.326388  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:53.327025  1537 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.333817  2234 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:53.334316  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:53.334918  1618 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.335233  1618 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:53.336047  1618 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:53.337616  1618 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.338093  1537 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:53.339416  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:53.340253  2696 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.344046  2696 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 12:59:53.344928  2696 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:53.348956  2707 raft_consensus.cc:993] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:53.349483  2739 raft_consensus.cc:1081] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:53.352756  1618 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:53.356689  1537 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54764
I20260709 12:59:53.357353  1537 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.357782  1537 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.363746  1440 catalog_manager.cc:5697] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:53.369360  1537 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:53.372339  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:53.372941  1618 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.384200  1618 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:53.385394  1537 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:53.385468  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:53.386840  2669 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.387117  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:53.387290  2669 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:53.387728  1692 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.388103  1692 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:53.388263  2669 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:53.388821  1692 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:53.390024  1692 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.400857  1692 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:53.404099  1440 catalog_manager.cc:5697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:53.414999  2678 raft_consensus.cc:993] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:53.415481  2694 raft_consensus.cc:1081] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:53.417037  2234 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eed65fd1add844e7821095e2215fb393"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:49846
I20260709 12:59:53.417579  2234 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.417896  2234 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.424310  2234 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:53.427546  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:53.427716  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:53.426882  2234 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:53.428157  1618 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.428299  1692 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.428541  1618 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:53.429337  1618 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:53.430518  1618 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.433480  1692 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:53.434594  2201 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:53.435237  2696 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.435683  2696 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 12:59:53.436511  2696 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:53.437615  1618 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:53.448184  1440 catalog_manager.cc:5697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:53.461859  2678 raft_consensus.cc:993] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:53.462365  2694 raft_consensus.cc:1081] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:53.463798  1541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 12:59:53.464330  1541 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.464668  1541 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.471453  1541 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:53.473810  1541 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:53.473965  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:53.474960  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:53.475549  1692 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.474998  1618 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.477397  1618 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:53.478183  1618 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:53.479074  1618 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.482893  1692 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:53.483946  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:53.484715  2669 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.485214  2669 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:53.485389  1618 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:53.486147  2669 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:53.492414  2234 raft_consensus.cc:1217] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:53.494285  2694 raft_consensus.cc:993] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:53.494787  2694 raft_consensus.cc:1081] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:53.496369  1537 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 12:59:53.496894  1537 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:53.497228  1537 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.503119  1441 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:53.506271  1537 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:53.508652  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:53.508848  1537 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:53.509191  1615 raft_consensus.cc:3055] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:53.509521  1615 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:53.510231  1615 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:53.511186  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:53.511358  1615 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.511744  1692 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:53.520274  1615 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:53.520514  1692 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:53.521544  1509 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:53.522313  2651 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:53.523003  2651 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:53.523902  2651 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:53.537611  1441 catalog_manager.cc:5697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:53.680986  2669 raft_consensus.cc:1064] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:53.683389  2669 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:53.690900  2234 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.690981  1692 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.692917  1618 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.693936  2651 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.696012  2651 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.699533  2669 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.714289  2651 raft_consensus.cc:2955] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:53.726747  2233 raft_consensus.cc:2955] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:53.724347  1692 raft_consensus.cc:2955] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:53.740904  1441 catalog_manager.cc:5697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:53.750036  1618 raft_consensus.cc:2955] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:53.766366  1537 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:53.776311  1692 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:53.776311  2233 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:53.777707  2651 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:53.780411  2749 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:53.783645  2669 raft_consensus.cc:2955] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:53.785650  1692 raft_consensus.cc:2955] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:53.790663  1618 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:53.792027  2651 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:53.794436  1541 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:53.795854  2696 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:53.799813  2233 raft_consensus.cc:2955] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:53.802615  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 380b2dca2259476bbe1f22b2eb110fd0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:59:53.854656  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46962.6us
I20260709 12:59:53.848858  1440 catalog_manager.cc:5697] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 3 to 4, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 12:59:53.862982  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 54273.1us
I20260709 12:59:53.870260  1692 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:53.883337  2697 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:59:53.887526  1615 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:53.897406  2697 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:53.911994  1606 tablet_service.cc:1558] Processing DeleteTablet for tablet 380b2dca2259476bbe1f22b2eb110fd0 with delete_type TABLET_DATA_TOMBSTONED (TS 8a6a5ae1b7f240408b3ddd30d8a18262 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:43382
I20260709 12:59:53.927639  2651 raft_consensus.cc:1064] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:53.930596  2651 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:53.935274  1617 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:53.936934  2757 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:53.937346  1617 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:53.940768  2707 raft_consensus.cc:1064] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:53.944968  1537 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:53.951532  2768 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:53.956373  2768 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:53.960136  1618 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:53.968335  2770 tablet_replica.cc:333] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 12:59:53.973230  2233 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:53.974049  2768 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:53.976363  2757 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:53.984742  2770 raft_consensus.cc:2243] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:53.985657  2770 raft_consensus.cc:2272] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:53.989365  2770 ts_tablet_manager.cc:1916] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:59:54.004907  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 124156us
I20260709 12:59:54.021507  1691 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.024362  1688 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:54.024466  2770 ts_tablet_manager.cc:1929] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:54.024922  2770 log.cc:1199] T 380b2dca2259476bbe1f22b2eb110fd0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/wals/380b2dca2259476bbe1f22b2eb110fd0
I20260709 12:59:54.027107  1688 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.028580  1428 catalog_manager.cc:5028] TS 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): tablet 380b2dca2259476bbe1f22b2eb110fd0 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:54.035763  2651 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:54.036597  2755 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:54.041930  2707 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:54.046875  1615 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:54.048655  1615 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.050912  2757 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:54.081863  2755 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:54.099224  2651 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:54.099901  1688 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:54.102063  2758 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:54.133395  2757 raft_consensus.cc:2955] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.140980  1692 raft_consensus.cc:2955] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.168332  1537 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:54.168665   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:54.177889  2764 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:54.181895  2233 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:54.181692  2234 raft_consensus.cc:2955] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.172593  1615 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:54.187387  2764 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:54.215651  2707 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:54.227010  1437 catalog_manager.cc:5697] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:54.242992  2233 raft_consensus.cc:2955] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.240919  2707 raft_consensus.cc:2955] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.300531  1618 raft_consensus.cc:2955] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.303148  1440 catalog_manager.cc:5697] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.303979  1617 raft_consensus.cc:2955] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.320914  1537 raft_consensus.cc:2955] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.334631  1541 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:54.340868  2234 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:54.343855  1618 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:54.346491  2651 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:54.356422  2755 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:54.358261  1690 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:54.360596  2651 raft_consensus.cc:2955] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.365582  2234 raft_consensus.cc:2955] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.370978  1617 raft_consensus.cc:2955] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.391271  2233 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:54.392243  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet bb1171fd6323498ea16dfea3e5356fe0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:54.394277  2707 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:54.398990  1441 catalog_manager.cc:5697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 3 to 4, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.402490  1618 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:54.405136  2707 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:54.409650  2764 raft_consensus.cc:2955] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.410552  1679 tablet_service.cc:1558] Processing DeleteTablet for tablet bb1171fd6323498ea16dfea3e5356fe0 with delete_type TABLET_DATA_TOMBSTONED (TS 984c9089d9804652846f3caeb609d56e not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:40104
I20260709 12:59:54.411187  2233 raft_consensus.cc:2955] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.424717  1618 raft_consensus.cc:2955] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:54.426765  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e4d8a1326e854e51b5539e7addc91fe5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:54.439201  1441 catalog_manager.cc:5697] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:54.452889  2791 tablet_replica.cc:333] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 12:59:54.454046  2791 raft_consensus.cc:2243] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:54.454818  1528 tablet_service.cc:1558] Processing DeleteTablet for tablet e4d8a1326e854e51b5539e7addc91fe5 with delete_type TABLET_DATA_TOMBSTONED (TS 83f6f5810950417195f707ff308a8125 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:59:54.468796   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:54.469352  2792 tablet_replica.cc:333] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 12:59:54.454826  2791 raft_consensus.cc:2272] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:54.475303  2792 raft_consensus.cc:2243] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:54.476034  2792 raft_consensus.cc:2272] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:54.476991  2791 ts_tablet_manager.cc:1916] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:54.486121  2792 ts_tablet_manager.cc:1916] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:54.492440  2791 ts_tablet_manager.cc:1929] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:54.492899  2791 log.cc:1199] T bb1171fd6323498ea16dfea3e5356fe0 P 984c9089d9804652846f3caeb609d56e: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/wals/bb1171fd6323498ea16dfea3e5356fe0
I20260709 12:59:54.494493  1426 catalog_manager.cc:5028] TS 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): tablet bb1171fd6323498ea16dfea3e5356fe0 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:54.503063  2792 ts_tablet_manager.cc:1929] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:54.503545  2792 log.cc:1199] T e4d8a1326e854e51b5539e7addc91fe5 P 83f6f5810950417195f707ff308a8125: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/wals/e4d8a1326e854e51b5539e7addc91fe5
I20260709 12:59:54.505378  1428 catalog_manager.cc:5028] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): tablet e4d8a1326e854e51b5539e7addc91fe5 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:54.518402  1541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:53670
I20260709 12:59:54.518935  1541 raft_consensus.cc:606] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:54.521293  1617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22357a3949344a1792e5bae4a0291987"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:54.522004  1617 raft_consensus.cc:606] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:54.524750  1541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:53670
I20260709 12:59:54.525300  1541 raft_consensus.cc:606] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:54.535323  1692 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:49352
I20260709 12:59:54.535933  1692 raft_consensus.cc:606] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:54.545001  1617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88cbab659ea041148748114fae549ae0"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:54.545974  1617 raft_consensus.cc:606] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:54.552170  1537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:53670
I20260709 12:59:54.552668  1537 raft_consensus.cc:606] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:54.553869   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:59:54.555454   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:54.673058  2757 raft_consensus.cc:993] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: : Instructing follower cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:54.673619  2651 raft_consensus.cc:1081] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Signalling peer cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:54.675295  2234 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:39466
I20260709 12:59:54.675887  2234 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:54.676322  2234 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.684333  2757 raft_consensus.cc:993] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: : Instructing follower 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 12:59:54.684926  2651 raft_consensus.cc:1081] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Signalling peer 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 12:59:54.685570  2234 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:54.686630  1618 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:43398
I20260709 12:59:54.687135  1618 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:54.687467  1618 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.692816  1618 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:54.694746  2234 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:54.695569  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:54.695883  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:54.696154  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:54.696424  1541 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:54.695569  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:54.696832  1618 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:54.697063  1692 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.696854  1690 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.696202  1537 raft_consensus.cc:3055] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:54.698153  1537 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:54.696857  1541 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:54.699338  1541 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:54.699049  1537 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:54.700541  1541 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.700982  1537 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.703938  1692 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 12:59:54.705899  1582 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:54.706661  2773 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:54.707126  2773 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:54.707157  1690 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:54.708177  1537 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 12:59:54.708122  2773 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:54.708649  2201 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:54.709733  2768 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:54.710306  2768 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 12:59:54.711200  2768 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:54.708583  1541 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:54.721170  1440 catalog_manager.cc:5697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 2, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:54.723786  1438 catalog_manager.cc:5697] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 1 to 2, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:54.815647  2678 raft_consensus.cc:993] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:54.816064  2773 raft_consensus.cc:1081] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:54.817406  1541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22357a3949344a1792e5bae4a0291987"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 12:59:54.817915  1541 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:54.818183  1541 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.822939  1541 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:54.824457  1541 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:54.825404  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:54.826073  1618 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:54.826035  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:54.826566  1618 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:54.826761  1690 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.827339  1618 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:54.828465  1618 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.833231  1690 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:54.834302  1618 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:54.834493  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:54.835294  2757 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:54.835760  2757 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:54.836516  2757 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:54.846508  1438 catalog_manager.cc:5697] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:54.889246  2757 raft_consensus.cc:993] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: : Instructing follower cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:54.889844  2758 raft_consensus.cc:1081] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Signalling peer cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:54.891215  2234 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:39466
I20260709 12:59:54.891759  2234 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:54.892077  2234 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.896975  2234 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:54.898803  2234 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:54.899555  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:54.899806  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:54.900100  1541 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:54.900347  1692 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.900455  1541 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:54.901144  1541 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:54.902305  1541 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:54.905133  1692 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:54.906461  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:54.907233  2768 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:54.908650  2768 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 12:59:54.909554  2768 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:54.910007  1541 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:54.920269  1438 catalog_manager.cc:5697] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 2, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:55.007691  2774 raft_consensus.cc:993] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:55.008152  2678 raft_consensus.cc:1081] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:55.009523  1541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88cbab659ea041148748114fae549ae0"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 12:59:55.009968  1541 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:55.010227  1541 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.014798  1541 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:55.016300  1541 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:55.017294  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:55.017961  1618 raft_consensus.cc:3055] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:55.017294  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:55.018316  1618 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:55.018616  1692 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.019049  1618 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:55.020196  1618 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.025467  1692 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:55.026258  1618 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:55.026441  1507 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:55.027307  2758 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:55.027822  2758 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:55.028654  2758 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:55.038234  1438 catalog_manager.cc:5697] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.074988  2767 raft_consensus.cc:993] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: : Instructing follower cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:55.075465  2764 raft_consensus.cc:1081] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Signalling peer cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:55.076884  2234 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:39452
I20260709 12:59:55.077513  2234 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:55.077883  2234 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.083886  2234 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:55.085359  2234 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:55.086419  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:55.086414  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:55.087037  1541 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.087083  1692 raft_consensus.cc:3055] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:55.087458  1692 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:55.088080  1692 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:55.088920  1692 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.092074  1541 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:55.093186  2203 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:55.093838  2768 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:55.094326  2768 raft_consensus.cc:697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 12:59:55.094559  1692 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:55.095180  2768 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:55.103969  1438 catalog_manager.cc:5697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 2, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:55.120191  1692 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.121383  2678 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.130347  1541 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.131753  2773 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.198735  1692 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:55.200129  2768 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:55.201110  1541 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:55.202414  2768 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:55.242672  1692 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.243801  2757 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.252887  1617 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.254748  2651 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.364991  1541 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:55.366184  2806 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:55.376896  1690 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:55.378535  2697 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:55.531329  1537 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:55.530792  1543 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:55.532594  2697 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:55.539642  1618 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:55.543727  1690 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:59:55.544046  1508 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 474d3b57c33542aba459b97733122bf0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:55.544458  2758 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:55.545176  2757 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.544466  1692 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:55.546696  2757 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:55.549624  1618 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:55.556314   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:55.556656  1687 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:55.562278  2768 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:55.579650  2757 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.647311  2808 raft_consensus.cc:2955] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:55.650964  1690 raft_consensus.cc:2955] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:55.657377  1618 raft_consensus.cc:2955] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:55.660899  1615 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:55.668509  1440 catalog_manager.cc:5697] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.671994  1543 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:55.673904  2773 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:59:55.670768  1583 consensus_peers.cc:597] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b833a9bd5950418885dc620b52d4558f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:55.682559  1690 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:55.684056  2774 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:55.704017  2774 raft_consensus.cc:2955] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:55.705328  1692 raft_consensus.cc:2955] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:55.724522  1541 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:55.725270  1543 raft_consensus.cc:2955] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:55.744648  1438 catalog_manager.cc:5697] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.750952   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:55.756047  1692 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:59:55.757987  1508 consensus_peers.cc:597] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 13adc029302444abaaffd4bed11760a3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:55.758147  2808 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:55.759265  1615 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:55.760689  2808 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:55.774883  2758 raft_consensus.cc:2955] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:55.777510  1692 raft_consensus.cc:2955] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:55.783073  1615 raft_consensus.cc:2955] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:55.799291  1438 catalog_manager.cc:5697] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.804489  1541 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:55.818527  1618 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:55.819729  2757 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:55.820751  1687 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:59:55.822273  1508 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3ce476a5b0534bc5b6915606f6d37f30. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:55.823635  2757 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:55.827067  1543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:53670
I20260709 12:59:55.828905  1543 raft_consensus.cc:606] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:55.833585  1615 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:55.835361  1615 raft_consensus.cc:606] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:55.838269  1617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:55.838781  1617 raft_consensus.cc:606] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:55.841204  1541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:53670
I20260709 12:59:55.841748  1541 raft_consensus.cc:606] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:55.849321  1687 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:49352
I20260709 12:59:55.849992  1687 raft_consensus.cc:606] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:55.856226   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:59:55.857802   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:55.857963  2651 raft_consensus.cc:2955] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:55.863845  1618 raft_consensus.cc:2955] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:55.865154  1687 raft_consensus.cc:2955] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:55.881896  1437 catalog_manager.cc:5697] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:55.906157  2825 raft_consensus.cc:993] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:55.906622  2774 raft_consensus.cc:1081] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:55.908205  1541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 12:59:55.908665  1541 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:55.908953  1541 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.915835  1541 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:55.917399  1541 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:55.918195  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:55.918761  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:55.918793  1618 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:55.919382  1687 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.919327  1618 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:55.920372  1618 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:55.921655  1618 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:55.924141  1687 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:55.925222  1507 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:55.925925  2651 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:55.926326  2651 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:55.927079  2651 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:55.928411  1618 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:55.937667  1438 catalog_manager.cc:5697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:55.964449  2836 ts_tablet_manager.cc:933] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
I20260709 12:59:55.966504  2836 tablet_copy_client.cc:323] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:43459
I20260709 12:59:55.968577  1553 tablet_copy_service.cc:140] P 83f6f5810950417195f707ff308a8125: Received BeginTabletCopySession request for tablet 474d3b57c33542aba459b97733122bf0 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:33722)
I20260709 12:59:55.969132  1553 tablet_copy_service.cc:161] P 83f6f5810950417195f707ff308a8125: Beginning new tablet copy session on tablet 474d3b57c33542aba459b97733122bf0 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:33722: session id = cecad4f86a114a7696e398b3e8cc511e-474d3b57c33542aba459b97733122bf0
I20260709 12:59:56.042079  2758 raft_consensus.cc:993] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: : Instructing follower cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:56.043223  2758 raft_consensus.cc:1081] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Signalling peer cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:56.046317  1553 tablet_copy_source_session.cc:215] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:56.047523  2235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:39466
I20260709 12:59:56.048437  2651 raft_consensus.cc:993] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: : Instructing follower cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:56.048580  2235 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.049252  2651 raft_consensus.cc:1081] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Signalling peer cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:56.049263  2235 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.051507  2233 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:39466
I20260709 12:59:56.052145  2233 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.052562  2233 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.054418  2836 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 474d3b57c33542aba459b97733122bf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:56.069931  2233 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:56.076035  2233 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:56.076872  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:56.077775  1618 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.073828  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:56.089603  1541 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:56.090054  1541 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:56.091255  1541 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:56.093215  1541 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.099583  2773 raft_consensus.cc:993] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:56.100278  2773 raft_consensus.cc:1081] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:56.102211  2237 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:49846
I20260709 12:59:56.102699  2237 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.103021  2237 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.110409  2237 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:56.120247  2836 tablet_copy_client.cc:806] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:56.113792  1618 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:56.112140  1541 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:56.121757  2836 tablet_copy_client.cc:670] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:56.122146  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:56.122745  1617 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:56.123064  1617 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:56.123375  2237 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:56.123910  1617 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:56.124847  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:56.125635  1687 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.125902  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:56.127013  2768 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:56.127238  1617 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.127614  2768 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 12:59:56.128756  2768 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:56.115955  2235 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:56.135067  2836 tablet_copy_client.cc:538] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:56.140965  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:56.141603  1541 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:56.141902  1541 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:56.142695  1541 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:56.143951  1687 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:56.144014  1541 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.145938  2235 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:56.147361  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:56.148190  1690 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.161626  1541 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:56.162961  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:56.164209  2697 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:56.164892  2697 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 12:59:56.166258  2697 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:56.168232  2201 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:56.169173  2806 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:56.170282  1617 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:56.165742  2836 tablet_bootstrap.cc:492] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:56.186342  2806 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 12:59:56.187613  2806 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:56.207952  1690 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:56.220228  1438 catalog_manager.cc:5697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 2, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:56.240828  2820 raft_consensus.cc:993] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: : Instructing follower 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 12:59:56.241439  2820 raft_consensus.cc:1081] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Signalling peer 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 12:59:56.255409  1617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:43412
I20260709 12:59:56.256063  1617 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:56.256490  1617 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.282542  2845 ts_tablet_manager.cc:933] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
W20260709 12:59:56.295648  2840 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:59:56.303256  1617 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:56.311908  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 12:59:56.312659  2236 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.314488  1617 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 12:59:56.319557  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:56.320650  1690 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:56.321079  1690 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:56.366878  1690 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:56.368373  1690 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:56.365137  2845 tablet_copy_client.cc:323] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:43459
I20260709 12:59:56.362257  2236 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 12:59:56.380746  1690 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 12:59:56.382530  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:56.383701  2844 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:56.384208  2844 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:56.384490  2233 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 12:59:56.394613  1438 catalog_manager.cc:5697] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:56.385244  2844 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:56.396633  1438 catalog_manager.cc:5697] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 2, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:56.401342  1553 tablet_copy_service.cc:140] P 83f6f5810950417195f707ff308a8125: Received BeginTabletCopySession request for tablet 13adc029302444abaaffd4bed11760a3 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:33722)
I20260709 12:59:56.402081  1553 tablet_copy_service.cc:161] P 83f6f5810950417195f707ff308a8125: Beginning new tablet copy session on tablet 13adc029302444abaaffd4bed11760a3 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:33722: session id = cecad4f86a114a7696e398b3e8cc511e-13adc029302444abaaffd4bed11760a3
I20260709 12:59:56.409386  2848 ts_tablet_manager.cc:933] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
I20260709 12:59:56.412972  2846 ts_tablet_manager.cc:933] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:56.417716  2836 tablet_bootstrap.cc:492] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:56.426667  1553 tablet_copy_source_session.cc:215] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:56.432359  2846 tablet_copy_client.cc:323] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43887
I20260709 12:59:56.451730  2836 tablet_bootstrap.cc:492] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:56.452684  2845 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13adc029302444abaaffd4bed11760a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:56.452710  2836 ts_tablet_manager.cc:1403] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.287s	user 0.023s	sys 0.017s
I20260709 12:59:56.456497  2836 raft_consensus.cc:359] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:56.439122  2848 tablet_copy_client.cc:323] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:43459
I20260709 12:59:56.458732  1627 tablet_copy_service.cc:140] P 8a6a5ae1b7f240408b3ddd30d8a18262: Received BeginTabletCopySession request for tablet b833a9bd5950418885dc620b52d4558f from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:45356)
I20260709 12:59:56.459417  1627 tablet_copy_service.cc:161] P 8a6a5ae1b7f240408b3ddd30d8a18262: Beginning new tablet copy session on tablet b833a9bd5950418885dc620b52d4558f from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:45356: session id = cecad4f86a114a7696e398b3e8cc511e-b833a9bd5950418885dc620b52d4558f
I20260709 12:59:56.464978  2836 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:56.466817  1553 tablet_copy_service.cc:140] P 83f6f5810950417195f707ff308a8125: Received BeginTabletCopySession request for tablet 3ce476a5b0534bc5b6915606f6d37f30 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:33722)
I20260709 12:59:56.467582  1553 tablet_copy_service.cc:161] P 83f6f5810950417195f707ff308a8125: Beginning new tablet copy session on tablet 3ce476a5b0534bc5b6915606f6d37f30 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:33722: session id = cecad4f86a114a7696e398b3e8cc511e-3ce476a5b0534bc5b6915606f6d37f30
I20260709 12:59:56.467986  1627 tablet_copy_source_session.cc:215] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:56.471477  2846 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b833a9bd5950418885dc620b52d4558f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:56.476042  2836 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:56.485199  2836 ts_tablet_manager.cc:1434] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.032s	user 0.004s	sys 0.003s
I20260709 12:59:56.487493  1552 tablet_copy_service.cc:342] P 83f6f5810950417195f707ff308a8125: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-474d3b57c33542aba459b97733122bf0 received from {username='slave'} at 127.0.0.1:33722
I20260709 12:59:56.487869  1553 tablet_copy_source_session.cc:215] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:59:56.488067  1552 tablet_copy_service.cc:434] P 83f6f5810950417195f707ff308a8125: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-474d3b57c33542aba459b97733122bf0 on tablet 474d3b57c33542aba459b97733122bf0 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:56.489197  2845 tablet_copy_client.cc:806] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:56.489893  2845 tablet_copy_client.cc:670] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:56.494271  2848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ce476a5b0534bc5b6915606f6d37f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:59:56.493700  1438 catalog_manager.cc:5697] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 1 to 2, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:56.505945  2236 raft_consensus.cc:1217] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:59:56.511317  1687 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.512979  2808 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:56.519352  2845 tablet_copy_client.cc:538] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:56.523717  1613 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:56.525902  2758 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:56.535902  2846 tablet_copy_client.cc:806] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:56.536720  2846 tablet_copy_client.cc:670] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:56.538760  2845 tablet_bootstrap.cc:492] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:56.543030  2846 tablet_copy_client.cc:538] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:56.558910  2846 tablet_bootstrap.cc:492] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:56.583891  1541 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:56.587819  2848 tablet_copy_client.cc:806] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 12:59:56.588560  2848 tablet_copy_client.cc:670] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 12:59:56.602859  1617 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:56.604354  2697 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:56.622655  1537 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:56.622895  2806 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:56.638059  2806 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:56.655666  1613 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:56.660184  2848 tablet_copy_client.cc:538] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:59:56.680243  2845 tablet_bootstrap.cc:492] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:56.681205  2845 tablet_bootstrap.cc:492] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:56.688777  2697 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:56.681902  2845 ts_tablet_manager.cc:1403] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.143s	user 0.018s	sys 0.017s
W20260709 12:59:56.700867  2859 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.056s	user 0.003s	sys 0.000s
I20260709 12:59:56.717133  1690 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:56.714072  1687 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:56.723708  2842 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:56.729936  2846 tablet_bootstrap.cc:492] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:56.730926  2846 tablet_bootstrap.cc:492] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:56.731640  2846 ts_tablet_manager.cc:1403] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.173s	user 0.022s	sys 0.012s
I20260709 12:59:56.734618  2845 raft_consensus.cc:359] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:56.735638  2845 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:56.740865  2845 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:56.743494  2845 ts_tablet_manager.cc:1434] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.050s	user 0.005s	sys 0.000s
I20260709 12:59:56.752784  2846 raft_consensus.cc:359] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:56.753489  2846 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:56.754156  2846 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:56.758471  2846 ts_tablet_manager.cc:1434] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.026s	user 0.002s	sys 0.007s
I20260709 12:59:56.774613  2848 tablet_bootstrap.cc:492] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 12:59:56.775174  1553 tablet_copy_service.cc:342] P 83f6f5810950417195f707ff308a8125: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-13adc029302444abaaffd4bed11760a3 received from {username='slave'} at 127.0.0.1:33722
I20260709 12:59:56.775646  1553 tablet_copy_service.cc:434] P 83f6f5810950417195f707ff308a8125: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-13adc029302444abaaffd4bed11760a3 on tablet 13adc029302444abaaffd4bed11760a3 with peer cecad4f86a114a7696e398b3e8cc511e
W20260709 12:59:56.784057  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 61835.9us
I20260709 12:59:56.785214  2842 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:56.819423  2236 raft_consensus.cc:1217] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:59:56.819788  1627 tablet_copy_service.cc:342] P 8a6a5ae1b7f240408b3ddd30d8a18262: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-b833a9bd5950418885dc620b52d4558f received from {username='slave'} at 127.0.0.1:45356
I20260709 12:59:56.820248  1627 tablet_copy_service.cc:434] P 8a6a5ae1b7f240408b3ddd30d8a18262: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-b833a9bd5950418885dc620b52d4558f on tablet b833a9bd5950418885dc620b52d4558f with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:56.843021  2848 tablet_bootstrap.cc:492] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:59:56.843973  2848 tablet_bootstrap.cc:492] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 12:59:56.844652  2848 ts_tablet_manager.cc:1403] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.070s	user 0.020s	sys 0.004s
I20260709 12:59:56.849339  2848 raft_consensus.cc:359] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:56.850207  2848 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 12:59:56.851370  2848 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:56.855523  2848 ts_tablet_manager.cc:1434] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 12:59:56.860968  1553 tablet_copy_service.cc:342] P 83f6f5810950417195f707ff308a8125: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-3ce476a5b0534bc5b6915606f6d37f30 received from {username='slave'} at 127.0.0.1:33722
I20260709 12:59:56.861491  1553 tablet_copy_service.cc:434] P 83f6f5810950417195f707ff308a8125: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-3ce476a5b0534bc5b6915606f6d37f30 on tablet 3ce476a5b0534bc5b6915606f6d37f30 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:56.859033   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:56.929111  2834 raft_consensus.cc:1064] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:56.933009  2834 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:56.944670  2236 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:56.946343  2834 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:56.948403  1613 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:56.948967  1690 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:56.951182  2651 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:56.978268  1687 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:56.978937  2651 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:56.989714  2844 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:56.992842  2235 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:56.996515  2825 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:56.994197   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:57.001623  2235 raft_consensus.cc:2955] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:56.999974  2651 raft_consensus.cc:2955] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.016613  1688 raft_consensus.cc:2955] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.046651  1617 raft_consensus.cc:2955] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.065411  1437 catalog_manager.cc:5697] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:57.103317  1541 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:57.108635  2236 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:57.109606  1690 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:57.109934  2879 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:57.110982  2651 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:57.121320  2651 raft_consensus.cc:2955] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.122781  2236 raft_consensus.cc:2955] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.123904  1690 raft_consensus.cc:2955] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.149751  1440 catalog_manager.cc:5697] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 4 to 5, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.151569  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 474d3b57c33542aba459b97733122bf0 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:57.162261  1606 tablet_service.cc:1558] Processing DeleteTablet for tablet 474d3b57c33542aba459b97733122bf0 with delete_type TABLET_DATA_TOMBSTONED (TS 8a6a5ae1b7f240408b3ddd30d8a18262 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43382
I20260709 12:59:57.164008  1690 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:49352
I20260709 12:59:57.164685  1690 raft_consensus.cc:606] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:57.169343  1617 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:57.169942  1617 raft_consensus.cc:606] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:57.173117  1541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:53670
I20260709 12:59:57.173718  1541 raft_consensus.cc:606] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:57.183094  1690 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 12:59:57.183691  1690 raft_consensus.cc:606] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:57.185823   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:59:57.185307  2236 raft_consensus.cc:1217] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:59:57.187551   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:57.189723  2884 tablet_replica.cc:333] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 12:59:57.196132  2884 raft_consensus.cc:2243] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:57.197001  2884 raft_consensus.cc:2272] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:57.205575  2884 ts_tablet_manager.cc:1916] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:57.212354  2825 raft_consensus.cc:1064] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:57.215245  2825 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:57.224390  2884 ts_tablet_manager.cc:1929] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:59:57.225016  2884 log.cc:1199] T 474d3b57c33542aba459b97733122bf0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/wals/474d3b57c33542aba459b97733122bf0
I20260709 12:59:57.226750  1428 catalog_manager.cc:5028] TS 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): tablet 474d3b57c33542aba459b97733122bf0 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:57.239787  2236 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:57.242972  2774 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:57.243646  2820 raft_consensus.cc:993] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:57.244270  2764 raft_consensus.cc:1081] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:57.246950  1537 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54764
I20260709 12:59:57.247525  1537 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.247848  1537 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.255977  1537 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:57.259155  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:57.259447  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:57.259755  1617 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.259984  1690 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:57.262316  1690 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:57.263182  1690 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:57.264272  1690 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.268116  2235 raft_consensus.cc:1217] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:59:57.276388  1537 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:57.281556  1688 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:57.283442  1537 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:59:57.283567  2825 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:59:57.285540  2880 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:57.289930  1617 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:57.291245  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:57.294327  2651 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:57.294951  2651 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:57.297132  2651 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:57.295938  2880 raft_consensus.cc:993] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:57.297784  1690 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:57.308826  2880 raft_consensus.cc:1081] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:57.309782  1543 raft_consensus.cc:2955] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.311558  1537 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 12:59:57.312067  1537 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.312390  1537 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.302023  1688 raft_consensus.cc:2955] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.300314  2825 raft_consensus.cc:2955] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.319128  1537 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:57.321918  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:57.322481  1617 raft_consensus.cc:3055] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:57.322790  1617 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:57.323140  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:57.323722  1690 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.323557  1617 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:57.324586  1617 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.325201  1537 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:57.331626  1617 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:57.332715  1509 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 12:59:57.333528  2879 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:57.364534  2879 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:57.365595  2879 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:57.363235  1690 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:57.374449  1440 catalog_manager.cc:5697] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.367398  1437 catalog_manager.cc:5697] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.382673  2758 raft_consensus.cc:993] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: : Instructing follower 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 12:59:57.383355  2808 raft_consensus.cc:1081] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Signalling peer 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 12:59:57.392387  1613 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:43398
I20260709 12:59:57.392907  1613 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.393265  1613 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:57.393105  2236 raft_consensus.cc:2955] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.400059  1613 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:57.402060  1613 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:57.403316  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:57.403932  1688 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:57.405988  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:57.406582  1543 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:57.406878  1543 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:57.407641  1543 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:57.408852  1543 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:57.410674  1688 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 12:59:57.411782  1582 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:57.413043  2825 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:57.413823  2825 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:57.420405  1543 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 12:59:57.414934  2825 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:57.428366  1613 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:57.430132  1440 catalog_manager.cc:5697] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.436154  2236 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:57.436607  1688 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:59:57.437728  2774 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:57.438642  2844 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:57.446592  2880 raft_consensus.cc:2955] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.448747  1688 raft_consensus.cc:2955] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.462687  2236 raft_consensus.cc:2955] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.470007  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b833a9bd5950418885dc620b52d4558f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:57.477845  1440 catalog_manager.cc:5697] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.492272  1441 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 2 to 3, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.497359  1528 tablet_service.cc:1558] Processing DeleteTablet for tablet b833a9bd5950418885dc620b52d4558f with delete_type TABLET_DATA_TOMBSTONED (TS 83f6f5810950417195f707ff308a8125 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 12:59:57.512471  2894 tablet_replica.cc:333] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 12:59:57.513742  2894 raft_consensus.cc:2243] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:57.514483  2894 raft_consensus.cc:2272] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:57.517627  2894 ts_tablet_manager.cc:1916] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:57.535867  2894 ts_tablet_manager.cc:1929] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:59:57.536396  2894 log.cc:1199] T b833a9bd5950418885dc620b52d4558f P 83f6f5810950417195f707ff308a8125: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/wals/b833a9bd5950418885dc620b52d4558f
I20260709 12:59:57.540290  1428 catalog_manager.cc:5028] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): tablet b833a9bd5950418885dc620b52d4558f (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:57.781111  2651 raft_consensus.cc:1064] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:57.782730  2764 raft_consensus.cc:993] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: : Instructing follower 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 12:59:57.783278  2764 raft_consensus.cc:1081] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Signalling peer 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 12:59:57.783648  2651 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:57.791700  2879 raft_consensus.cc:1064] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 12:59:57.794322  2879 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:57.795533  2236 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:57.796543  1618 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:43412
I20260709 12:59:57.797143  1618 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:57.797509  1618 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.800169  2808 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:57.808030  1613 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:57.816509  1618 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:57.817790  2808 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:57.835433  1690 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:57.837170  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:57.837757  1688 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:57.837877  1618 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 12:59:57.838155  1688 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:57.839687  1688 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:57.840817  1688 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.839968  1687 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:57.842079  2808 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:57.842905  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 12:59:57.843464  2236 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:57.844563  1690 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:57.851028  1618 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:57.858217  2651 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:57.858873  2758 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:57.860212  2651 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:57.883628  2236 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 12:59:57.885141  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:57.886049  2844 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:57.886550  2844 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:57.887477  2844 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:57.890089  1618 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:57.893057  2651 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:57.857101  1688 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 12:59:57.913784  1617 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:57.942847  2651 raft_consensus.cc:2955] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.945468  1613 raft_consensus.cc:2955] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.921161  2808 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:59:57.979079  1692 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:57.980973  1440 catalog_manager.cc:5697] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:57.985531  2236 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:59:57.988324  2233 raft_consensus.cc:2955] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:57.988649  1688 raft_consensus.cc:2955] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.004097  1692 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:58.006227  2844 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:58.012024  2909 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:58.022795  2905 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:59:58.030294  1438 catalog_manager.cc:5697] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 1 to 2, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:58.043654  1543 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:58.059401  2235 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:58.061041  1543 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:58.062160  1613 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:58.063937  2844 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:58.065807  2758 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:58.066845  2909 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:58.093724  2651 raft_consensus.cc:2955] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.105504  1618 raft_consensus.cc:2955] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.099202  2808 raft_consensus.cc:2955] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.127251  2233 raft_consensus.cc:2955] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.133533  1614 raft_consensus.cc:2955] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.141782  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 13adc029302444abaaffd4bed11760a3 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:58.142971  1688 raft_consensus.cc:2955] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.147513  2235 raft_consensus.cc:2955] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.154578  1438 catalog_manager.cc:5697] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.156114  1441 catalog_manager.cc:5697] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index 4 to 5, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.179790  1679 tablet_service.cc:1558] Processing DeleteTablet for tablet 13adc029302444abaaffd4bed11760a3 with delete_type TABLET_DATA_TOMBSTONED (TS 984c9089d9804652846f3caeb609d56e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:40104
I20260709 12:59:58.188817   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:58.234373  2916 tablet_replica.cc:333] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 12:59:58.235507  2916 raft_consensus.cc:2243] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:58.236244  2916 raft_consensus.cc:2272] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:58.242542  2916 ts_tablet_manager.cc:1916] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:58.264964  2916 ts_tablet_manager.cc:1929] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:59:58.265511  2916 log.cc:1199] T 13adc029302444abaaffd4bed11760a3 P 984c9089d9804652846f3caeb609d56e: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/wals/13adc029302444abaaffd4bed11760a3
I20260709 12:59:58.267325  1426 catalog_manager.cc:5028] TS 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): tablet 13adc029302444abaaffd4bed11760a3 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:58.271209  1543 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:58.280135  2235 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:58.280771  1687 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:58.282207  2758 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:58.283080  2651 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:58.290364  2758 raft_consensus.cc:2955] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.292816  2235 raft_consensus.cc:2955] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.294819  1687 raft_consensus.cc:2955] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.302740  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3ce476a5b0534bc5b6915606f6d37f30 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:59:58.313006   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:58.315380  1438 catalog_manager.cc:5697] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 4 to 5, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 12:59:58.326442  1606 tablet_service.cc:1558] Processing DeleteTablet for tablet 3ce476a5b0534bc5b6915606f6d37f30 with delete_type TABLET_DATA_TOMBSTONED (TS 8a6a5ae1b7f240408b3ddd30d8a18262 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:43382
I20260709 12:59:58.333768  2917 tablet_replica.cc:333] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 12:59:58.334857  2917 raft_consensus.cc:2243] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:59:58.335647  2917 raft_consensus.cc:2272] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:59:58.339668  2917 ts_tablet_manager.cc:1916] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:59:58.352389  2917 ts_tablet_manager.cc:1929] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:59:58.352870  2917 log.cc:1199] T 3ce476a5b0534bc5b6915606f6d37f30 P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/wals/3ce476a5b0534bc5b6915606f6d37f30
I20260709 12:59:58.354465  1428 catalog_manager.cc:5028] TS 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): tablet 3ce476a5b0534bc5b6915606f6d37f30 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 12:59:58.362118  1618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12adfb7855f340329f110249c207f409"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:58.362639  1618 raft_consensus.cc:606] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:58.364853  1543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:53670
I20260709 12:59:58.365322  1543 raft_consensus.cc:606] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 12:59:58.368347  1542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:53670
I20260709 12:59:58.368875  1542 raft_consensus.cc:606] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 12:59:58.371059  1618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 12:59:58.371598  1618 raft_consensus.cc:606] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:58.372555   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:59:58.373642   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:58.406406  2235 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:58.408697  2844 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:59:58.421765  1688 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:58.427922  2774 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:58.497063  2774 raft_consensus.cc:993] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:58.497555  2844 raft_consensus.cc:1081] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:58.499050  1542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 12:59:58.499469  1542 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:58.499701  1542 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:58.504814  1542 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:58.506584  1542 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:58.507381  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:58.507577  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:58.507997  1618 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:58.508044  1688 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:58.508407  1618 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:58.509209  1618 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:58.510408  1618 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:58.514891  1688 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:58.516220  1507 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:58.517073  2758 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:58.517869  1618 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:58.518999  2758 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:58.520048  2758 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:58.536755  1440 catalog_manager.cc:5697] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:58.645283  2844 raft_consensus.cc:993] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:58.645888  2893 raft_consensus.cc:1081] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:58.647490  1543 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "12adfb7855f340329f110249c207f409"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 12:59:58.648064  1543 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:58.648392  1543 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:58.657267  1543 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:58.659616  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:58.660207  1614 raft_consensus.cc:3055] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:58.660508  1614 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:58.660835  1543 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 12:59:58.661473  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 12:59:58.661334  1614 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:58.662056  2235 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:58.662326  2651 raft_consensus.cc:993] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: : Instructing follower 984c9089d9804652846f3caeb609d56e to start an election
I20260709 12:59:58.662544  1614 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:58.662830  2808 raft_consensus.cc:1081] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Signalling peer 984c9089d9804652846f3caeb609d56e to start an election
I20260709 12:59:58.669157  2235 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:58.669147  1614 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 12:59:58.669927  1688 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40124
I20260709 12:59:58.670557  1508 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:58.671124  1688 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:58.671583  1688 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:58.672312  2758 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:58.672799  2758 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:58.673619  2758 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:58.678400  1688 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:58.680413  1688 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 12:59:58.681140  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:58.681813  1542 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:58.682189  1542 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:58.682717  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:58.682988  1542 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:58.683557  1614 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:58.684087  1542 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:58.689152  1440 catalog_manager.cc:5697] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:58.690263  1542 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 12:59:58.691566  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:58.692366  2764 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:58.692857  2764 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:58.693687  2764 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:58.698874  1614 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 12:59:58.703713  2808 raft_consensus.cc:993] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: : Instructing follower 984c9089d9804652846f3caeb609d56e to start an election
I20260709 12:59:58.704224  2758 raft_consensus.cc:1081] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Signalling peer 984c9089d9804652846f3caeb609d56e to start an election
I20260709 12:59:58.705822  1688 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40124
I20260709 12:59:58.706338  1688 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:58.706651  1688 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:58.710520  1438 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 3, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 3 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 12:59:58.713219  1688 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:58.715682  1688 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 12:59:58.715901  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:58.716468  1542 raft_consensus.cc:3055] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:58.716514  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 12:59:58.716825  1542 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:58.717123  2235 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:58.717504  1542 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:58.718581  1542 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:58.722887  2235 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 12:59:58.723282  1542 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 12:59:58.724030  1657 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:58.724807  2923 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:58.725267  2923 raft_consensus.cc:697] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:58.726069  2923 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:58.736299  1438 catalog_manager.cc:5697] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 3, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 3 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:58.947544  1614 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:58.948642  2758 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:58.960328  1688 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:59:58.962402  2808 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:59.114324  1614 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:59.115440  2839 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:59.125056  1542 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:59:59.127862  2764 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:59.143405  1618 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:59.146821  2909 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:59.162218  2235 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:59:59.163527  2651 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:59:59.175932  2233 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:59:59.179384  2764 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:59:59.199358  1542 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:59:59.201613  2839 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:59:59.374439   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:59.398589  1614 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
W20260709 12:59:59.408308  1583 consensus_peers.cc:597] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2f07ede48ffe4a5eb0b162da083502c2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:59.409576  1542 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:59:59.411691  1688 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:59:59.413169  2893 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:59.414081  2774 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:59.425669  2844 raft_consensus.cc:2955] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:59.426770  1687 raft_consensus.cc:2955] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:59.428632  1542 raft_consensus.cc:2955] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:59.445166  1687 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:59.448321  1438 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 4, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 3 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
W20260709 12:59:59.459893  1657 consensus_peers.cc:597] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 532fefba367345759887da9fcc9872e3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:59.460690  1543 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:59.463335   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:59:59.463634  2820 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:59:59.464956  1614 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:59:59.467314  2820 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:59:59.490731  2839 raft_consensus.cc:2955] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:59.491933  1543 raft_consensus.cc:2955] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:59.503700  1614 raft_consensus.cc:2955] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:59.515632  1441 catalog_manager.cc:5697] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:59.533304  1687 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:59.551452  1543 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22357a3949344a1792e5bae4a0291987"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:53670
I20260709 12:59:59.552122  1543 raft_consensus.cc:606] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:59.557596  1542 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:59:59.560431  1688 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 12:59:59.561039  1688 raft_consensus.cc:606] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 12:59:59.560083  1614 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 12:59:59.562366  1657 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 674516b0482b47aaa9380469245f9d2e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:59.563683  2764 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:59:59.564872  1542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:53670
I20260709 12:59:59.565255  2764 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:59:59.565536  1542 raft_consensus.cc:606] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 12:59:59.572475  1687 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:49352
I20260709 12:59:59.573143  1687 raft_consensus.cc:606] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 12:59:59.572896  2839 raft_consensus.cc:2955] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:59.574846   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:59:59.576458   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 12:59:59.577183  1613 raft_consensus.cc:2955] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:59.580714  1542 raft_consensus.cc:2955] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:59.589586  1543 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 12:59:59.595647  1441 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 4, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 3 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:59:59.603020  1508 consensus_peers.cc:597] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 88cbab659ea041148748114fae549ae0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:59:59.605777  1613 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:59.605744  1687 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:59:59.607252  2651 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:59.608712  2758 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:59:59.621330  2758 raft_consensus.cc:2955] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:59.630134  1688 raft_consensus.cc:2955] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:59.639072  1613 raft_consensus.cc:2955] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 12:59:59.652338  1437 catalog_manager.cc:5697] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:59.801255  2651 raft_consensus.cc:993] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: : Instructing follower 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 12:59:59.801843  2651 raft_consensus.cc:1081] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Signalling peer 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 12:59:59.804117  1613 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22357a3949344a1792e5bae4a0291987"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:43398
I20260709 12:59:59.804625  1613 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:59.805037  1613 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.810262  1613 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:59.811932  1613 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:59.812999  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:59.813056  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:59.813751  1543 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:59:59.814093  1689 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.814119  1543 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:59.821782  1543 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:59.823236  1543 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:59:59.823487  1689 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 12:59:59.825124  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 12:59:59.826385  2844 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:59:59.826958  2844 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:59.827843  2844 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:59.844404  1543 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 12:59:59.852454  1438 catalog_manager.cc:5697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 2 to 3, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:59.854640  2909 raft_consensus.cc:993] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: : Instructing follower cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:59.855262  2909 raft_consensus.cc:1081] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Signalling peer cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 12:59:59.858501  2235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:39466
I20260709 12:59:59.860256  2235 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:59.861552  2235 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:59.873340  2235 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:59.878053  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:59.878906  1689 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:59.878353  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 12:59:59.881521  1543 raft_consensus.cc:3055] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:59.882767  1543 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 12:59:59.883762  1543 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:59.885485  1543 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:59.890192  2235 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:59.893098  1543 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:59.894147  2203 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:59.894055  1689 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 12:59:59.895015  2806 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:59.895578  2806 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 12:59:59.896663  2806 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:59.909911  2839 raft_consensus.cc:993] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: : Instructing follower 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 12:59:59.910420  2839 raft_consensus.cc:1081] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Signalling peer 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 12:59:59.912215  1613 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:43412
I20260709 12:59:59.912746  1613 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:59.913080  1613 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:59.925820  1438 catalog_manager.cc:5697] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 2, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:59:59.931553  1613 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:59.934262  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:59.934835  1688 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:59.935122  1688 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:59.935848  1688 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:59.936887  1688 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:59.936897  1613 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 12:59:59.942139  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 12:59:59.942704  2235 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:59.948477  2235 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 12:59:59.953886  1688 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 12:59:59.954421  1583 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 12:59:59.955183  2844 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:59:59.955646  2844 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 12:59:59.956375  2844 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 12:59:59.969862  2764 raft_consensus.cc:993] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:59.970319  2764 raft_consensus.cc:1081] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 12:59:59.971935  1542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54764
I20260709 12:59:59.972482  1542 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:59:59.972936  1542 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:59.971954  1438 catalog_manager.cc:5697] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 1 to 2, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:59:59.983506  1542 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:59.986457  1542 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 12:59:59.987706  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 12:59:59.987669  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 12:59:59.988442  1614 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:59:59.988511  1688 raft_consensus.cc:3055] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:59:59.988905  1688 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 12:59:59.989776  1688 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 12:59:59.991147  1688 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.002069  1688 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:00.003051  1507 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:00.003935  2955 ts_tablet_manager.cc:933] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
I20260709 13:00:00.003911  2651 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:00.004594  2651 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:00.007083  2651 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:00.019502  2957 ts_tablet_manager.cc:933] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:00.030721  1614 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:00.036953  2957 tablet_copy_client.cc:323] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43887
I20260709 13:00:00.038198  2954 ts_tablet_manager.cc:933] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:00.039906  1627 tablet_copy_service.cc:140] P 8a6a5ae1b7f240408b3ddd30d8a18262: Received BeginTabletCopySession request for tablet 2f07ede48ffe4a5eb0b162da083502c2 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:45356)
I20260709 13:00:00.040498  1627 tablet_copy_service.cc:161] P 8a6a5ae1b7f240408b3ddd30d8a18262: Beginning new tablet copy session on tablet 2f07ede48ffe4a5eb0b162da083502c2 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:45356: session id = cecad4f86a114a7696e398b3e8cc511e-2f07ede48ffe4a5eb0b162da083502c2
I20260709 13:00:00.041672  2953 ts_tablet_manager.cc:933] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:00.044641  2954 tablet_copy_client.cc:323] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 13:00:00.046845  2953 tablet_copy_client.cc:323] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 13:00:00.047039  1701 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet 532fefba367345759887da9fcc9872e3 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 13:00:00.047686  1701 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet 532fefba367345759887da9fcc9872e3 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-532fefba367345759887da9fcc9872e3
I20260709 13:00:00.049189  1627 tablet_copy_source_session.cc:215] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:00.049855  1702 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet 674516b0482b47aaa9380469245f9d2e from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 13:00:00.050546  1702 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet 674516b0482b47aaa9380469245f9d2e from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-674516b0482b47aaa9380469245f9d2e
I20260709 13:00:00.056682  1701 tablet_copy_source_session.cc:215] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:00.056012  2957 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f07ede48ffe4a5eb0b162da083502c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:00.060456  1438 catalog_manager.cc:5697] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:00.062160  2954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 532fefba367345759887da9fcc9872e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:00.069880  2955 tablet_copy_client.cc:323] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:43459
I20260709 13:00:00.071758  1702 tablet_copy_source_session.cc:215] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:00.079126  1553 tablet_copy_service.cc:140] P 83f6f5810950417195f707ff308a8125: Received BeginTabletCopySession request for tablet 88cbab659ea041148748114fae549ae0 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:33722)
I20260709 13:00:00.079717  1553 tablet_copy_service.cc:161] P 83f6f5810950417195f707ff308a8125: Beginning new tablet copy session on tablet 88cbab659ea041148748114fae549ae0 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:33722: session id = cecad4f86a114a7696e398b3e8cc511e-88cbab659ea041148748114fae549ae0
I20260709 13:00:00.085393  2954 tablet_copy_client.cc:806] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:00.085829  2953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 674516b0482b47aaa9380469245f9d2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:00.086284  2954 tablet_copy_client.cc:670] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:00.094964  1553 tablet_copy_source_session.cc:215] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:00.095757  2957 tablet_copy_client.cc:806] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:00.096441  2957 tablet_copy_client.cc:670] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:00.099881  2955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88cbab659ea041148748114fae549ae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:00.101987  2954 tablet_copy_client.cc:538] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:00.109964  2957 tablet_copy_client.cc:538] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:00.120153  2954 tablet_bootstrap.cc:492] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 13:00:00.124898  2957 tablet_bootstrap.cc:492] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 13:00:00.134528  2955 tablet_copy_client.cc:806] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:00.135190  2955 tablet_copy_client.cc:670] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:00.141209  2955 tablet_copy_client.cc:538] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:00.141896  2953 tablet_copy_client.cc:806] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:00.142486  2953 tablet_copy_client.cc:670] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:00.147400  2953 tablet_copy_client.cc:538] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:00.156985  2955 tablet_bootstrap.cc:492] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 13:00:00.162264  2953 tablet_bootstrap.cc:492] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 13:00:00.188915  2954 tablet_bootstrap.cc:492] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:00.193928  2954 tablet_bootstrap.cc:492] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 13:00:00.194061  2957 tablet_bootstrap.cc:492] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:00.194648  2954 ts_tablet_manager.cc:1403] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.075s	user 0.023s	sys 0.006s
I20260709 13:00:00.197197  2957 tablet_bootstrap.cc:492] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 13:00:00.198010  2957 ts_tablet_manager.cc:1403] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.073s	user 0.035s	sys 0.015s
I20260709 13:00:00.207299  2954 raft_consensus.cc:359] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:00.208441  2954 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 13:00:00.209462  2954 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:00.214228  2954 ts_tablet_manager.cc:1434] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.001s
I20260709 13:00:00.216001  1702 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-532fefba367345759887da9fcc9872e3 received from {username='slave'} at 127.0.0.1:40014
I20260709 13:00:00.216455  1702 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-532fefba367345759887da9fcc9872e3 on tablet 532fefba367345759887da9fcc9872e3 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:00.227221  2953 tablet_bootstrap.cc:492] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:00.228251  2953 tablet_bootstrap.cc:492] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 13:00:00.228960  2953 ts_tablet_manager.cc:1403] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.067s	user 0.017s	sys 0.009s
I20260709 13:00:00.231371  2955 tablet_bootstrap.cc:492] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:00.232563  2955 tablet_bootstrap.cc:492] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 13:00:00.231890  2953 raft_consensus.cc:359] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:00.233911  2953 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 13:00:00.233305  2955 ts_tablet_manager.cc:1403] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.077s	user 0.019s	sys 0.009s
I20260709 13:00:00.234583  2953 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:00.236609  2953 ts_tablet_manager.cc:1434] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.007s
I20260709 13:00:00.238353  1702 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-674516b0482b47aaa9380469245f9d2e received from {username='slave'} at 127.0.0.1:40014
I20260709 13:00:00.238859  1702 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-674516b0482b47aaa9380469245f9d2e on tablet 674516b0482b47aaa9380469245f9d2e with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:00.238157  2955 raft_consensus.cc:359] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:00.239341  2955 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 13:00:00.240000  2955 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:00.245193  2955 ts_tablet_manager.cc:1434] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260709 13:00:00.250504  1553 tablet_copy_service.cc:342] P 83f6f5810950417195f707ff308a8125: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-88cbab659ea041148748114fae549ae0 received from {username='slave'} at 127.0.0.1:33722
I20260709 13:00:00.250968  1553 tablet_copy_service.cc:434] P 83f6f5810950417195f707ff308a8125: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-88cbab659ea041148748114fae549ae0 on tablet 88cbab659ea041148748114fae549ae0 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:00.263015  1542 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:00.264264  2844 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:00.206672  2957 raft_consensus.cc:359] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:00.271335  2957 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 13:00:00.271960  2957 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:00.294589  2957 ts_tablet_manager.cc:1434] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 13:00:00.296387  1627 tablet_copy_service.cc:342] P 8a6a5ae1b7f240408b3ddd30d8a18262: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-2f07ede48ffe4a5eb0b162da083502c2 received from {username='slave'} at 127.0.0.1:45356
I20260709 13:00:00.296852  1627 tablet_copy_service.cc:434] P 8a6a5ae1b7f240408b3ddd30d8a18262: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-2f07ede48ffe4a5eb0b162da083502c2 on tablet 2f07ede48ffe4a5eb0b162da083502c2 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:00.303267  1687 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:00.306291  2950 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:00.313474  1542 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:00.314823  2933 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:00.410775  1613 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:00.411424  1689 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:00.412729  2758 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:00.415974  2967 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:00.431197  1689 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:00.434302  2909 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:00.473560  2235 raft_consensus.cc:1217] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
W20260709 13:00:00.475190  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 112046us
I20260709 13:00:00.492049  2233 raft_consensus.cc:1217] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 13:00:00.535786  1689 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:00.536571  2233 raft_consensus.cc:1217] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:00:00.539243  2977 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
W20260709 13:00:00.527962  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 50319.1us
I20260709 13:00:00.576399  2235 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:00.624500  2975 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:00:00.634284   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:00.709411  2237 raft_consensus.cc:1217] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:00.831488   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:00.880084  1613 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:00:00.880666  1613 raft_consensus.cc:606] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:00:00.883359  1541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:53670
I20260709 13:00:00.883948  1541 raft_consensus.cc:606] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:00:00.887071  1613 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:00:00.887665  1613 raft_consensus.cc:606] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:00:00.888931   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:00:00.890746   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:00.917011  2909 raft_consensus.cc:993] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: : Instructing follower 984c9089d9804652846f3caeb609d56e to start an election
I20260709 13:00:00.917729  2909 raft_consensus.cc:1081] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Signalling peer 984c9089d9804652846f3caeb609d56e to start an election
I20260709 13:00:00.920671  1687 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40124
I20260709 13:00:00.921257  1687 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:00.921619  1687 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.926141  1687 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:00.927944  1687 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:00.929458  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:00.929690  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:00.930029  1542 raft_consensus.cc:3055] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:00.930303  1613 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.930364  1542 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:00.932786  1542 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:00.934121  1542 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:00.936259  1613 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:00.937834  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:00.938656  2764 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:00.939109  2764 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:00.939822  2764 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:00.940657  1542 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:00.950814  1441 catalog_manager.cc:5697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 3, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 3 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:00.963109  2774 raft_consensus.cc:993] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:00:00.963716  2774 raft_consensus.cc:1081] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:00:00.966311  1542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 13:00:00.966866  1542 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:00.967196  1542 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.976559  2758 raft_consensus.cc:1064] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 13:00:00.979004  1542 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:00.979004  2758 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:00.982100  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:00.982686  1687 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.983598  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:00.984153  1613 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:00.984443  1613 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:00.985241  1613 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:00.986730  1613 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:00.993307  1687 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:00.993938  1542 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:00.993384  1613 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:00.995549  1507 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:00.998093  2651 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:01.017910  2651 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:01.019326  2651 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:01.033228  2235 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:01.036990  2909 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:01.041116  1687 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:01.043557  2909 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:01.051203  2764 raft_consensus.cc:1064] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 13:00:01.056900  2764 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:01.059917  1441 catalog_manager.cc:5697] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:01.082118  1618 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:01.088652  2233 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 3 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:01.089249  1618 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:01.091893  2994 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:01.092171  2985 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:01.105266  1542 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:01.113003  2982 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:01.089556  2995 raft_consensus.cc:2955] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.110963  1688 raft_consensus.cc:2955] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.118724  2774 raft_consensus.cc:993] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 984c9089d9804652846f3caeb609d56e to start an election
I20260709 13:00:01.118561  2985 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:01.119237  2774 raft_consensus.cc:1081] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 984c9089d9804652846f3caeb609d56e to start an election
I20260709 13:00:01.122601  1687 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40114
I20260709 13:00:01.123224  1687 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:01.125844  1687 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.143864  2976 raft_consensus.cc:1064] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 13:00:01.147106  2976 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:01.188843  1687 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:01.195432  1687 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:01.197598  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:01.202373  2236 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.209625  2236 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:01.210179  2235 raft_consensus.cc:2955] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.213685  1687 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:01.215473  2975 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:01.218072  2236 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEARNER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:01.222254  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:01.223198  2985 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:01.223610  2985 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:01.224284  2985 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:01.227190  2975 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:01.226641  1441 catalog_manager.cc:5697] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:01.234928  2981 raft_consensus.cc:1064] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 13:00:01.237509  2981 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:01.257588  1440 catalog_manager.cc:5697] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 2 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:01.258934  2982 raft_consensus.cc:2955] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:00:01.265695  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 60922.4us
I20260709 13:00:01.279372  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:01.280221  1613 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:01.280524  1613 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:01.280889  1541 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:01.267129  1618 raft_consensus.cc:2955] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.281241  1613 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:01.282441  1613 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:01.289310  2975 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:01.293008  1613 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:01.300433  2774 raft_consensus.cc:2955] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.306125  1688 raft_consensus.cc:2955] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.311578  1614 raft_consensus.cc:2955] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.314890  2235 raft_consensus.cc:2955] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.328315  2236 raft_consensus.cc:2955] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.338066  1440 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 4 to 5, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:00:01.356667  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46330us
I20260709 13:00:01.358680  1441 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 4 to 5, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.361325  2236 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:01.372326  3017 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:01.379665  1541 raft_consensus.cc:2955] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.381541  1537 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:01.389235  1614 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:01.387854  3011 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:00:01.408967  3016 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:01.422345  1618 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:01.423417  1541 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:01.426331  3011 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:01.429023  3011 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:01.440083  1688 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:01.450349  2981 raft_consensus.cc:2955] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.454958  2236 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:01.457106  3016 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:01.457003  1613 raft_consensus.cc:2955] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.463394  1541 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:01.462981  1542 raft_consensus.cc:2955] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.465570  3018 raft_consensus.cc:2955] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.466161  1538 raft_consensus.cc:2955] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.470201  3011 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:01.470141  2236 raft_consensus.cc:2955] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.477471  1541 raft_consensus.cc:2955] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.481881  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 674516b0482b47aaa9380469245f9d2e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:01.492057  1438 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 5 to 6, VOTER 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) evicted. New cstate: current_term: 3 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:01.525309  1437 catalog_manager.cc:5697] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.554672  1606 tablet_service.cc:1558] Processing DeleteTablet for tablet 674516b0482b47aaa9380469245f9d2e with delete_type TABLET_DATA_TOMBSTONED (TS 8a6a5ae1b7f240408b3ddd30d8a18262 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:43382
I20260709 13:00:01.566979  1618 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:01.578305  2235 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:01.581738  2774 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:01.586347  3027 tablet_replica.cc:333] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:00:01.587561  3027 raft_consensus.cc:2243] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:01.588102  1541 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:01.588570  3027 raft_consensus.cc:2272] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:01.589769  2977 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:01.596447  2774 raft_consensus.cc:2955] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.599988  1541 raft_consensus.cc:2955] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.610924  2235 raft_consensus.cc:2955] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.620546  2236 raft_consensus.cc:2955] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.633354  3027 ts_tablet_manager.cc:1916] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:01.642148  1537 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:01.644868  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2f07ede48ffe4a5eb0b162da083502c2 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:01.648274  1686 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:01.649717  2995 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:01.652661  1686 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:01.659013  2233 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:01.661162  2981 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:00:01.662549  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 52652.8us
I20260709 13:00:01.668721  1437 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index 5 to 6, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New cstate: current_term: 3 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:01.671790  2235 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:01.682601  1618 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:01.687157  1618 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:01.674221  2994 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:00:01.688625  3017 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:01.693815  2233 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:01.698400  2764 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:01.703177  1679 tablet_service.cc:1558] Processing DeleteTablet for tablet 2f07ede48ffe4a5eb0b162da083502c2 with delete_type TABLET_DATA_TOMBSTONED (TS 984c9089d9804652846f3caeb609d56e not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:40104
I20260709 13:00:01.710491  2235 raft_consensus.cc:2955] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.712322  1618 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:01.717700  2758 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:01.722965  3016 raft_consensus.cc:2955] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.736488  2758 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:01.737255  1613 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:01.749513  3011 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:00:01.766706  2995 raft_consensus.cc:2955] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.773940  1614 raft_consensus.cc:2955] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.784765  1617 raft_consensus.cc:2955] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:01.787690  3030 tablet_replica.cc:333] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:00:01.788853  3030 raft_consensus.cc:2243] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:01.789811  3030 raft_consensus.cc:2272] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:01.775234  2237 raft_consensus.cc:2955] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:00:01.796792  3029 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 13:00:01.822613  3027 ts_tablet_manager.cc:1929] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:00:01.824707  3027 log.cc:1199] T 674516b0482b47aaa9380469245f9d2e P 8a6a5ae1b7f240408b3ddd30d8a18262: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-1-root/wals/674516b0482b47aaa9380469245f9d2e
I20260709 13:00:01.827337  1426 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 532fefba367345759887da9fcc9872e3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:01.832100  1428 catalog_manager.cc:5028] TS 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): tablet 674516b0482b47aaa9380469245f9d2e (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 13:00:01.838786  1437 catalog_manager.cc:5697] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:01.859216  3030 ts_tablet_manager.cc:1916] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:01.865001  1528 tablet_service.cc:1558] Processing DeleteTablet for tablet 532fefba367345759887da9fcc9872e3 with delete_type TABLET_DATA_TOMBSTONED (TS 83f6f5810950417195f707ff308a8125 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
I20260709 13:00:01.891851   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:01.895082  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 88cbab659ea041148748114fae549ae0 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:01.927690  1437 catalog_manager.cc:5697] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 4 to 5, VOTER 984c9089d9804652846f3caeb609d56e (127.0.107.195) evicted. New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:01.928805  3030 ts_tablet_manager.cc:1929] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 13:00:01.929545  3030 log.cc:1199] T 2f07ede48ffe4a5eb0b162da083502c2 P 984c9089d9804652846f3caeb609d56e: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/wals/2f07ede48ffe4a5eb0b162da083502c2
I20260709 13:00:01.935514  3037 tablet_replica.cc:333] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:00:01.936546  3037 raft_consensus.cc:2243] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:01.937355  3037 raft_consensus.cc:2272] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:01.937984  1426 catalog_manager.cc:5028] TS 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): tablet 2f07ede48ffe4a5eb0b162da083502c2 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 13:00:01.977142  3037 ts_tablet_manager.cc:1916] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:01.983772  1679 tablet_service.cc:1558] Processing DeleteTablet for tablet 88cbab659ea041148748114fae549ae0 with delete_type TABLET_DATA_TOMBSTONED (TS 984c9089d9804652846f3caeb609d56e not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:40104
I20260709 13:00:01.984762  3030 tablet_replica.cc:333] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:00:01.985809  3030 raft_consensus.cc:2243] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:01.986593  3030 raft_consensus.cc:2272] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:01.990201  3030 ts_tablet_manager.cc:1916] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:00:02.010192  3037 ts_tablet_manager.cc:1929] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:02.010681  3037 log.cc:1199] T 532fefba367345759887da9fcc9872e3 P 83f6f5810950417195f707ff308a8125: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/wals/532fefba367345759887da9fcc9872e3
I20260709 13:00:02.012341  1428 catalog_manager.cc:5028] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): tablet 532fefba367345759887da9fcc9872e3 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 13:00:02.017513  3030 ts_tablet_manager.cc:1929] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:00:02.017899  3030 log.cc:1199] T 88cbab659ea041148748114fae549ae0 P 984c9089d9804652846f3caeb609d56e: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-2-root/wals/88cbab659ea041148748114fae549ae0
I20260709 13:00:02.019143  1426 catalog_manager.cc:5028] TS 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): tablet 88cbab659ea041148748114fae549ae0 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 13:00:02.145292   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:02.182945  1537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:53670
I20260709 13:00:02.183470  1537 raft_consensus.cc:606] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:02.185837  1688 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:00:02.186322  1688 raft_consensus.cc:606] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:00:02.188839  1688 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:00:02.189394  1688 raft_consensus.cc:606] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:02.190450   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:00:02.191738   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:02.404829  3014 raft_consensus.cc:993] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:00:02.405368  3014 raft_consensus.cc:1081] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:00:02.406884  1537 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54764
I20260709 13:00:02.407354  1537 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:02.407657  1537 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.412371  1537 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:02.414225  1537 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:02.415292  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:02.415336  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:02.416006  1688 raft_consensus.cc:3055] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:02.416113  2236 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.416354  1688 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:02.417171  1688 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:02.418404  1688 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.423455  2236 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:02.426581  1508 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:02.427547  2994 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:02.429719  1688 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:02.438905  2994 raft_consensus.cc:697] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:02.439901  2994 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:02.450834  1437 catalog_manager.cc:5697] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:02.479322  3017 raft_consensus.cc:993] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: : Instructing follower 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 13:00:02.479862  3014 raft_consensus.cc:1081] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Signalling peer 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 13:00:02.481361  1614 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:43412
I20260709 13:00:02.481938  1614 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:02.482283  1614 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.487846  1614 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:02.490859  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:02.491564  1688 raft_consensus.cc:3055] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:02.491921  1688 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:02.492746  1688 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:02.493954  1688 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.490792  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:02.494758  2236 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.490239  1614 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:02.500798  1688 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:02.500814  2236 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:02.502147  1583 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:02.503173  2975 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:02.503598  2975 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:02.504437  2975 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:02.514282  1437 catalog_manager.cc:5697] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 1 to 2, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:02.570497  2994 raft_consensus.cc:993] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: : Instructing follower cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 13:00:02.570935  3039 raft_consensus.cc:1081] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Signalling peer cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 13:00:02.572520  2236 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:39466
I20260709 13:00:02.573051  2236 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:02.573354  2236 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.579370  2236 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:02.581144  2236 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:02.582336  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:02.582983  1537 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:02.582922  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:02.583467  1537 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:02.583660  1613 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.584282  1537 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:02.585458  1537 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:02.590502  1613 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:02.591699  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:02.593273  3032 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:02.593794  3032 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:02.594708  3032 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:02.592119  1537 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:02.606948  1437 catalog_manager.cc:5697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 2, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 2 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:02.943084  1688 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:02.944473  2994 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:02.954746  2237 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:02.956410  2236 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:02.957996  2975 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:02.959530  3039 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:02.981593  1613 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:02.984351  3032 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:03.026582  1537 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:03.028558  3054 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:03.038820  1688 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:03.052621  3050 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:03.125993  1614 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
W20260709 13:00:03.132092  1583 consensus_peers.cc:597] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c65847e774e445909d83c69bf486b7fc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:03.132634  1537 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:00:03.134248  1688 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:00:03.139638  2975 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:00:03.144524  2975 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:03.151412  2975 raft_consensus.cc:2955] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:00:03.155889  1688 raft_consensus.cc:2955] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:00:03.158072  1537 raft_consensus.cc:2955] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:00:03.167816  1614 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:03.173115  1440 catalog_manager.cc:5697] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 3, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 2 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:03.189487  1688 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:00:03.190789  1583 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: caae9b2a426d42fb8c2353e1c978d0fb. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:03.191143  3058 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:03.191903  1537 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:03.192565   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:03.193609  2774 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:03.230916  3058 raft_consensus.cc:2955] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:00:03.239358  1541 raft_consensus.cc:2955] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:00:03.240455  1688 raft_consensus.cc:2955] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:00:03.257103  1537 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:03.264235  1437 catalog_manager.cc:5697] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
W20260709 13:00:03.285414  1508 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab5b971190d342d2b2568c30f3c0c81f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:03.288565  1686 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:03.289893  2995 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:03.293954  1613 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:03.295224  2995 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:03.303076  3039 raft_consensus.cc:2955] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:00:03.305028  1686 raft_consensus.cc:2955] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:00:03.306701  1613 raft_consensus.cc:2955] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:00:03.328267  1440 catalog_manager.cc:5697] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:00:03.331766  1613 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
W20260709 13:00:03.345182  1583 consensus_peers.cc:597] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 488823792cd54090a36c758519a28875. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:00:03.351578  1688 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:03.353868  2774 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:03.355973  1537 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:00:03.358115  2774 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:03.367126  3058 raft_consensus.cc:2955] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:00:03.369768  1537 raft_consensus.cc:2955] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:00:03.373149  1686 raft_consensus.cc:2955] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:00:03.384512  1438 catalog_manager.cc:5697] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index -1 to 2, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:03.557842   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:03.581868  3074 ts_tablet_manager.cc:933] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:03.590790  3074 tablet_copy_client.cc:323] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43887
I20260709 13:00:03.593268  1627 tablet_copy_service.cc:140] P 8a6a5ae1b7f240408b3ddd30d8a18262: Received BeginTabletCopySession request for tablet caae9b2a426d42fb8c2353e1c978d0fb from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:45356)
I20260709 13:00:03.593878  1627 tablet_copy_service.cc:161] P 8a6a5ae1b7f240408b3ddd30d8a18262: Beginning new tablet copy session on tablet caae9b2a426d42fb8c2353e1c978d0fb from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:45356: session id = cecad4f86a114a7696e398b3e8cc511e-caae9b2a426d42fb8c2353e1c978d0fb
I20260709 13:00:03.601598  1627 tablet_copy_source_session.cc:215] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:03.605175  3074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caae9b2a426d42fb8c2353e1c978d0fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:03.625547  3074 tablet_copy_client.cc:806] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:03.627159  3074 tablet_copy_client.cc:670] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:03.627923  1537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8377b7234e4472ea27005f16dede669"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:53670
I20260709 13:00:03.628538  1537 raft_consensus.cc:606] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:00:03.636626  1613 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:00:03.637240  1613 raft_consensus.cc:606] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:00:03.640930  3074 tablet_copy_client.cc:538] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:03.640972   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:00:03.644518   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:03.653636  3074 tablet_bootstrap.cc:492] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 13:00:03.686568  3074 tablet_bootstrap.cc:492] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:03.687276  3074 tablet_bootstrap.cc:492] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 13:00:03.687822  3074 ts_tablet_manager.cc:1403] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.034s	user 0.024s	sys 0.008s
I20260709 13:00:03.690011  3074 raft_consensus.cc:359] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:03.690616  3074 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 13:00:03.691139  3074 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:03.693194  3074 ts_tablet_manager.cc:1434] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:00:03.695263  1627 tablet_copy_service.cc:342] P 8a6a5ae1b7f240408b3ddd30d8a18262: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-caae9b2a426d42fb8c2353e1c978d0fb received from {username='slave'} at 127.0.0.1:45356
I20260709 13:00:03.695725  1627 tablet_copy_service.cc:434] P 8a6a5ae1b7f240408b3ddd30d8a18262: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-caae9b2a426d42fb8c2353e1c978d0fb on tablet caae9b2a426d42fb8c2353e1c978d0fb with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:03.703778  3074 ts_tablet_manager.cc:933] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:03.714282  3074 tablet_copy_client.cc:323] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43887
I20260709 13:00:03.715916  1627 tablet_copy_service.cc:140] P 8a6a5ae1b7f240408b3ddd30d8a18262: Received BeginTabletCopySession request for tablet c65847e774e445909d83c69bf486b7fc from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:45356)
I20260709 13:00:03.716459  1627 tablet_copy_service.cc:161] P 8a6a5ae1b7f240408b3ddd30d8a18262: Beginning new tablet copy session on tablet c65847e774e445909d83c69bf486b7fc from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:45356: session id = cecad4f86a114a7696e398b3e8cc511e-c65847e774e445909d83c69bf486b7fc
I20260709 13:00:03.724162  1627 tablet_copy_source_session.cc:215] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:03.727509  3074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c65847e774e445909d83c69bf486b7fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:03.743806  3078 ts_tablet_manager.cc:933] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:03.744904  3074 tablet_copy_client.cc:806] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:03.745726  3074 tablet_copy_client.cc:670] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:03.746388  3078 tablet_copy_client.cc:323] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.194:43887
I20260709 13:00:03.748641  1628 tablet_copy_service.cc:140] P 8a6a5ae1b7f240408b3ddd30d8a18262: Received BeginTabletCopySession request for tablet 488823792cd54090a36c758519a28875 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:45356)
I20260709 13:00:03.749179  1628 tablet_copy_service.cc:161] P 8a6a5ae1b7f240408b3ddd30d8a18262: Beginning new tablet copy session on tablet 488823792cd54090a36c758519a28875 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:45356: session id = cecad4f86a114a7696e398b3e8cc511e-488823792cd54090a36c758519a28875
I20260709 13:00:03.749693  3074 tablet_copy_client.cc:538] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:03.757458  1628 tablet_copy_source_session.cc:215] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:03.763549  3074 tablet_bootstrap.cc:492] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 13:00:03.766450  3078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 488823792cd54090a36c758519a28875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:03.796453  3078 tablet_copy_client.cc:806] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:03.797051  3078 tablet_copy_client.cc:670] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:03.806061  3078 tablet_copy_client.cc:538] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:03.806715  3074 tablet_bootstrap.cc:492] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:03.807657  3074 tablet_bootstrap.cc:492] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 13:00:03.808336  3074 ts_tablet_manager.cc:1403] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.045s	user 0.026s	sys 0.010s
I20260709 13:00:03.811785  3074 raft_consensus.cc:359] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:03.812681  3074 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 13:00:03.813292  3078 tablet_bootstrap.cc:492] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 13:00:03.813354  3074 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:03.816159  3074 ts_tablet_manager.cc:1434] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:00:03.818078  1628 tablet_copy_service.cc:342] P 8a6a5ae1b7f240408b3ddd30d8a18262: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-c65847e774e445909d83c69bf486b7fc received from {username='slave'} at 127.0.0.1:45356
I20260709 13:00:03.818488  1628 tablet_copy_service.cc:434] P 8a6a5ae1b7f240408b3ddd30d8a18262: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-c65847e774e445909d83c69bf486b7fc on tablet c65847e774e445909d83c69bf486b7fc with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:03.842491  3078 tablet_bootstrap.cc:492] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:03.843269  3078 tablet_bootstrap.cc:492] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 13:00:03.843873  3078 ts_tablet_manager.cc:1403] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.007s
I20260709 13:00:03.846225  3078 raft_consensus.cc:359] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:03.846858  3078 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 13:00:03.847512  3078 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:03.849819  3078 ts_tablet_manager.cc:1434] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 13:00:03.852023  1628 tablet_copy_service.cc:342] P 8a6a5ae1b7f240408b3ddd30d8a18262: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-488823792cd54090a36c758519a28875 received from {username='slave'} at 127.0.0.1:45356
I20260709 13:00:03.852435  1628 tablet_copy_service.cc:434] P 8a6a5ae1b7f240408b3ddd30d8a18262: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-488823792cd54090a36c758519a28875 on tablet 488823792cd54090a36c758519a28875 with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:03.861259  3078 ts_tablet_manager.cc:933] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
I20260709 13:00:03.869673  3078 tablet_copy_client.cc:323] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:43459
I20260709 13:00:03.871721  1553 tablet_copy_service.cc:140] P 83f6f5810950417195f707ff308a8125: Received BeginTabletCopySession request for tablet ab5b971190d342d2b2568c30f3c0c81f from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:33722)
I20260709 13:00:03.872285  1553 tablet_copy_service.cc:161] P 83f6f5810950417195f707ff308a8125: Beginning new tablet copy session on tablet ab5b971190d342d2b2568c30f3c0c81f from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:33722: session id = cecad4f86a114a7696e398b3e8cc511e-ab5b971190d342d2b2568c30f3c0c81f
I20260709 13:00:03.880081  1553 tablet_copy_source_session.cc:215] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:00:03.883728  3078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab5b971190d342d2b2568c30f3c0c81f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:00:03.905550  3078 tablet_copy_client.cc:806] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 13:00:03.906313  3078 tablet_copy_client.cc:670] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:00:03.910408  3078 tablet_copy_client.cc:538] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:00:03.919979  3078 tablet_bootstrap.cc:492] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 13:00:03.937077  3078 tablet_bootstrap.cc:492] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:00:03.937948  3078 tablet_bootstrap.cc:492] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 13:00:03.938599  3078 ts_tablet_manager.cc:1403] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.003s
I20260709 13:00:03.941553  3078 raft_consensus.cc:359] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:03.942219  3078 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 13:00:03.943648  3078 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:03.946445  3078 ts_tablet_manager.cc:1434] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:00:03.949124  1553 tablet_copy_service.cc:342] P 83f6f5810950417195f707ff308a8125: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-ab5b971190d342d2b2568c30f3c0c81f received from {username='slave'} at 127.0.0.1:33722
I20260709 13:00:03.949604  1553 tablet_copy_service.cc:434] P 83f6f5810950417195f707ff308a8125: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-ab5b971190d342d2b2568c30f3c0c81f on tablet ab5b971190d342d2b2568c30f3c0c81f with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:03.962036  2236 raft_consensus.cc:1217] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:04.043193  2774 raft_consensus.cc:993] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:00:04.043655  3058 raft_consensus.cc:1081] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:00:04.045537  1541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 13:00:04.046119  1541 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:04.046497  1541 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.053694  1541 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:04.057250  1541 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:04.058909  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:04.059496  2236 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.064967  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:04.065681  2236 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:04.067107  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:04.065802  1614 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:04.067993  1614 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:04.068145  3039 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:04.078756  1614 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:04.082628  1614 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.085116  3039 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:04.086151  3039 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:04.086778  2994 raft_consensus.cc:993] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: : Instructing follower 984c9089d9804652846f3caeb609d56e to start an election
I20260709 13:00:04.087370  2994 raft_consensus.cc:1081] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Signalling peer 984c9089d9804652846f3caeb609d56e to start an election
I20260709 13:00:04.093796  1686 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c8377b7234e4472ea27005f16dede669"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40124
I20260709 13:00:04.094470  1686 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:04.094883  1686 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.098704  1614 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:04.107239  1686 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:04.110145  1686 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:04.113070  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:04.112562  1438 catalog_manager.cc:5697] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:04.113680  1541 raft_consensus.cc:3055] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:04.114023  1541 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:04.114718  1541 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:04.115850  1541 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.117769  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:04.118336  1614 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:04.125450  1614 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:04.128118  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:04.129451  1541 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:04.129877  3014 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:04.130435  3014 raft_consensus.cc:697] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:04.131281  3014 consensus_queue.cc:237] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:04.141700  2236 raft_consensus.cc:1217] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:04.157269  1438 catalog_manager.cc:5697] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 1 to 2, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 2 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:04.208992  2236 raft_consensus.cc:1217] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:00:04.253998  2236 raft_consensus.cc:1217] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:00:04.521000  2774 raft_consensus.cc:1064] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 13:00:04.523373  2774 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:04.531244  2236 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:04.531749  1686 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:04.532245  1537 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:04.532769  2774 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:04.533562  2975 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:04.534515  3058 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:04.554481  3058 raft_consensus.cc:2955] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:04.557866  2237 raft_consensus.cc:2955] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:04.567751  1688 raft_consensus.cc:2955] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:04.556183  1537 raft_consensus.cc:2955] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:04.583171  1438 catalog_manager.cc:5697] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:04.588515  1613 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:04.595713  3085 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:04.602965  2236 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:04.604846  3085 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:04.623222  1614 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:04.631577  2233 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:04.633219  3058 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:04.639179  2975 raft_consensus.cc:1064] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 13:00:04.644227  2975 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:04.645622   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:04.662039  1687 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:00:04.680379  3101 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:04.698336  3100 raft_consensus.cc:1064] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 13:00:04.716707  1541 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:04.718590  3014 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:00:04.751845  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 83318.3us
I20260709 13:00:04.712918  3058 raft_consensus.cc:2955] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:04.815587  1688 raft_consensus.cc:2955] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:04.815917  3100 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:04.830690  1614 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:00:04.837102  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 75777us
I20260709 13:00:04.845162  2234 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:04.846913  2233 raft_consensus.cc:2955] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:04.851482  3101 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:04.853336  1542 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:04.855311  3014 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:04.875016  3107 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:04.878963  1687 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:00:04.890148  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 71279.9us
I20260709 13:00:04.891881  2994 raft_consensus.cc:1064] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 13:00:04.894256  2237 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 2 LEARNER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:04.896634  3107 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:04.898357  2994 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:04.900440  1440 catalog_manager.cc:5697] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 3 to 4, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:04.902155  1541 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:04.904394  3102 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:04.924860  1428 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 488823792cd54090a36c758519a28875 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:00:04.934791  2237 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 LEARNER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:04.936265  1687 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:04.944295  3039 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:04.946517  1528 tablet_service.cc:1558] Processing DeleteTablet for tablet 488823792cd54090a36c758519a28875 with delete_type TABLET_DATA_TOMBSTONED (TS 83f6f5810950417195f707ff308a8125 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54754
W20260709 13:00:04.948765  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 115156us
W20260709 13:00:04.955020  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 93821.5us
I20260709 13:00:04.954856  2975 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:04.958921  1542 raft_consensus.cc:2955] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:04.963488  3106 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:00:04.966883  2774 raft_consensus.cc:2955] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:04.975625  1618 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:04.974046  1541 raft_consensus.cc:2955] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:04.980259  3122 tablet_replica.cc:333] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:00:04.981837  3122 raft_consensus.cc:2243] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:00:04.982592  3122 raft_consensus.cc:2272] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:00:05.030256  1687 raft_consensus.cc:2955] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:04.947791  3102 raft_consensus.cc:2955] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:05.154918  1440 catalog_manager.cc:5697] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:05.166356  1688 raft_consensus.cc:2955] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:00:05.152619  3109 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
W20260709 13:00:05.152737  3121 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:00:05.177263  1690 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:00:05.177035  2233 raft_consensus.cc:2955] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:00:05.199090  3125 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
W20260709 13:00:05.284404  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 178035us
W20260709 13:00:05.423821  3124 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
W20260709 13:00:05.439677  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 442537us
W20260709 13:00:05.693392  1655 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.011s	sys 0.019s
W20260709 13:00:05.694039  1655 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.011s	sys 0.020s
I20260709 13:00:05.696030  3122 ts_tablet_manager.cc:1916] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 13:00:05.758330  1509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.761s	user 0.001s	sys 0.024s
W20260709 13:00:05.758738  1509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.001s	sys 0.024s
W20260709 13:00:05.759022  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 764306us
I20260709 13:00:05.760442  3112 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:05.769606  3122 ts_tablet_manager.cc:1929] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:00:05.770080  3122 log.cc:1199] T 488823792cd54090a36c758519a28875 P 83f6f5810950417195f707ff308a8125: Deleting WAL directory at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0/minicluster-data/ts-0-root/wals/488823792cd54090a36c758519a28875
W20260709 13:00:05.862401  1542 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.853s	user 0.008s	sys 0.012s
W20260709 13:00:05.862831  1542 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.009s	sys 0.013s
W20260709 13:00:05.871873  1584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.009s	sys 0.019s
W20260709 13:00:05.872244  1584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.010s	sys 0.020s
W20260709 13:00:05.872576  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 585923us
W20260709 13:00:05.883332  1541 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.010s	sys 0.017s
W20260709 13:00:05.883733  1541 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.010s	sys 0.017s
W20260709 13:00:05.900556  1583 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.000s	sys 0.025s
W20260709 13:00:05.900954  1583 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.000s	sys 0.026s
I20260709 13:00:05.912328  2237 raft_consensus.cc:2955] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:05.921823  1428 catalog_manager.cc:5028] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): tablet 488823792cd54090a36c758519a28875 (table test-workload [id=98ee3c50fcb1447c9e4b9fef489d68e8]) successfully deleted
I20260709 13:00:05.929968  3101 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:05.930567  3101 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:05.932793  3101 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:05.954784  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:06.006376  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 44551.3us
I20260709 13:00:06.265661   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:06.390671  3143 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:06.391492  3143 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:06.427807  3143 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:06.470867  3143 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:06.471733  3143 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:06.474826  3143 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:06.511299  3141 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:06.631075  3141 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:06.634334  3141 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:06.635078  3141 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:06.635756  3141 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:06.638279  3141 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:06.641773  3155 log.cc:927] Time spent T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.124s	user 0.002s	sys 0.000s
I20260709 13:00:06.650822  3143 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:06.651412  3143 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:06.654345  3143 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:06.655159  1611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.000s	sys 0.011s
W20260709 13:00:06.658490  1611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.000s	sys 0.011s
I20260709 13:00:06.660359  3143 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:06.661005  3143 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:06.663774  3143 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:06.664378  3143 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:06.665047  3143 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:06.668613  3143 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.107507  3143 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.110658  3143 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.122416  3112 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:07.128654  3143 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.136915  3159 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.137917  3159 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.140668  3159 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:07.149547  3043 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.151118  3159 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.151863  3159 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.150638  3043 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.154783  3014 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.155061  3095 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.155689  3014 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.156273  3043 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:07.156057  3095 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.158392  3014 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:07.159561  3095 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:07.160338  3095 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:07.161643  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 1.21654e+06us
I20260709 13:00:07.165241  3102 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:07.166124  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 1.1582e+06us
I20260709 13:00:07.161441  3095 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.170265  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 193) took 1219 ms. Trace:
I20260709 13:00:07.170833  1616 rpcz_store.cc:276] 0709 13:00:05.950530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.168642 (+1218112us) service_pool.cc:224] Handling call
0709 13:00:07.169219 (+   577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:07.169673 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:07.169707 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:07.169763 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:07.169801 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:07.169882 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:07.170040 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":38272}
I20260709 13:00:07.167379  3159 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:07.176216  3095 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:07.181545  3014 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.182459  3014 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.180150  3102 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.191267  2975 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.192274  2975 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.202018  3138 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.203018  3138 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.205844  3014 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:07.218360  3101 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.219512  3101 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.230877  3117 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.231815  3117 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.235212  3162 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.235813  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:07.236340  3162 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.251192  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 220) took 1431 ms. Trace:
I20260709 13:00:07.251823  1537 rpcz_store.cc:276] 0709 13:00:05.819108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.249323 (+1430215us) service_pool.cc:224] Handling call
0709 13:00:07.250134 (+   811us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:07.250572 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:07.250604 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:07.250653 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:07.250678 (+    25us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:07.250743 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:07.250927 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:07.253667  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:07.254808  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:07.258936  3117 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.260113  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:07.264717  3102 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.272006  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:07.273663  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 216) took 1563 ms. Trace:
I20260709 13:00:07.275096  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:07.276542  1690 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:07.273090  1686 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:07.277300  3117 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.278199  3117 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.278668  3043 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.279300  3043 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.225008  3095 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.274248  1688 rpcz_store.cc:276] 0709 13:00:05.707364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.707499 (+   135us) service_pool.cc:224] Handling call
0709 13:00:07.268535 (+1561036us) spinlock_profiling.cc:243] Waited 5.51 ms on lock 0x7b5800010e98. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:00:07.270331 (+  1796us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:07.270737 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:07.270761 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:07.270811 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:07.270841 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:07.270894 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:07.271042 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12671232}
I20260709 13:00:07.285720  3159 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.286585  3159 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.287189  2933 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.287542  3043 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.298540  3043 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.299428  3043 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.300063  3115 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.300808  3115 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.305402  3043 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.306480  3115 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.288151  2933 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.308579  3095 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.309576  2933 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:07.307561  3159 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:07.317044  3162 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:07.318032  3117 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.318169  3115 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.319226  3115 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.322084  3158 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.324668  3111 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.325496  3112 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.328729  3112 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.330818  3143 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.331009  3157 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.331646  3143 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.328150  2933 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.317059  3095 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:07.335985  3085 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.336450  3112 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.332163  3157 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.343930  3168 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.343641  3158 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.341687  3043 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:00:07.348117  1639 debug-util.cc:398] Leaking SignalData structure 0x7b08000ab700 after lost signal to thread 434
I20260709 13:00:07.325667  3111 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.336728  3085 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.346063  2933 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.358168  3043 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.358196  3134 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.342338  3162 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.346990  3163 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.416363  2933 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:07.364836  3112 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.396446  3159 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.433657  3160 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.416142  3164 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.401728  3162 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.434700  3160 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.459506  3102 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.460260  3102 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.461513  3160 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:07.462224  3160 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.462898  3160 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.465231  3160 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:07.466524  3129 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.467343  3129 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.468087  2933 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.451032  3014 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.399500  3163 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.477586  2933 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.483747  3101 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.465816  3102 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:07.442411  3169 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.514405  3158 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.516181  3111 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.478447  3157 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:07.528918  3171 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.468408  3112 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.510993  3143 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.458446  3134 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:07.508203  3159 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.479530  3170 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.517083  3014 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.685431  3176 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.535001  3111 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.533378  3102 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.529860  3171 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.458240  3168 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:00:07.688047  3138 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.597514  2933 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:07.604974  3112 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.534523  3115 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.647468  3056 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.590775  3178 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.639225  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:07.580693  3174 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.647681  3177 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.639379  3179 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.629617  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:07.538116  3157 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.552789  3101 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.580364  3165 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.624365  3184 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.527871  3160 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.517227  3164 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:07.686442  1639 debug-util.cc:398] Leaking SignalData structure 0x7b08000f0fc0 after lost signal to thread 1475
I20260709 13:00:07.694303  3102 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.647286  3127 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:00:07.710739  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 384793us
I20260709 13:00:07.703456  2201 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:07.693502  3043 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:07.531394  3158 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.591132  3116 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.697412  3163 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:07.603371  3095 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:07.697638  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 354768us
I20260709 13:00:07.691089  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:07.736704  3132 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.694281  3111 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.708986  3143 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.512202  3117 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.733765  3138 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.738998  3115 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.747711  3162 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:07.746850  3183 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.742563  3176 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.605952  3169 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.762094  3056 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.760524  3129 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:08.007212  3194 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:09.558288  3194 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.560869  3194 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:08.028339  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:08.327656  3201 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:00:09.562047  1536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1504) took 1913 ms (client timeout 1700 ms). Trace:
I20260709 13:00:09.562175  3201 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:00:09.562515  1536 rpcz_store.cc:269] 0709 13:00:07.695083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.703884 (+  8801us) service_pool.cc:224] Handling call
0709 13:00:09.561907 (+1858023us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.563854  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.564505  3201 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:08.684321  3199 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:08.714425  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:08.716349  3203 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:09.565856  3203 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:09.566135  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.566320  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.567468  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.567600  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.568065  1538 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:08.729519  3202 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:09.569013  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:08.738919  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1501) took 1163 ms. Trace:
I20260709 13:00:08.750373  3207 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:08.758641  3208 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:08.786767  3209 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:08.787405  3212 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:09.569887  3207 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:09.569619  1537 rpcz_store.cc:276] 0709 13:00:07.575754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.575964 (+   210us) service_pool.cc:224] Handling call
0709 13:00:07.576450 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:08.736119 (+1159669us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:08.736153 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:08.736207 (+    54us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:08.736255 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:08.738578 (+  2323us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54001d7ed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:08.738631 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:08.738818 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365293312}
I20260709 13:00:09.570081  3208 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.570638  3212 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.571519  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.572449  3207 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.572979  3208 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:08.819404  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.565263  3199 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:09.573860  3212 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.574534  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.575251  1535 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 13:00:09.575865  3199 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.576223  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.576339  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:08.831651  3215 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:09.576897  1613 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:09.577167  3215 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.577661  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.578500  1611 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:09.579581  3215 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:08.847307  3216 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:09.579723  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:08.870898  3213 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:09.580473  3216 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.580632  3213 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:08.860493  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:09.004433  1507 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.531s (SENT) [suppressed 5 similar messages]
W20260709 13:00:09.005231  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 155770us
I20260709 13:00:09.582367  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.582898  3213 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.583676  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.584370  1537 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:09.584496  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:08.853258  1438 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40172 (request call id 226) took 2886 ms. Trace:
I20260709 13:00:09.585587  1438 rpcz_store.cc:276] 0709 13:00:05.967112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.967270 (+   158us) service_pool.cc:224] Handling call
0709 13:00:08.853011 (+2885741us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 13:00:08.916384  3214 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:08.803158  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.592195  1612 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:08.795992  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:08.820920  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.598429  1617 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:08.789145  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:08.782698  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.597254  1614 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
W20260709 13:00:08.007918  1639 debug-util.cc:398] Leaking SignalData structure 0x7b08001d6b00 after lost signal to thread 1509
I20260709 13:00:08.834111  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:08.822674  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 46456.8us
I20260709 13:00:09.038981  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:07.762320  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:07.956516  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:08.759979  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:09.608551  1685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 233) took 1865 ms (client timeout 1594 ms). Trace:
W20260709 13:00:09.608974  1685 rpcz_store.cc:269] 0709 13:00:07.742820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.749377 (+  6557us) service_pool.cc:224] Handling call
0709 13:00:09.608436 (+1859059us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.610160  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:09.610994  1685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1081) took 1562 ms (client timeout 1593 ms). Trace:
W20260709 13:00:09.611346  1685 rpcz_store.cc:269] 0709 13:00:08.048664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.609864 (+1561200us) service_pool.cc:224] Handling call
0709 13:00:09.610919 (+  1055us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.612246  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:09.613034  1685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1082) took 1564 ms (client timeout 1791 ms). Trace:
I20260709 13:00:08.744904  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:07.629364  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:08.837567  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:07.766125  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:08.834165  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:09.069058  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:07.769246  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 233336us
I20260709 13:00:08.720534  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:08.727246  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:08.724884  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:08.729079  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:08.715354  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:09.366282  2234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 367) took 1840 ms (client timeout 1841 ms). Trace:
I20260709 13:00:09.200920  2933 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.955542  3014 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:09.371939  3056 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:08.730851  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:08.734784  3085 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:08.753247  3098 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:07.792613  3101 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.991041  3102 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:07.937769  3116 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.853036  3127 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:08.011966  3134 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:07.814137  3157 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:08.014710  3159 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:07.989739  3164 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:08.034346  2975 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:07.877064  3095 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.800619  3165 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.905498  3158 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.766929  3160 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.365826  3138 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.365904  3132 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:08.011826  3168 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:07.766615  3170 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.974243  3171 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:08.858958  3172 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.811924  3173 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.751919  3174 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.812600  3175 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:09.373016  3176 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:07.789871  3177 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.789649  3178 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:07.904933  3179 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.766067  3180 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.790988  3181 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.955818  3182 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.813221  3184 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:07.812793  3185 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:07.956347  3186 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:07.812981  3188 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:07.811460  3189 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:09.370142  3143 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:08.731355  1658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.008s	sys 0.028s
I20260709 13:00:09.378731  3219 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:09.396543  1440 catalog_manager.cc:5697] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:09.414248  3217 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:09.365933  3117 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:09.440251  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:09.457224  1509 connection.cc:256] RPC call timeout handler was delayed by -1.4201789360000134s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
I20260709 13:00:09.470738  3183 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.496832  3115 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.529780  3221 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:09.530292  3228 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:09.531880  3111 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.539647  3169 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.543532  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 634) took 1215 ms. Trace:
I20260709 13:00:09.550418  3162 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:09.555008  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 442) took 2215 ms. Trace:
I20260709 13:00:09.568125  3203 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:08.725947  3204 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:09.568814  3202 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:09.570188  3209 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:09.579761  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.586822  3214 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:08.730273  1582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.004s	sys 0.019s
W20260709 13:00:08.682370  1583 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.065s	sys 0.137s
I20260709 13:00:09.589401  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.594344  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:09.599107  3216 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.599639  1535 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:09.600054  1616 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
W20260709 13:00:09.600713  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 596371us
W20260709 13:00:09.604501  1639 debug-util.cc:398] Leaking SignalData structure 0x7b0800341fa0 after lost signal to thread 1567
I20260709 13:00:09.607546  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:09.805902  1687 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 227) took 2226 ms (client timeout 1780 ms). Trace:
W20260709 13:00:09.806468  1687 rpcz_store.cc:269] 0709 13:00:07.579045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.595629 (+ 16584us) service_pool.cc:224] Handling call
0709 13:00:08.833899 (+1238270us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b5800010e98. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:09.805805 (+971906us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6684032}
I20260709 13:00:09.807299  3127 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.808077  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1560) took 1020 ms. Trace:
I20260709 13:00:09.808352  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:09.808502  1616 rpcz_store.cc:276] 0709 13:00:08.788239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.788728 (+   489us) service_pool.cc:224] Handling call
0709 13:00:09.808003 (+1019275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.811067  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1502) took 2233 ms. Trace:
I20260709 13:00:09.811331  3173 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.811928  3177 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.811529  1540 rpcz_store.cc:276] 0709 13:00:07.577838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.591186 (+ 13348us) service_pool.cc:224] Handling call
0709 13:00:07.603646 (+ 12460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.809985 (+2206339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.810022 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.810095 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:09.810144 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.810694 (+   550us) spinlock_profiling.cc:243] Waited 2.21 s on lock 0x7b54000ed8d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:09.810756 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.810888 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5073526144}
I20260709 13:00:09.812923  3219 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:09.813333  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:09.613386  1685 rpcz_store.cc:269] 0709 13:00:08.048933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.611998 (+1563065us) service_pool.cc:224] Handling call
0709 13:00:09.612978 (+   980us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.814190  3173 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.815105  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:09.815220  1689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 228) took 2209 ms (client timeout 1522 ms). Trace:
I20260709 13:00:09.815351  3219 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.809315  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 636) took 1177 ms. Trace:
W20260709 13:00:09.838165  1583 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.823s	user 0.068s	sys 0.139s
I20260709 13:00:09.839003  3199 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:09.839071  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 237) took 1020 ms. Trace:
I20260709 13:00:09.839762  3199 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:09.839859  1535 rpcz_store.cc:276] 0709 13:00:08.818217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.575902 (+757685us) service_pool.cc:224] Handling call
0709 13:00:09.838813 (+262911us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b54004187d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:09.838993 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52167552}
I20260709 13:00:09.840708  3209 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.841478  3221 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.841969  1542 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:09.843149  3143 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.843634  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.843832  3221 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.844242  1542 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:09.844836  3199 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.845129  3132 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.839771  3194 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:09.845896  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 231) took 2150 ms. Trace:
I20260709 13:00:09.845959  1684 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:09.846004  3194 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.846359  1691 rpcz_store.cc:276] 0709 13:00:07.700972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.733956 (+ 32984us) service_pool.cc:224] Handling call
0709 13:00:07.742906 (+  8950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.844910 (+2102004us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.844940 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.844980 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:09.845010 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.845555 (+   545us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b540028f0d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:09.845595 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.845715 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4833603328}
W20260709 13:00:09.847294  1684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 234) took 2104 ms (client timeout 1838 ms). Trace:
I20260709 13:00:09.847867  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:09.848376  3194 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.848883  1691 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:09.848769  3217 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:09.849506  1691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1090) took 1798 ms (client timeout 1995 ms). Trace:
I20260709 13:00:09.849746  1686 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
W20260709 13:00:09.849855  1691 rpcz_store.cc:269] 0709 13:00:08.051281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.847613 (+1796332us) service_pool.cc:224] Handling call
0709 13:00:09.849454 (+  1841us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:09.850872  1686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1083) took 1801 ms (client timeout 1531 ms). Trace:
I20260709 13:00:09.851014  3217 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.851850  3116 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.851285  3204 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:09.851315  1686 rpcz_store.cc:269] 0709 13:00:08.049215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.759580 (+710365us) service_pool.cc:224] Handling call
0709 13:00:09.850726 (+1091146us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54002ce054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:09.850813 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554169088}
I20260709 13:00:09.850872  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:09.853485  3160 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.852994  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:09.853957  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1511) took 1135 ms. Trace:
I20260709 13:00:09.854467  1686 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 13:00:09.854264  3172 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:09.855027  1686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1084) took 1805 ms (client timeout 2077 ms). Trace:
W20260709 13:00:09.855379  1686 rpcz_store.cc:269] 0709 13:00:08.049538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.852712 (+1803174us) service_pool.cc:224] Handling call
0709 13:00:09.854953 (+  2241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.855719  3204 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.855911  3178 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.856341  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:09.856575  3172 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.857136  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 638) took 1161 ms. Trace:
I20260709 13:00:09.854463  1534 rpcz_store.cc:276] 0709 13:00:08.718196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.841044 (+122848us) service_pool.cc:224] Handling call
0709 13:00:08.841507 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.852802 (+1011295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.852840 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.852886 (+    46us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:09.852924 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.853527 (+   603us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540012d4d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:09.853576 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.853732 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2325199104}
I20260709 13:00:09.858527  3098 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:09.859684  1688 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 1844) took 1847 ms (client timeout 1539 ms). Trace:
I20260709 13:00:09.859544  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.860836  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1566) took 1043 ms. Trace:
I20260709 13:00:09.853678  1691 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
W20260709 13:00:09.862718  1691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 1845) took 1850 ms (client timeout 2100 ms). Trace:
I20260709 13:00:09.862883  2201 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:09.861178  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.863770  2230 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 13:00:09.861260  1610 rpcz_store.cc:276] 0709 13:00:08.816951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.817141 (+   190us) service_pool.cc:224] Handling call
0709 13:00:08.817682 (+   541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.859750 (+1042068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.859785 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.859838 (+    53us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:09.859874 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.860428 (+   554us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002bdc54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:09.860476 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.860635 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2396027648}
I20260709 13:00:09.864670  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.865135  3208 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:09.864979  3186 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.865694  3208 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:00:09.857100  1686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1089) took 1806 ms (client timeout 2147 ms). Trace:
W20260709 13:00:09.866550  1686 rpcz_store.cc:269] 0709 13:00:08.050989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.856063 (+1805074us) service_pool.cc:224] Handling call
0709 13:00:09.857026 (+   963us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.867376  3186 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.867575  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:09.867921  3157 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.865099  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 644) took 1145 ms. Trace:
I20260709 13:00:09.866127  3208 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:09.863498  3098 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.868340  3189 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:09.869004  3189 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:09.868886  3181 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:09.871248  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 1840) took 2080 ms. Trace:
I20260709 13:00:09.871378  3181 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:09.871717  1683 rpcz_store.cc:276] 0709 13:00:07.813960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.905201 (+ 91241us) service_pool.cc:224] Handling call
0709 13:00:07.987830 (+ 82629us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.870166 (+1882336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.870199 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.870271 (+    72us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:09.870307 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.870856 (+   549us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b5400375854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:09.870904 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.871035 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4328341248}
I20260709 13:00:09.872254  3185 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:09.873881  1683 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:00:09.874810  1683 raft_consensus.cc:606] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
W20260709 13:00:09.875438  1658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.823s	user 0.013s	sys 0.028s
W20260709 13:00:09.875801  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 1.8231e+06us
I20260709 13:00:09.877243  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1076) took 1884 ms. Trace:
I20260709 13:00:09.877699  1683 rpcz_store.cc:276] 0709 13:00:07.992519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.876364 (+1883845us) service_pool.cc:224] Handling call
0709 13:00:09.876705 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.876892 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.876919 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.876965 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:09.876996 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.877053 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.877183 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.879104  3208 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.879571  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1077) took 1875 ms. Trace:
I20260709 13:00:09.880029  1683 rpcz_store.cc:276] 0709 13:00:08.005394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.878706 (+1873312us) service_pool.cc:224] Handling call
0709 13:00:09.879033 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.879218 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.879249 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.879295 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:09.879354 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.879415 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.879517 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.865931  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.859000  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 645) took 1131 ms. Trace:
I20260709 13:00:09.838374  3162 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.862104  3202 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.881345  3208 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.897454  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.919768  3180 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:09.920499  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.920677  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1571) took 1088 ms. Trace:
I20260709 13:00:09.921196  1613 rpcz_store.cc:276] 0709 13:00:08.832751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.587974 (+755223us) service_pool.cc:224] Handling call
0709 13:00:09.588263 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.919770 (+331507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.919808 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.919869 (+    61us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:09.919905 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.920434 (+   529us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b54003c8cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:09.920476 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.920617 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":761821440}
I20260709 13:00:09.817564  3115 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.922387  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.806470  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.923189  3180 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:09.923668  1692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 232) took 2211 ms (client timeout 1520 ms). Trace:
W20260709 13:00:09.858103  1582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.845s	user 0.004s	sys 0.023s
I20260709 13:00:09.913213  1687 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
W20260709 13:00:09.860066  1688 rpcz_store.cc:269] 0709 13:00:08.011954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.744632 (+732678us) service_pool.cc:224] Handling call
0709 13:00:09.859620 (+1114988us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.926064  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 1846) took 1913 ms. Trace:
I20260709 13:00:09.926512  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:09.918403  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 310694us
I20260709 13:00:09.926687  1688 rpcz_store.cc:276] 0709 13:00:08.012584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.925270 (+1912686us) service_pool.cc:224] Handling call
0709 13:00:09.925606 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.925766 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.925789 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.925831 (+    42us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:09.925859 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.925910 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.926012 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.816382  1618 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:09.845178  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.857647  2233 rpcz_store.cc:276] 0709 13:00:08.695881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.696090 (+   209us) service_pool.cc:224] Handling call
0709 13:00:09.857069 (+1160979us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.900880  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.909232  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:09.863165  1691 rpcz_store.cc:269] 0709 13:00:08.012262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.850619 (+1838357us) service_pool.cc:224] Handling call
0709 13:00:09.862661 (+ 12042us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.874890  3185 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.930164  1617 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:09.924674  1690 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
W20260709 13:00:09.939311  1690 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1080) took 1891 ms (client timeout 1561 ms). Trace:
W20260709 13:00:09.926625  1687 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1088) took 1876 ms (client timeout 1841 ms). Trace:
W20260709 13:00:09.940104  1687 rpcz_store.cc:269] 0709 13:00:08.050694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.808074 (+1757380us) service_pool.cc:224] Handling call
0709 13:00:09.926460 (+118386us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540038aed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:09.926564 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238664960}
W20260709 13:00:09.933542  1509 connection.cc:256] RPC call timeout handler was delayed by -1.3170598399999562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 13:00:09.883170  3095 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.934567  3202 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:09.934813  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.882124  3184 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.935456  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.944751  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 236) took 1126 ms. Trace:
W20260709 13:00:09.867982  2234 rpcz_store.cc:269] 0709 13:00:07.525382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.569724 (+ 44342us) service_pool.cc:224] Handling call
0709 13:00:09.366204 (+1796480us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8576}
I20260709 13:00:09.936260  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:09.892485  1639 debug-util.cc:398] Leaking SignalData structure 0x7b08002020e0 after lost signal to thread 1642
I20260709 13:00:09.856611  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1512) took 1138 ms. Trace:
W20260709 13:00:09.924086  1692 rpcz_store.cc:269] 0709 13:00:07.718412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.737664 (+ 19252us) service_pool.cc:224] Handling call
0709 13:00:09.923597 (+2185933us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.927992  2236 rpcz_store.cc:276] 0709 13:00:08.328019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.328234 (+   215us) service_pool.cc:224] Handling call
0709 13:00:08.328520 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.370751 (+1042231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.370791 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.370852 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:09.370910 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.543137 (+172227us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540065a9d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:09.543171 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.543350 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2396393344}
I20260709 13:00:09.951709  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:09.952312  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.941852  2933 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:09.939689  1690 rpcz_store.cc:269] 0709 13:00:08.048375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.837249 (+788874us) service_pool.cc:224] Handling call
0709 13:00:09.939251 (+1102002us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:09.943933  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 349727us
I20260709 13:00:09.944552  1614 raft_consensus.cc:2410] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 2 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 13:00:09.941267  1509 connection.cc:256] RPC call timeout handler was delayed by -1.0987343219999843s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1791s for a response
I20260709 13:00:09.923904  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.930910  2237 rpcz_store.cc:276] 0709 13:00:08.637856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.639274 (+  1418us) service_pool.cc:224] Handling call
0709 13:00:08.643088 (+  3814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.805995 (+1162907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.806026 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.806067 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:09.806098 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.809059 (+  2961us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54005c0dd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:09.809119 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.809261 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2673803008}
I20260709 13:00:09.882808  2232 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 13:00:09.952870  1617 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:09.810595  2235 rpcz_store.cc:276] 0709 13:00:07.339000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.339141 (+   141us) service_pool.cc:224] Handling call
0709 13:00:07.491008 (+151867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.538373 (+2047365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.538404 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.538448 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:09.538477 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.554633 (+ 16156us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b540065aed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:09.554673 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.554830 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4709533312}
I20260709 13:00:09.862381  3158 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.946818  1541 rpcz_store.cc:276] 0709 13:00:08.817918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.439805 (+621887us) service_pool.cc:224] Handling call
0709 13:00:09.944692 (+504887us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.898232  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.815953  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:09.949374  1686 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:09.969501  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1559) took 1182 ms. Trace:
I20260709 13:00:09.898795  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.897924  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:09.977980  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:09.847720  1684 rpcz_store.cc:269] 0709 13:00:07.743242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.750883 (+  7641us) service_pool.cc:224] Handling call
0709 13:00:09.847124 (+2096241us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b5400355054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:09.847235 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548462464}
I20260709 13:00:09.952442  2233 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:09.923854  3101 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.968710  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1078) took 1951 ms. Trace:
I20260709 13:00:09.839003  1540 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
W20260709 13:00:09.815809  1689 rpcz_store.cc:269] 0709 13:00:07.621255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:07.628843 (+  7588us) service_pool.cc:224] Handling call
0709 13:00:09.815148 (+2186305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.950534  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1079) took 1898 ms. Trace:
I20260709 13:00:09.884943  3183 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.970358  1618 rpcz_store.cc:276] 0709 13:00:08.780988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.781627 (+   639us) service_pool.cc:224] Handling call
0709 13:00:09.962093 (+1180466us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54002bd254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:09.962179 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496137088}
I20260709 13:00:09.818243  3174 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.951313  3095 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:09.923199  1534 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 13:00:09.985240  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 647) took 1252 ms. Trace:
I20260709 13:00:10.007323  2233 rpcz_store.cc:276] 0709 13:00:08.730824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.942029 (+1211205us) service_pool.cc:224] Handling call
0709 13:00:09.983388 (+ 41359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.985987  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:09.986342  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.008363  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:09.981796  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.008414  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.009361  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:10.007242  2933 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:09.987914  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.886906  3179 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:09.951486  2228 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:09.955778  3102 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:09.928419  2231 rpcz_store.cc:276] 0709 13:00:08.727487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.728099 (+   612us) service_pool.cc:224] Handling call
0709 13:00:09.858936 (+1130837us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.950543  1539 rpcz_store.cc:276] 0709 13:00:08.718613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.068548 (+349935us) service_pool.cc:224] Handling call
0709 13:00:09.856550 (+788002us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.983322  1688 rpcz_store.cc:276] 0709 13:00:08.017731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.937335 (+1919604us) service_pool.cc:224] Handling call
0709 13:00:09.941714 (+  4379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.960635 (+ 18921us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.960671 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.968212 (+  7541us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:09.968488 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.968554 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.968659 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.961524  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.999761  1687 rpcz_store.cc:276] 0709 13:00:08.048028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.940690 (+1892662us) service_pool.cc:224] Handling call
0709 13:00:09.940913 (+   223us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.945602 (+  4689us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.945634 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.945682 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:09.945712 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:09.945769 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:09.945870 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:09.955581  1612 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:10.002529  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1086) took 1952 ms. Trace:
I20260709 13:00:09.873518  2230 rpcz_store.cc:276] 0709 13:00:08.719310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.723235 (+  3925us) service_pool.cc:224] Handling call
0709 13:00:09.864928 (+1141693us) spinlock_profiling.cc:243] Waited 7.02 ms on lock 0x7b5400599cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:09.865032 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16143744}
I20260709 13:00:09.964510  3213 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:09.999051  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:09.987469  1537 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:10.010272  1543 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:09.894991  3165 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:09.988672  1616 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:09.857806  3175 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.017346  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.018301  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.035879  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.019011  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.019826  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.036298  2237 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:09.988330  3202 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.021677  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:09.919224  3189 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.023782  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.054826  3159 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:10.027050  2228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 643) took 1306 ms. Trace:
W20260709 13:00:10.026871  1685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 1842) took 2023 ms (client timeout 2043 ms). Trace:
I20260709 13:00:10.060288  1541 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:10.026690  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 1843) took 2015 ms. Trace:
I20260709 13:00:09.979058  1610 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
W20260709 13:00:10.028117  1686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1091) took 1972 ms (client timeout 2236 ms). Trace:
I20260709 13:00:10.029682  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:10.067963  1686 rpcz_store.cc:269] 0709 13:00:08.051575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.867312 (+1815737us) service_pool.cc:224] Handling call
0709 13:00:10.023587 (+156275us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b5400375354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:10.023683 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185923456}
I20260709 13:00:10.063186  1536 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:10.030598  2933 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:10.030099  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.031658  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.005823  3095 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:10.031812  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.026646  1690 rpcz_store.cc:276] 0709 13:00:08.050129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.953242 (+1903113us) service_pool.cc:224] Handling call
0709 13:00:09.962436 (+  9194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.993294 (+ 30858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.993321 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.002149 (+  8828us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:10.002196 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.002363 (+   167us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.002476 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.027926  3180 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:09.838505  3228 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.035048  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1563) took 1202 ms. Trace:
I20260709 13:00:10.035223  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.083577  1543 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:10.049542  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.035254  3208 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:10.050551  3230 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:10.050948  3117 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:10.051265  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.051576  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.064239  2228 rpcz_store.cc:276] 0709 13:00:08.718944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.722303 (+  3359us) service_pool.cc:224] Handling call
0709 13:00:10.025365 (+1303062us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.051867  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.093452  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.094776  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.051896  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.053172  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.032411  3213 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.035343  3102 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.054150  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.054201  1684 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
W20260709 13:00:10.030856  1639 debug-util.cc:398] Leaking SignalData structure 0x7b08001c4440 after lost signal to thread 1716
I20260709 13:00:10.018791  3158 raft_consensus.cc:2955] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:09.974080  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:10.065963  2201 connection.cc:256] RPC call timeout handler was delayed by -1.0862563720000367s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15200000000000002s for a response
W20260709 13:00:10.115121  2201 connection.cc:256] RPC call timeout handler was delayed by -1.0844563720000338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
I20260709 13:00:10.115741  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.116874  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:09.924024  3182 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:09.963228  3188 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.024966  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.099175  1689 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:10.061256  3138 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:10.082777  1615 rpcz_store.cc:276] 0709 13:00:08.808027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.810640 (+  2613us) service_pool.cc:224] Handling call
0709 13:00:08.810997 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.959754 (+1148757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.959802 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:09.959879 (+    77us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:09.960075 (+   196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.010358 (+ 50283us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540008c0d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:10.010401 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.010517 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2623092608}
I20260709 13:00:10.085769  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.061371  3159 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.091650  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.063463  3214 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:10.077229  3127 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:10.082494  3095 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.098726  1538 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:10.133028  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.085578  3169 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:00:10.064028  1685 rpcz_store.cc:269] 0709 13:00:07.998400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.814576 (+1816176us) service_pool.cc:224] Handling call
0709 13:00:10.015788 (+201212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.102962  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.101652  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:10.105938  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.106417  1535 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:10.086520  3208 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:10.107201  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.077395  2933 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.099428  3230 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:10.092734  3180 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.100545  1540 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:10.119221  2235 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:10.107093  1616 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:10.123323  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.123488  3175 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:10.124130  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.128408  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.066779  1683 rpcz_store.cc:276] 0709 13:00:08.011575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.881142 (+1869567us) service_pool.cc:224] Handling call
0709 13:00:09.949118 (+ 67976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:09.983044 (+ 33926us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:09.989609 (+  6565us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.002832 (+ 13223us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:10.005174 (+  2342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.012529 (+  7355us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.023364 (+ 10835us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.131441  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.129211  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.127553  2229 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:10.136045  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.121486  3162 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:10.139680  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.138943  1537 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:10.177232  1543 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:10.174835  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.154129  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.179320  1683 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:10.179425  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.179831  3170 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:10.180310  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.181320  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1087) took 2130 ms. Trace:
I20260709 13:00:10.155547  3138 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:10.159344  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:10.161132  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 139633us
I20260709 13:00:10.162866  2933 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:10.163086  3180 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:10.158167  3230 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.138175  3169 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:10.143797  1688 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:10.136616  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:10.143832  1639 debug-util.cc:398] Leaking SignalData structure 0x7b080033e3a0 after lost signal to thread 2261
I20260709 13:00:10.171916  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.169467  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.169772  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.156682  3208 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.143630  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.195737  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.195883  1684 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:10.153172  3228 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:10.181015  1683 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 13:00:10.181730  1692 rpcz_store.cc:276] 0709 13:00:08.050420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.180512 (+2130092us) service_pool.cc:224] Handling call
0709 13:00:10.180806 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.180998 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.181024 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.181094 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:10.181124 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.181175 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.181270 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.183058  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.184316  3159 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:10.184767  3189 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:10.164194  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 239) took 1345 ms. Trace:
I20260709 13:00:10.200505  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.206333  1537 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:10.206413  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.186615  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:10.143348  3127 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.191469  3162 raft_consensus.cc:993] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:00:10.191311  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.191641  3182 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:10.197896  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.198884  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.201462  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 231) took 1389 ms. Trace:
I20260709 13:00:10.202898  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 646) took 1473 ms. Trace:
I20260709 13:00:10.199337  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.204229  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 642) took 1486 ms. Trace:
I20260709 13:00:10.189070  3169 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.200064  1542 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 13:00:10.207937  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.185966  3138 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:10.182026  1687 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 13:00:10.362783  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.285969  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.209201  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.193979  1610 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:10.227465  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:10.343145  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.228529  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.510046  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:10.318838  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.513302  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.282716  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.512027  1613 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:10.325140  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1513) took 1604 ms. Trace:
W20260709 13:00:10.380719   436 debug-util.cc:398] Leaking SignalData structure 0x7b080000f700 after lost signal to thread 1658
I20260709 13:00:10.192408  3213 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:10.513290  2236 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:10.509374  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.513142  1538 rpcz_store.cc:276] 0709 13:00:08.811788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.568803 (+757015us) service_pool.cc:224] Handling call
0709 13:00:10.201282 (+632479us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b5400343fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:10.201365 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1217150336}
I20260709 13:00:10.237941  1537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88cbab659ea041148748114fae549ae0"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:53670
I20260709 13:00:10.558714  1692 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
W20260709 13:00:10.380825  1564 debug-util.cc:398] Leaking SignalData structure 0x7b08001046c0 after lost signal to thread 434
W20260709 13:00:10.237752  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 51206.7us
I20260709 13:00:10.561344  2235 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
W20260709 13:00:10.567948  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 57906.8us
I20260709 13:00:10.570199  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.566704  1539 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:10.562670  1537 raft_consensus.cc:606] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:10.578106  1687 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:10.574359  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:10.579502  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 352091us
I20260709 13:00:10.580539  2203 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:10.571528  1543 rpcz_store.cc:276] 0709 13:00:08.720967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.317571 (+1596604us) service_pool.cc:224] Handling call
0709 13:00:10.318064 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.324610 (+  6546us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.324659 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.324732 (+    73us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:10.324779 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.324867 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.325039 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.581275  1584 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:10.515908  3138 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.266444  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.283263  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:10.325982  3236 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:00:10.202935  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.205499  3202 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:10.266554  3168 raft_consensus.cc:1081] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:00:10.589792  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.591228  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.324990  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.593008  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.286545  3180 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:10.593016  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.594516  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.413286  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.397846  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:10.579042  3213 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:10.641273  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.453969  3102 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:10.348539  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.571614  3188 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:10.505139  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1516) took 1732 ms. Trace:
I20260709 13:00:10.592664  3189 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:10.597985  2233 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:10.557461  1535 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:10.639931  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1085) took 2590 ms. Trace:
I20260709 13:00:10.650399  1683 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:10.562707  1540 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:10.644402  1688 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:10.209821  1536 rpcz_store.cc:276] 0709 13:00:08.818784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.567280 (+748496us) service_pool.cc:224] Handling call
0709 13:00:10.164025 (+596745us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b5400343ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:10.164135 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1120393472}
I20260709 13:00:10.587875  3186 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:10.647464  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.505625  3095 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.639361  3180 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:10.711256  1691 rpcz_store.cc:276] 0709 13:00:08.049854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:09.949934 (+1900080us) service_pool.cc:224] Handling call
0709 13:00:09.964962 (+ 15028us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:10.007723 (+ 42761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.007749 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:10.050787 (+ 43038us) log.cc:844] Serialized 564 byte log entry
0709 13:00:10.118217 (+ 67430us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:10.118254 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.120059 (+  1805us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:10.639722 (+519663us) raft_consensus.cc:1685] finished
0709 13:00:10.639741 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.639874 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":66827,"threads_started":1,"wal-append.queue_time_us":115041}
I20260709 13:00:10.648873  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 227) took 1863 ms. Trace:
I20260709 13:00:10.718888  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.715698  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.719220  1539 rpcz_store.cc:276] 0709 13:00:08.785531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.592204 (+1806673us) service_pool.cc:224] Handling call
0709 13:00:10.648293 (+ 56089us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.648513 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.648545 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.648599 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:10.648638 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.648704 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.648813 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.653324  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 228) took 1842 ms. Trace:
I20260709 13:00:10.656677  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1518) took 1931 ms. Trace:
I20260709 13:00:10.656839   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:00:10.661557  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.720484  1543 rpcz_store.cc:276] 0709 13:00:08.810675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.589744 (+1779069us) service_pool.cc:224] Handling call
0709 13:00:10.645453 (+ 55709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.652968 (+  7515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.653003 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.653044 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:10.653081 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.653144 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.653252 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.665239  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:10.651785  1564 debug-util.cc:398] Leaking SignalData structure 0x7b0800103100 after lost signal to thread 1475
I20260709 13:00:10.649752  3202 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:10.639530  2231 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:10.660190  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.712029  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.516479  2229 rpcz_store.cc:276] 0709 13:00:08.718225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.718439 (+   214us) service_pool.cc:224] Handling call
0709 13:00:10.204011 (+1485572us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54002a3354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:10.204166 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514486528}
I20260709 13:00:10.650461  3213 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:10.663398  3189 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:10.512897  2232 rpcz_store.cc:276] 0709 13:00:08.729529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.730551 (+  1022us) service_pool.cc:224] Handling call
0709 13:00:10.202741 (+1472190us) spinlock_profiling.cc:243] Waited 34.2 ms on lock 0x7b54007404d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:10.202842 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78647552}
I20260709 13:00:10.640467  1689 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:10.716864  3180 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.717506  2235 raft_consensus.cc:2393] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:10.717784  3127 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:10.713810  3186 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:10.657284  3230 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.724673  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.725030  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.725345  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.730780  3202 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:10.720592  1542 rpcz_store.cc:276] 0709 13:00:08.724919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.584821 (+1859902us) service_pool.cc:224] Handling call
0709 13:00:10.651728 (+ 66907us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.656335 (+  4607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.656371 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.656437 (+    66us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:10.656465 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.656519 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.656625 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.722452   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:10.715270  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.738592  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.739291  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.738692  3189 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:10.742990  1617 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:10.657864  1534 rpcz_store.cc:276] 0709 13:00:08.723425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.374534 (+1651109us) service_pool.cc:224] Handling call
0709 13:00:10.391552 (+ 17018us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.392473 (+   921us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.392530 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.392608 (+    78us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:10.392660 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.403419 (+ 10759us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.456137 (+ 52718us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.726315  3095 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:10.736570  3213 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:10.742748  1564 debug-util.cc:398] Leaking SignalData structure 0x7b080020f400 after lost signal to thread 1567
I20260709 13:00:10.742221  3186 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:10.764498  1541 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:10.741645  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.743724  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.769693  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:10.763078  1564 debug-util.cc:398] Leaking SignalData structure 0x7b0800044bc0 after lost signal to thread 1568
I20260709 13:00:10.772360  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.758293  3202 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.778365  2234 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:10.778151  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.744386  3169 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:10.783686  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.779783  1615 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:10.784673  3230 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:10.785498  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.785645  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 229) took 1974 ms. Trace:
I20260709 13:00:10.785686  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.786264  1538 rpcz_store.cc:276] 0709 13:00:08.811107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.784692 (+1973585us) service_pool.cc:224] Handling call
0709 13:00:10.785031 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.785228 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.785256 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.785320 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:10.785357 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.785439 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.785565 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.788313  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.788426  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 233) took 1976 ms. Trace:
I20260709 13:00:10.789922  1538 rpcz_store.cc:276] 0709 13:00:08.812461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.787557 (+1975096us) service_pool.cc:224] Handling call
0709 13:00:10.787854 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.788092 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.788122 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.788178 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:10.788212 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.788267 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.788371 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.791471  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 226) took 2010 ms. Trace:
I20260709 13:00:10.791694  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 235) took 1974 ms. Trace:
I20260709 13:00:10.791903  1541 rpcz_store.cc:276] 0709 13:00:08.781564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.590688 (+1809124us) service_pool.cc:224] Handling call
0709 13:00:10.642167 (+ 51479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.766623 (+124456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.791200 (+ 24577us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54001e5354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:10.791384 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278741888}
I20260709 13:00:10.792127  1538 rpcz_store.cc:276] 0709 13:00:08.817617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.790922 (+1973305us) service_pool.cc:224] Handling call
0709 13:00:10.791227 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.791405 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.791430 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.791471 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:10.791501 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.791550 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.791644 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.773337  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:10.799966  1564 debug-util.cc:398] Leaking SignalData structure 0x7b0800227e60 after lost signal to thread 1642
I20260709 13:00:10.794876  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 240) took 1318 ms. Trace:
I20260709 13:00:10.777272  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.803597  1538 rpcz_store.cc:276] 0709 13:00:09.476403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.793283 (+1316880us) service_pool.cc:224] Handling call
0709 13:00:10.793592 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.793824 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.793852 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.793893 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:10.793922 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.793972 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.794505 (+   533us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.826467  2237 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:10.831264  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.774307  1535 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:10.841032  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 238) took 1994 ms. Trace:
W20260709 13:00:10.801283  1564 debug-util.cc:398] Leaking SignalData structure 0x7b080030e8c0 after lost signal to thread 1716
I20260709 13:00:10.872045  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2015) took 1287 ms. Trace:
I20260709 13:00:10.872462  1539 rpcz_store.cc:276] 0709 13:00:09.584808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.805612 (+1220804us) service_pool.cc:224] Handling call
0709 13:00:10.840329 (+ 34717us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.861533 (+ 21204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.861562 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.871755 (+ 10193us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:10.871803 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.871868 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.871988 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40576}
I20260709 13:00:10.852986  1541 rpcz_store.cc:276] 0709 13:00:08.818491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.793081 (+1974590us) service_pool.cc:224] Handling call
0709 13:00:10.793369 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.793934 (+   565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.793959 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.812298 (+ 18339us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:10.812355 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.812437 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.812597 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.800623  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2010) took 1217 ms. Trace:
W20260709 13:00:10.877319  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 296067us
I20260709 13:00:10.809382  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:10.881958  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 289372us
I20260709 13:00:10.884961  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2019) took 1296 ms. Trace:
I20260709 13:00:10.885391  1543 rpcz_store.cc:276] 0709 13:00:09.588782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.884174 (+1295392us) service_pool.cc:224] Handling call
0709 13:00:10.884479 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.884659 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.884684 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.884725 (+    41us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:10.884754 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.884804 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.884903 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.897105  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.898741  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:10.899505  1582 connection.cc:256] RPC call timeout handler was delayed by -2.1548122479999847s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1539s for a response
W20260709 13:00:10.899911  1582 connection.cc:256] RPC call timeout handler was delayed by -1.7012122479999903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
W20260709 13:00:10.862026  1564 debug-util.cc:398] Leaking SignalData structure 0x7b080028f140 after lost signal to thread 1717
I20260709 13:00:10.902608  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:10.889107  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2020) took 1300 ms. Trace:
I20260709 13:00:10.908541  1537 rpcz_store.cc:276] 0709 13:00:09.589061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.888306 (+1299245us) service_pool.cc:224] Handling call
0709 13:00:10.888597 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.888798 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.888824 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.888864 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:10.888895 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.888953 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.889055 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.914098  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:10.914069  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.870765  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.915300  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.917372  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.918427  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.920727  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2016) took 1335 ms. Trace:
I20260709 13:00:10.923899  1538 rpcz_store.cc:276] 0709 13:00:09.585076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.827819 (+1242743us) service_pool.cc:224] Handling call
0709 13:00:10.857583 (+ 29764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.861868 (+  4285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.861900 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.861944 (+    44us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:10.861973 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.862039 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.920367 (+ 58328us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b5400343fd0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:10.920632 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39691392}
I20260709 13:00:10.927937  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.929858  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.932134  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.870708  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:10.934800  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.886519  1657 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:10.945118  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2018) took 1330 ms. Trace:
W20260709 13:00:10.945461  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 59011.6us
I20260709 13:00:10.945513  1541 rpcz_store.cc:276] 0709 13:00:09.588475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.875874 (+1287399us) service_pool.cc:224] Handling call
0709 13:00:10.911829 (+ 35955us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.918288 (+  6459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.918322 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.918371 (+    49us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:10.918402 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.918465 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.918591 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.935758  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.918236  3240 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:10.939601  3241 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:10.940544  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2028) took 1092 ms. Trace:
I20260709 13:00:10.941464  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:10.944067  1582 connection.cc:256] RPC call timeout handler was delayed by -1.649912248000021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
I20260709 13:00:10.942936  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.950482  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:10.952661  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.952805  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.887526  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.956635  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.961519  2237 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:10.958369  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.959127  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2017) took 1373 ms. Trace:
I20260709 13:00:10.962491  1539 rpcz_store.cc:276] 0709 13:00:09.585346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.873465 (+1288119us) service_pool.cc:224] Handling call
0709 13:00:10.910343 (+ 36878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.950004 (+ 39661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.950049 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.958838 (+  8789us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:10.958878 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.958936 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.959049 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:10.966084  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.952868  3184 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:10.952164  3240 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:10.952625  3241 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:10.955497  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.985841  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1526) took 1093 ms. Trace:
I20260709 13:00:10.988036  1541 rpcz_store.cc:276] 0709 13:00:09.892619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.946526 (+1053907us) service_pool.cc:224] Handling call
0709 13:00:10.954505 (+  7979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.985511 (+ 31006us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.985543 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.985591 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:10.985621 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.985680 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.985789 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:10.985476  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 110441us
I20260709 13:00:10.986099  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:10.964660  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:10.996721  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:10.996515  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:10.978875  1564 debug-util.cc:398] Leaking SignalData structure 0x7b08002e4bc0 after lost signal to thread 2261
I20260709 13:00:10.994403  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:10.997300  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:11.000197  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:11.001467  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2037) took 1053 ms. Trace:
I20260709 13:00:10.879837  1536 rpcz_store.cc:276] 0709 13:00:09.583410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.799728 (+1216318us) service_pool.cc:224] Handling call
0709 13:00:10.800075 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.800281 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.800310 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.800358 (+    48us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:10.800388 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.800445 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.800566 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:11.021953  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:11.022149  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1527) took 1083 ms. Trace:
I20260709 13:00:11.000775  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:10.996558  2229 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 2: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:11.023430  2203 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:10.999176  1564 debug-util.cc:398] Leaking SignalData structure 0x7b0800303ee0 after lost signal to thread 2262
I20260709 13:00:11.033072  1509 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:10.998826  1684 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:10.981478  3184 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:10.954005  1618 raft_consensus.cc:2955] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:11.031042  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:10.994393  3241 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.995285  3240 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:10.952140  1540 rpcz_store.cc:276] 0709 13:00:09.848594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.939653 (+1091059us) service_pool.cc:224] Handling call
0709 13:00:10.939998 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:10.940208 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:10.940240 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:10.940282 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:10.940316 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:10.940371 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:10.940473 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:11.033999  1539 rpcz_store.cc:276] 0709 13:00:09.938920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.969618 (+1030698us) service_pool.cc:224] Handling call
0709 13:00:10.994626 (+ 25008us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.021802 (+ 27176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:11.021832 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:11.021879 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:11.021910 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.021976 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:11.022094 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:11.034848  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:11.034895  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.035557  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:11.032465  1542 rpcz_store.cc:276] 0709 13:00:09.948326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.973362 (+1025036us) service_pool.cc:224] Handling call
0709 13:00:10.996846 (+ 23484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.001095 (+  4249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:11.001122 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:11.001161 (+    39us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:11.001190 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.001245 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:11.001369 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:11.037222  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:11.037755  3159 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.038259  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:11.039516  3189 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.041093  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:11.043556  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:11.044495  1508 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:11.047982  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 456009us
W20260709 13:00:11.027834  3208 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.833s	user 0.009s	sys 0.003s
W20260709 13:00:11.027832  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 74819.2us
I20260709 13:00:11.048404  3188 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:11.050465  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 98693.6us
I20260709 13:00:11.068881  2201 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:11.009893  1689 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:11.068332  3188 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:11.072331  3095 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.072777  3188 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.057013  2233 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 13:00:11.053936  3184 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.053716  3228 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.063598  2228 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:11.063964  3182 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:11.069555  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:11.032006  3169 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:11.059259  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.084457  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:11.086400  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:11.073731  2201 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:11.052300  1564 debug-util.cc:398] Leaking SignalData structure 0x7b0800310f00 after lost signal to thread 3058
W20260709 13:00:11.087352  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45524.5us
I20260709 13:00:11.089270  2237 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:11.089740  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:11.065455  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2027) took 1214 ms. Trace:
I20260709 13:00:11.074133  3208 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.090886  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:11.072854  3095 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:11.082147  3170 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.082700  3228 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:11.103704  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:11.091516  3117 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:11.086071  3182 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:11.092600  3162 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:11.108110  2236 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 2: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:00:11.101692  1564 debug-util.cc:398] Leaking SignalData structure 0x7b08003089a0 after lost signal to thread 3139
I20260709 13:00:11.095507  3243 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.109484  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:11.110843  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 51681.6us
I20260709 13:00:11.101184  3095 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:11.126293  1564 debug-util.cc:398] Leaking SignalData structure 0x7b0800367b20 after lost signal to thread 3141
I20260709 13:00:11.104679  3115 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:11.112445  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 48898.3us
W20260709 13:00:11.150910  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 43660.4us
I20260709 13:00:11.151643  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:11.107949  3168 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:11.107278  3228 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:11.148912  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 59596.8us
W20260709 13:00:11.167028  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 79082.1us
W20260709 13:00:11.135557  1564 debug-util.cc:398] Leaking SignalData structure 0x7b08002dce60 after lost signal to thread 3146
I20260709 13:00:11.111783  3182 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.084395  3127 consensus_queue.cc:1059] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:00:11.108130  3117 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:11.196293  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 69380.7us
W20260709 13:00:11.197154  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 87777.1us
I20260709 13:00:11.183910  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:11.184991  3261 consensus_queue.cc:1059] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:11.111908  3162 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:11.186264  3168 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:11.110857  3255 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.106477  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:11.133929  3243 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:11.190651  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 63413.7us
I20260709 13:00:11.198252  1582 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.199070  3246 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.200743  1583 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:11.202440  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 50854.6us
I20260709 13:00:11.134925  3159 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.158236  3115 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:11.190382  3260 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.134348  3180 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.214191  3260 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:11.214605  3260 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.191493  2230 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:11.201189  1564 debug-util.cc:398] Leaking SignalData structure 0x7b08003692e0 after lost signal to thread 3166
I20260709 13:00:11.201433  3117 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:11.096030  1535 rpcz_store.cc:276] 0709 13:00:09.848285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.913771 (+1065486us) service_pool.cc:224] Handling call
0709 13:00:10.980425 (+ 66654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.022921 (+ 42496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:11.027743 (+  4822us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:11.036081 (+  8338us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:11.049167 (+ 13086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:11.053486 (+  4319us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:11.062223 (+  8737us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:11.212148  1509 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:11.213726  3261 raft_consensus.cc:3055] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:11.204638  3168 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:11.203902  3162 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.228587  3246 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.229010  1583 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:11.229727  1582 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.206198  3255 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:11.209350  3243 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:11.255293  3158 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.085s	user 0.012s	sys 0.008s
I20260709 13:00:11.215631  3115 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.221995  3127 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:11.210995  3159 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:11.222333  3249 raft_consensus.cc:3055] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:11.231323  3256 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.229869  3186 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:11.286428  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 212752us
I20260709 13:00:11.261528  3255 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.255959  3261 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:11.216219  3213 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.274255  3159 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.268944  3127 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:11.274794  3249 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:11.279093  3246 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:11.301854  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 72916.1us
I20260709 13:00:11.277714  3247 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.302054  1687 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.302872  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:11.291954  3257 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.324978  1583 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:11.293783  3208 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:11.330636  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 120267us
I20260709 13:00:11.298632  3127 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:11.331535  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 147661us
W20260709 13:00:11.334336  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 122303us
I20260709 13:00:11.337189  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:11.309885  3247 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:11.342845  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:11.327958  3189 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:11.372926  3274 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:11.373631  3245 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.344408  3268 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:11.414825  1507 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.531s (SENT)
I20260709 13:00:11.416254  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:11.336028  3266 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.414850  1657 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:11.451009  2203 leader_election.cc:336] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.522s (SENT)
I20260709 13:00:11.446830  1507 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:11.341825  3247 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.458600  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:11.452816  3182 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.371135  3265 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.465451  2203 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.466787  3273 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.482931  3127 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.487227  3127 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:11.484555  3245 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:11.462049  3138 consensus_meta.cc:287] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.809s	user 0.011s	sys 0.002s
I20260709 13:00:11.495126  3138 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:11.497375  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 157499us
I20260709 13:00:11.491544  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:11.492650  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:11.494645  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 228922us
I20260709 13:00:11.455627  3266 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:11.488734  3263 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.463148  3265 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:11.503108  1582 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.516281  1582 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.518081  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.521967  3274 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.522879  3274 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.523669  3274 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.533177  1582 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.502019  3213 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:11.511508  3266 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.534590  3138 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:11.585667  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:11.518091  3265 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:11.591187  3239 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.058s	user 0.001s	sys 0.007s
I20260709 13:00:11.528105  3269 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.585582  2231 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:11.642570  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 303833us
I20260709 13:00:11.595443  3186 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:11.491321  3261 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:11.659735  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 364310us
I20260709 13:00:11.663764  3095 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.672888  2201 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:11.672331  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 421823us
I20260709 13:00:11.675508  2201 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:11.677371  2201 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:11.679050  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:11.679672  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 147063us
I20260709 13:00:11.680927  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.682108  1582 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:11.682694  3127 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:11.676045  3271 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.688282  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:11.684478  3273 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:11.684948  3127 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:11.684180  3270 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.685098  3277 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.701012  1658 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:11.695302  3265 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.694710  3273 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:11.721738  3273 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:11.722097  1543 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.692221  3182 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:11.714210  3251 raft_consensus.cc:993] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: : Instructing follower 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 13:00:11.714507  3278 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.715580  3159 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.717965  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:11.719013  1535 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.720026  3265 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:11.685237  3281 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.702203  3127 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:11.730693  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:11.723245  3251 raft_consensus.cc:1081] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Signalling peer 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 13:00:11.694607  3286 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:11.724514  3202 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.913s	user 0.009s	sys 0.004s
W20260709 13:00:11.732721  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 48548us
I20260709 13:00:11.725915  3279 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.742884  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:11.725412  3278 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:11.729005  3267 consensus_queue.cc:1059] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:00:11.744313  3202 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:11.756036  3202 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:11.747975  3278 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.760402  3168 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:11.771549  3127 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.777547  3247 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.778170  3127 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:11.780548  3243 consensus_meta.cc:287] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.000s
I20260709 13:00:11.743965  3279 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:11.781661  3279 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.751699  3288 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.744144  3272 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.793467  3159 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:11.804301  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 121052us
I20260709 13:00:11.807410  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.710948  3263 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:00:11.809608  3275 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.810130  3272 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:11.810570  3272 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:11.811374  3272 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.812664  3272 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:11.814122  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.781172  3243 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:11.814708  1509 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.531s (SENT)
W20260709 13:00:11.824285  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 143377us
I20260709 13:00:11.784694  3162 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:11.827360  1507 leader_election.cc:336] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.561s (SENT)
I20260709 13:00:11.827855  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:11.828477  2201 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.831341  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:11.832209  3102 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:11.832391  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:11.832767  3102 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:11.833078  3277 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.791057  3255 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:11.837633  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:11.802812  3291 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:11.845180  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.835053  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.817499  3275 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.849493  2235 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.851994  1613 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:11.853041  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.854718  2237 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:11.855371  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.855720  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.857110  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.858594  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.860023  2233 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:11.856230  1617 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:11.833338  3102 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:11.850672  3115 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.571s	user 0.006s	sys 0.005s
I20260709 13:00:11.862841  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.866047  3266 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.866555  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.869683  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.846582  3277 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:11.882696  1618 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:11.872040  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.870316  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:11.825898  3260 consensus_meta.cc:287] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.610s	user 0.004s	sys 0.008s
I20260709 13:00:11.847086  3159 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:11.880468  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 53119.3us
I20260709 13:00:11.881489  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.872537  2234 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:11.894424  1507 leader_election.cc:336] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.593s (SENT)
I20260709 13:00:11.851279  3291 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:11.894860  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.895807  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.895936  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:11.862363  3298 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:11.883900  3228 consensus_meta.cc:287] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.681s	user 0.013s	sys 0.004s
I20260709 13:00:11.901661  2228 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:11.899264  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:11.902580  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 70587.4us
I20260709 13:00:11.903630  2201 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.905349  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:11.885080  3277 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.906126  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:11.907650  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:11.904723  1615 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:11.905761  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.891319  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:11.898890  3241 consensus_meta.cc:287] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.837s	user 0.011s	sys 0.002s
I20260709 13:00:11.968422  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:11.969399  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:11.971149  1690 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.972878  2203 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:11.892562  3260 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:11.929117  1509 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.791s (SENT)
I20260709 13:00:11.912984  1614 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:11.909482  1508 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:11.981652  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 52591.9us
I20260709 13:00:11.981941  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:11.915706  2236 raft_consensus.cc:2955] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:11.911285  3228 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:11.913086  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:11.985069  1688 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:11.960322  3184 consensus_meta.cc:287] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.842s	user 0.008s	sys 0.003s
I20260709 13:00:11.911263  3247 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:11.914229  3127 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:11.903718  3296 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:11.903131  3291 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.914028  3168 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:11.913720  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:11.961843  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.962445  3241 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.006526  3241 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:11.977650  3159 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.006974  3255 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:12.007205  3159 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:12.007659  3159 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:11.994144  3184 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.009501  3184 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:12.009641  1616 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.987442  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:11.978809  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:11.964062  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:11.938673  3288 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:11.905511  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.020768  1543 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:11.980895  3273 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:11.984416  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 75029.8us
I20260709 13:00:11.907078  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:11.988801  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:11.885386  3115 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:11.995079  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 252267us
I20260709 13:00:12.011230  1692 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:11.978843  3278 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.012524  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:12.014046  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:12.015388  3117 consensus_meta.cc:287] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.759s	user 0.001s	sys 0.008s
I20260709 13:00:11.990850  3298 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.016136  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:12.035130  2237 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.016690  3266 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:12.003073  3267 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:00:12.017637  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:11.982563  1509 leader_election.cc:336] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.806s (SENT)
W20260709 13:00:12.037782  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 55261.5us
I20260709 13:00:12.022440  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.023092  3162 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:12.023686  3188 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.950s	user 0.018s	sys 0.000s
W20260709 13:00:12.023878  3240 consensus_meta.cc:287] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.957s	user 0.005s	sys 0.005s
I20260709 13:00:12.025679  1688 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:12.027513  1508 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:12.052417  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.055291  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.055171  3188 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.055635  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.056017  1618 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:12.056864  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.031756  1536 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.031951  3115 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:12.032377  3243 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:12.021739  2201 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:12.127270  3228 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:12.127594  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 105908us
I20260709 13:00:12.128854  2201 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:12.130956  1543 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:12.131577  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:12.037657  3127 raft_consensus.cc:3055] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.028688  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:12.057855  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.141733  3127 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:12.060158  3285 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:00:12.142269  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 113660us
I20260709 13:00:12.142685  3285 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:12.034937  3117 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.143054  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:12.143214  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:12.143957  2234 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 2: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:12.144241  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.144788  3117 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:12.056924  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.075548  3240 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.145408  3117 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:12.146008  3272 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:12.149396  3102 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.165854  3117 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:12.166173  3272 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:12.166415  3117 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.166625  3272 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.122480  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:12.123572  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.147805  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 253) took 1149 ms. Trace:
I20260709 13:00:12.168067  1542 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.154176  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:12.168650  2201 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:12.040429  3180 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:12.154433  3267 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.130470  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.170109  3180 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.170799  3261 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.078119  3188 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:12.133352  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:12.080751  3278 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:12.134876  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:12.132447  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:12.145328  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:12.181545  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:12.155120  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:12.168113  3102 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:12.187515  2232 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:12.188166  3249 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.189347  3249 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.185518  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.168082  1687 rpcz_store.cc:276] 0709 13:00:10.998171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.013955 (+ 15784us) service_pool.cc:224] Handling call
0709 13:00:11.057848 (+ 43893us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b5800010e98. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:00:11.058096 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:11.302806 (+244710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:12.146394 (+843588us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b540003f054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:12.147544 (+  1150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":565440000}
I20260709 13:00:12.126921  3279 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.130240  3159 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.176190  1689 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:12.176306  1618 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 3: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:12.147404  3242 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.180660  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:12.185082  3240 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:12.185506  3260 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:12.185648  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 53289.7us
I20260709 13:00:12.150703  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.186340  1610 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 3: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:12.171062  1609 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:12.187033  3268 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:00:12.180379  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 140351us
I20260709 13:00:12.155872  3303 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:12.192958  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:12.205181  1509 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.192783  3273 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:12.194841  3279 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:12.194998  3298 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:12.195864  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.196487  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.143214  3285 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.148522  1684 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.197746  2235 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:12.185715  3115 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.147584  1536 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:12.182091  3259 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:12.201570  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.201488  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.142721  3127 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.196346  3278 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.185257  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:12.207847  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:12.193696  3246 raft_consensus.cc:3055] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:12.209098  3260 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.210332  1618 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 3: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:00:12.212002  1509 leader_election.cc:336] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.032s (SENT)
I20260709 13:00:12.222247  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:12.192010  2231 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:12.209159  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:12.221683  3246 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:12.227697  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:12.223531  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.226467  1584 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:12.225263  3243 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.213876  3287 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.211330  3268 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:12.211619  3303 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:12.213821  3115 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:12.228844  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:12.224579  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.235200  3278 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.270246  3278 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:12.270655  3278 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.271034  1609 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.225967  3260 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:00:12.273178  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44415.3us
W20260709 13:00:12.274714  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 62742.2us
I20260709 13:00:12.233982  3243 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:12.299923  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.315430  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.317953  3168 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.224341  3259 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.320401  1542 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:12.320595  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.321529  2203 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:12.268936  3188 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:12.314874  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.278494  3288 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.272208  3115 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.327389  1616 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:12.330049  3188 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:12.332603  3117 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.321830  3242 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.322708  3243 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.279771  3260 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.325112  3273 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:12.325356  3168 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.301532  3257 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.337333  3288 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:12.367750  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.369724  3159 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:12.370252  1440 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 338) took 6403 ms. Trace:
I20260709 13:00:12.370498  3117 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:12.371104  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:11.782235   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:12.385248  3278 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.388468  3278 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:12.347177  3168 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:12.398468  1543 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.401113  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:12.401836  2203 leader_election.cc:336] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.809s (SENT)
I20260709 13:00:12.340780  3267 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:00:12.360695  3288 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.409816  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.410530  1542 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.370637  3188 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.420715  1534 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.319422  3246 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.377605  3273 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:12.335534  3303 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.339851  3268 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.427114  3278 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 13:00:12.432399  3249 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:00:12.405864  3168 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.442721  3278 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:12.443136  3278 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:12.370663  1440 rpcz_store.cc:276] 0709 13:00:05.966786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:05.967250 (+   464us) service_pool.cc:224] Handling call
0709 13:00:12.370174 (+6402924us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:10.109546 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:10.174431 (+ 64885us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:10.174488 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:00:10.174514 (+    26us) tablet.cc:662] Decoding operations
0709 13:00:10.174930 (+   416us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:10.175023 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:10.175052 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:00:10.190781 (+ 15729us) tablet.cc:701] Row locks acquired
0709 13:00:10.199102 (+  8321us) write_op.cc:260] PREPARE: finished
0709 13:00:10.285151 (+ 86049us) write_op.cc:270] Start()
0709 13:00:10.285344 (+   193us) write_op.cc:276] Timestamp: P: 1783602010285056 usec, L: 0
0709 13:00:10.285387 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 13:00:10.395106 (+109719us) log.cc:844] Serialized 599 byte log entry
0709 13:00:11.052077 (+656971us) op_driver.cc:464] REPLICATION: finished
0709 13:00:11.200539 (+148462us) write_op.cc:301] APPLY: starting
0709 13:00:11.204452 (+  3913us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:11.215443 (+ 10991us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:11.282159 (+ 66716us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:11.330678 (+ 48519us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:11.418870 (+ 88192us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:00:11.453539 (+ 34669us) write_op.cc:312] APPLY: finished
0709 13:00:11.503433 (+ 49894us) log.cc:844] Serialized 23 byte log entry
0709 13:00:11.536068 (+ 32635us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:11.633361 (+ 97293us) write_op.cc:454] Released schema lock
0709 13:00:11.645365 (+ 12004us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:11.684071 (+ 38706us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:11.686249 (+  2178us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":682146,"reported_tablets":1,"spinlock_wait_cycles":43904,"child_traces":[["op",{"apply.queue_time_us":146763,"apply.run_cpu_time_us":21130,"apply.run_wall_time_us":476575,"num_ops":1,"prepare.queue_time_us":92383,"prepare.run_cpu_time_us":75043,"prepare.run_wall_time_us":625623,"raft.queue_time_us":202223,"raft.run_cpu_time_us":14821,"raft.run_wall_time_us":153266,"replication_time_us":744522,"thread_start_us":368136,"threads_started":4,"wal-append.queue_time_us":354884}]]}
I20260709 13:00:12.443976  3278 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.445221  3278 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.449649  3095 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:12.450511  3168 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.453488  3168 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:12.455127  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:12.456557  1687 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.459713  1508 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.460446  3287 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.465337  2201 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:12.465582  1687 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.471756  3273 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:12.483645  3277 consensus_meta.cc:287] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.000s
I20260709 13:00:12.487314  3272 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:12.497880  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 57402.3us
W20260709 13:00:12.500206  2203 leader_election.cc:336] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.838s (SENT)
I20260709 13:00:12.500638  2203 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:12.503211  3271 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.506726  3170 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:12.507107  3180 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.507313  3170 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.507637  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:12.507854  3180 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:12.508849  3180 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.509490  3127 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.510264  3170 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:12.511071  1611 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.510789  3290 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:12.511768  3290 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.513983  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 671) took 1057 ms. Trace:
I20260709 13:00:12.514389  2229 rpcz_store.cc:276] 0709 13:00:11.456464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.458391 (+  1927us) service_pool.cc:224] Handling call
0709 13:00:12.513257 (+1054866us) spinlock_profiling.cc:243] Waited 6.72 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:00:12.513469 (+   212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:12.513669 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:12.513697 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:12.513738 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:12.513772 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:12.513830 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:12.513928 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15464192}
I20260709 13:00:12.509171  3180 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:12.491189  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:12.523619  1542 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
W20260709 13:00:12.520341  2203 leader_election.cc:336] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.846s (SENT)
I20260709 13:00:12.530349  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:12.532554  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:12.532727  2203 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.883s (SENT)
I20260709 13:00:12.532625  1687 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:12.533952  2203 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.884s (SENT)
I20260709 13:00:12.540288  1688 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.393484  3291 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.542753  1582 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:12.550148  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:12.551152  1691 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:12.539075  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:12.552652  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:12.554275  1692 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:12.518752  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:12.559815  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.559919  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:12.560302  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:12.560604  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:12.510810  1610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1100) took 1736 ms (client timeout 2032 ms). Trace:
I20260709 13:00:12.559815  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.484802  3277 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.563374  1690 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 2: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:00:12.571672  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.571911  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:12.571091  1687 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.552700  3169 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.553129  3256 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.577234  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.584268  3243 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.584928  2228 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.586898  1613 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.574990  3186 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:00:12.576316  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:12.588405  3186 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:12.588732  3186 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:12.588965  3294 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.589610  3186 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.590837  1614 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88cbab659ea041148748114fae549ae0"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:43398
I20260709 13:00:12.591475  1614 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:12.591797  1614 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:12.593394  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:12.578349  3288 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.595149  2236 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.596187  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:12.596460  1542 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.578685  3260 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:12.598342  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:12.599177  3260 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:12.556555  3253 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.578550  3180 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.600626  3180 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.601362  1611 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.603235  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.603900  1611 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 13:00:12.604816  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.579185  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.601636  2237 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.562084  3241 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
W20260709 13:00:12.564177  1610 rpcz_store.cc:269] 0709 13:00:10.774454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:10.797770 (+ 23316us) service_pool.cc:224] Handling call
0709 13:00:12.510603 (+1712833us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54002bd754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:12.510752 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2840570752}
I20260709 13:00:12.607339  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.575398  3159 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.594583  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:12.607645  1611 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 3: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:12.611225  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:12.614821  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:12.575197  3202 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:00:12.620018  3259 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:12.620266  1438 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40172 (request call id 227) took 1579 ms. Trace:
I20260709 13:00:12.622726  1614 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.645455  1613 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 4. Ops: []
I20260709 13:00:12.550690  3095 consensus_queue.cc:1059] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:00:12.647426  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:12.620594  1438 rpcz_store.cc:276] 0709 13:00:11.041180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:11.041363 (+   183us) service_pool.cc:224] Handling call
0709 13:00:11.172098 (+130735us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:00:12.620181 (+1448083us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":223591,"reported_tablets":1,"spinlock_wait_cycles":251002240}
I20260709 13:00:12.649545  1614 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:12.666953  1689 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:12.608047  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.669245  1688 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.664287  1543 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.671811  1543 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.672931  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:12.673226  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:12.673415  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:12.674010  1689 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:12.593674  1535 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:12.673581  3247 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:12.681805  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.670214  1541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54764
I20260709 13:00:12.682647  1543 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:12.682890  1541 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:00:12.683398  1541 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.622400  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:12.675554  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.689900  1692 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:12.690415  1507 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.791s (SENT)
I20260709 13:00:12.690881  1507 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:12.691854  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:12.692088  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:12.676666  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 65316.5us
I20260709 13:00:12.692502  1687 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:12.692709  1688 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:12.659204  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:12.646250  3189 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.679351  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:12.692401  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:12.667349  1683 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.694011  1542 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:12.695065  1615 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.697074  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:12.674666  3251 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:12.692294  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:12.706018  1539 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.666985  3272 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:12.693470  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:12.666606  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:12.658341  3257 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:00:12.680160  1616 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.683089  3186 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:12.710686  1507 leader_election.cc:336] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.841s (SENT)
I20260709 13:00:12.716182  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:12.711598  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 52479.2us
W20260709 13:00:12.712401  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 120410us
I20260709 13:00:12.717357  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:12.694391  3189 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:12.700177  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:12.718047  1583 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.682406  3267 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
W20260709 13:00:12.723616  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 50978.8us
I20260709 13:00:12.704265  3287 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:12.721088  3266 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.728529  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:12.709249  3251 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:12.714632  3290 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:12.713544  3202 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:00:12.710057  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:12.732170  1537 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:12.736032  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:12.722285  3277 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:12.751531  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.721998  3189 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.730044  3241 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.749846  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:12.730752  3273 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.724632  3308 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.756120  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.783948  3253 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:12.731895  3188 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:12.788232  1509 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.737241  3169 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.780042  1685 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:12.759222  3115 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.781127  3230 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.742569  3251 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.731515  3254 consensus_queue.cc:1059] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 13:00:12.762051  3288 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:12.767464  3291 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:12.766748  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.861485  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.862709  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.862991  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.863698  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.864764  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.800500  3246 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.753450  3266 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:00:12.732646  3245 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.773247  2231 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:12.816126  3189 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.874418  3254 raft_consensus.cc:3055] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:12.874564  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:12.874902  3254 raft_consensus.cc:740] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:12.875133  3189 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:12.816674  2233 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:12.875329  1536 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:12.875854  3254 consensus_queue.cc:260] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.876461  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.817462  1541 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.818073  3240 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.878161  3240 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:12.879163  1541 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:12.878964  3240 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.811345  3294 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.810742  2232 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.880122  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.880200  3240 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.818172  1616 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:12.807121  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.774974  3300 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.881788  3300 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:12.852597  3230 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:12.882138  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.882974  3266 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.883419  3266 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:12.883147  3300 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.883832  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.794170  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:12.884464  3300 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.884869  1689 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:12.884291  3266 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.772121  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:12.782945  1437 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55374 (request call id 319) took 4736 ms. Trace:
I20260709 13:00:12.881233  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.777058  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:12.821611  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 71843.2us
I20260709 13:00:12.886722  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:12.882305  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:12.822113  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 81655.5us
W20260709 13:00:12.828780  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 98240.8us
I20260709 13:00:12.831377  3287 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:00:12.836851  3306 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.080s	user 0.002s	sys 0.000s
I20260709 13:00:12.838114  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.891547  1613 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:12.839818  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.840144  1535 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:12.897480  3277 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:00:12.898507  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.899513  3115 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:12.899849  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:12.840369  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.836915  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.840523  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.841794  3259 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:12.842933  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.843475  3260 consensus_queue.cc:1059] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:12.907521  3243 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:12.907603  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.908542  1614 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 2: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:00:12.908640  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 65791.4us
I20260709 13:00:12.909389  1658 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.847865  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.910612  3291 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.910900  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.911073  1658 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.911255  3291 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.911679  1618 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.911849  3288 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:12.907023  3189 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:12.913075  1617 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.827181  3159 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.831619  3257 consensus_queue.cc:1059] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:00:12.913614  3159 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:12.857633  3170 raft_consensus.cc:3055] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.914063  3159 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.913923  3251 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.914386  3170 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:12.915477  3170 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.916391  3250 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.916514  3251 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:12.916837  3250 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:12.916957  3170 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.917373  3272 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.917729  3250 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.918459  3260 raft_consensus.cc:3055] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.918839  3260 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:12.919040  3250 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.907053  1534 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:12.921334  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:12.864590  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.790755  3303 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.850981  3318 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:12.922387  1543 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.869306  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.926844  3303 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:12.927069  1612 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:12.926928  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.879426  3271 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.927940  3271 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:12.927757  3303 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.928768  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:12.928846  3271 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.929596  1685 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.930190  3271 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.849157  3188 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.914153  3257 raft_consensus.cc:3055] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:12.931367  3257 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:12.932418  3188 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:12.789371  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:12.884473  1616 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 13:00:12.933540  1683 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:12.763599  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.885432  1437 rpcz_store.cc:276] 0709 13:00:08.046427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:08.820387 (+773960us) service_pool.cc:224] Handling call
0709 13:00:12.782758 (+3962371us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":740693,"reported_tablets":1}
I20260709 13:00:12.934485  1689 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:12.889818  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:12.935846  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 163815us
I20260709 13:00:12.892380  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:12.883253  3230 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.942451  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:12.907861  3259 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:12.959668  3259 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:12.960398  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:12.960642  3259 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.961611  3272 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.961997  3259 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.909564  1614 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:12.829697  3285 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.607s	user 0.005s	sys 0.007s
I20260709 13:00:12.868168  1537 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:12.911693  3291 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:12.919716  3260 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.925993  3302 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.932289  3257 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:12.952538  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:12.945637  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.877290  3254 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:12.952292  3189 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.954496  3159 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:12.955044  3230 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.955205  3242 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:12.928846  3303 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.957507  3290 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:12.957737  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:12.994616  1689 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:12.969300  3095 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.959075  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:12.967108  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:12.967593  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:12.964637  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 112968us
I20260709 13:00:12.963025  3294 consensus_queue.cc:1059] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:00:12.963416  3188 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
W20260709 13:00:12.984517  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:12.920652  3277 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:00:12.962723  3253 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:00:12.990224  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:12.991802  1614 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:12.969488  3291 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:12.974628  3102 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.998793  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:12.996543  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:13.016389  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:13.017515  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.017618  3168 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.018348  3168 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:13.018620  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:13.019748  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:13.021009  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.021610  3117 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.021911  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:13.022293  3117 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.022846  3117 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.023000  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:13.023509  3117 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:13.024080  3117 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.993633  3242 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:12.997408  3290 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:12.999995  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:13.000620  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:13.001611  3272 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:12.970803  3302 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:12.936321  3314 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:00:12.974081  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:13.004963  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:13.005290  3294 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:12.970289  3285 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.024734  3262 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:13.010210  2203 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:13.012070  3308 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.971642  1583 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:13.007844  3257 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.070292  3308 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:13.075449  3308 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:13.006111  3189 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.076725  3308 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:13.069998  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 59859.8us
I20260709 13:00:13.082252  1657 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.083663  1537 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.084568  3117 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:12.976854  3256 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
W20260709 13:00:12.978246  3243 consensus_queue.cc:1175] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" responder_term: 2 status { last_received { term: 2 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 2 index: 2 } }
W20260709 13:00:13.053750  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.086138  3256 raft_consensus.cc:3055] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:13.086483  3256 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:13.087246  3256 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.088413  3256 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.054461  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:13.054602  1613 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:13.093838  1614 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.094386  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:12.970892  3115 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:00:13.042007  3290 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.018993  3168 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.032227  3242 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.067118  1538 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:13.059818  1658 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.055750  3294 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:13.057874  1610 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.070896  3262 consensus_queue.cc:1243] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 8a6a5ae1b7f240408b3ddd30d8a18262 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:00:13.115859  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.058271  3162 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:00:13.108963  3186 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:00:13.066695  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 92674.1us
I20260709 13:00:13.123342  1614 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:13.124406  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:13.069110  3188 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.099098  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:13.099844  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 128282us
I20260709 13:00:13.101357  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:13.101688  3115 raft_consensus.cc:3055] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:13.101780  1535 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.136068  3162 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:13.136482  3162 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:13.131139  3186 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:13.136986  3186 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.137341  3162 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:13.140066  3162 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.102114  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:13.102232  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:13.066219  3314 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:13.145936  3314 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:13.147825  3159 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:13.148808  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.116765  1610 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:13.150092  1610 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:13.150944  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.152209  3247 consensus_queue.cc:1059] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:13.152500  1583 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.153751  3241 raft_consensus.cc:3055] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:13.153821  3247 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.154188  3241 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:13.154654  3247 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.089694  3288 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:00:13.155390  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:13.155073  3241 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.155827  3288 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.156141  3320 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:13.156320  3288 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:13.156716  3320 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:13.156819  3241 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.141891  3115 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:13.157771  3320 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:13.158370  3115 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.133945  3287 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.134228  3246 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:00:13.161584  1609 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:13.107910  1534 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:13.108314  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:13.162626  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:13.163560  3288 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.164752  3288 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.102820  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:13.110260  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50530.2us
I20260709 13:00:13.173995  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:13.110280  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.068617  3285 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:13.128515  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.111058  3272 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.151149  1610 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.161099  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.133397  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:13.180670  1583 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.174953  1657 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:13.180846  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:13.178728  3322 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.183722  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:13.185129  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:13.187929  1539 raft_consensus.cc:3055] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.185279  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:13.186990  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:13.191915  3262 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.192468  3240 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:13.192982  3240 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:13.189700  1539 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:13.179240  1534 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.199159  3322 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.199676  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:13.187268  3241 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.186343  3256 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:00:13.196595  3323 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:00:13.202652  2203 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:13.202952  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:13.194204  3115 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.190055  3321 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:00:13.220940  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.224357  3322 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:13.225354  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:13.228843  3159 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:13.201344  3246 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.238045  1611 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 4. Ops: []
I20260709 13:00:13.238824  3186 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.218370  1539 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:13.250005  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.250260  3186 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:13.250051  1534 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:13.252856  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:13.249581  3322 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:13.253825  1685 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:13.255690  3242 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.259455  3242 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:13.243968  3321 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:13.260512  3321 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.256956  3322 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:13.247946  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47548.8us
W20260709 13:00:13.267488  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 68473.4us
I20260709 13:00:13.273924  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:13.195596  3240 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.272912  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.282750  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:13.283679  2231 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:13.285004  3188 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.269747  3186 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 13:00:13.293649  1612 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:13.296455  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:13.303632  3290 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:13.305730  3288 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:13.308168  3291 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.274592  3313 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 13:00:13.313139  1615 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:13.267223  3320 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:00:13.318481  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2056) took 1018 ms. Trace:
I20260709 13:00:13.318892  1538 rpcz_store.cc:276] 0709 13:00:12.300034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:12.300148 (+   114us) service_pool.cc:224] Handling call
0709 13:00:12.300452 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:13.109837 (+809385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:13.317851 (+208014us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b5400343fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:13.318423 (+   572us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1762742144}
I20260709 13:00:13.320297  3303 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.320863  3303 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.322336  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:13.318385  3266 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:13.339164  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 58248.1us
I20260709 13:00:13.340080  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:13.341470  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:13.348397  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:13.349278  1538 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:13.347961  1610 raft_consensus.cc:2410] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:13.356186  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:13.364912  1582 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:13.367062  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:13.367664  1541 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.347679  3259 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:00:13.344281  3328 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.374300  1534 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:13.376799  3202 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:13.377087  3325 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:13.377254  3202 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:13.377501  3325 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:13.378705  3202 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:13.378782  3325 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:13.380076  3325 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.344596  3326 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.363087  3290 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:13.388747  1612 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.391839  1609 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:13.369863  3241 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.365764  3303 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:13.405683  1539 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.414815  1539 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:13.416227  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:13.416066  3314 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.417207  3316 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:13.417749  3316 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:00:13.418987  3251 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 13:00:13.418674  3316 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.420900  3246 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:13.422488  1618 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:13.422645  1541 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:13.425482  3294 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.434201  3268 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:13.434649  3268 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:13.434881  1437 catalog_manager.cc:5697] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:13.435566  3268 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.436915  3268 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.437582  3291 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:13.438426  1542 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.439055  3260 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.444483  1690 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.446192  1685 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:00:13.447181  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.447221  1688 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.436576  1437 catalog_manager.cc:5697] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 3, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 3 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:13.450515  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:13.451164  1691 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:13.451327  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:13.451969  2232 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.454011  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:13.463824  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:13.464864  2235 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 2: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:13.465433  1543 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.482944  2233 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:13.479281  1437 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 3 to 4, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 4 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:13.484591  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:13.457453  3254 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:00:13.489845  2201 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:13.487789  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:13.461242  3314 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.495793  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:13.498137  3308 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:13.498558  3308 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:13.498919  3308 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.502293  3202 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.503674  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.504141  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:13.504755  1658 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:13.505663  3180 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.506697  3168 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:13.506836  3288 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:13.507272  3168 raft_consensus.cc:697] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:13.507408  3288 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:13.508790  3189 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:13.508700  3288 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.509514  3253 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:13.494004  2228 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:13.509366  3189 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.511479  1509 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.465579  3259 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:13.513289  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:13.400871  3328 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.090s
I20260709 13:00:13.542629  3300 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.542989  3328 consensus_queue.cc:1243] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 8a6a5ae1b7f240408b3ddd30d8a18262 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:00:13.515223  1615 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:13.493541  3159 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:13.518976  1583 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.519290  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.521375  3168 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.546180  3328 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:13.546695  3328 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:13.547528  3328 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.548676  3240 raft_consensus.cc:3055] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:13.522286  3250 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:13.527729  1613 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:13.490269  1684 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:13.493034  3273 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.535571  2235 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.557672  3180 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:13.543493  3261 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:13.513911  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:13.486418  3127 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:00:13.575594  2234 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.569654  3322 consensus_queue.cc:1059] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:00:13.556550  3273 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:13.576715  3189 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:13.577338  3322 raft_consensus.cc:3055] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:13.548708  3291 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 13:00:13.555794  3259 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.549121  3240 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:13.592669  1615 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.555600  3254 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:00:13.595849  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 706143us
I20260709 13:00:13.596796  3261 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:00:13.603413  1507 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.077s (SENT)
I20260709 13:00:13.577706  3322 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:00:13.606518  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:13.609498  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 67066.4us
I20260709 13:00:13.571695  3250 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:13.613922  3159 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:13.614194  3240 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.614987  3259 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:13.608991  3230 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:00:13.617564  1618 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:13.578415  3251 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:00:13.578052  3228 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:00:13.577616  3273 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.620131  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:13.594982  3257 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:00:13.641340  1684 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:13.633262  1616 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:13.627563  3285 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.643375  3285 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.644033  1615 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:13.645727  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:13.636307  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:13.644335  3336 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:00:13.660360  3273 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.663239  3271 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.652001  3325 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 13:00:13.653848  3261 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.678745  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:13.694311  3259 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:13.721869  3259 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:13.722886  3259 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.724275  3259 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.687414  3271 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.681376  3319 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:13.726155  3319 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:13.726783  3308 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.727442  3271 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:13.734120  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 67920.1us
I20260709 13:00:13.684760  3095 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:13.686751  3342 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.659722  3322 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:13.662753  3250 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.745195  1507 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.705535  3285 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:13.727234  3319 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:13.752923  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 107266us
I20260709 13:00:13.759537  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:13.760202  3303 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:13.760620  3303 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:13.764591  1684 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:13.765588  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:13.766176  1692 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.766371  3314 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:13.747469  3336 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:13.752815  3115 consensus_meta.cc:287] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.000s
W20260709 13:00:13.819424  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.819370  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:13.820173  2234 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:13.775698  3230 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:13.817957  3336 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.746025  3342 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:13.822096  1692 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
W20260709 13:00:13.826489  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.764649  3308 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:13.830335  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:13.830669  3342 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.832651  3342 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:13.798738  3168 consensus_queue.cc:765] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Error trying to read ahead of the log while preparing peer request: Incomplete: Op with index 2 is ahead of the local log (next sequential op: 2). Destination peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.285s
I20260709 13:00:13.825825  3303 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.834978  2201 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:13.840296  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:13.842197  1542 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:13.826951  3291 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:13.844132  2203 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:00:13.845793  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 92937.5us
I20260709 13:00:13.838222  3339 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:13.942963  3339 raft_consensus.cc:697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:13.943960  3339 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.835923  3319 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.837394  3266 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:13.935288  3308 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:13.937156  3336 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:13.953943  1542 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:13.941372  3291 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.943041  1609 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:13.948167  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:13.961457  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:13.967582  3291 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:13.967672  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:13.956712  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.968842  1692 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:13.969352  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:13.841840  1657 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.949630  3277 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:13.949949  3308 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:13.952142  3266 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:13.985533  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 143771us
I20260709 13:00:13.986413  3346 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:00:13.988057  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:13.989499  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 145453us
I20260709 13:00:13.993263  1507 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:13.995687  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:13.995244  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:13.998665  3257 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:13.993062  3277 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:13.994100  1535 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:14.000764  3277 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:14.002344  3266 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:14.002606  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:14.003892  3318 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:13.986634  3344 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:13.993836  3260 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.006593  3294 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.010416  2234 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:14.011938  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:14.019208  1692 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:14.012970  1542 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:14.014845  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:13.982610  1534 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:13.989010  3291 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.015844  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:14.011336  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:13.994686  1685 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
W20260709 13:00:14.008509  2232 consensus_meta.cc:287] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.556s	user 0.008s	sys 0.003s
I20260709 13:00:14.045704  2232 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:14.040099  3260 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.046094  3325 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:14.046526  3325 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.046868  3325 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.046994  2231 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:14.047811  3260 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:14.015909  3349 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:00:14.028898  2237 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:13.988008  3308 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.018185  3344 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:14.057366  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:14.053300  3268 consensus_meta.cc:287] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.003s
I20260709 13:00:14.059445  3341 raft_consensus.cc:2764] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:00:14.060221  3251 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:14.061820  3251 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:14.062320  3251 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:14.067025  3277 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:14.067629  3277 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:14.068392  3302 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:14.069679  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:14.070295  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:14.071017  1613 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:14.053007  3288 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:00:14.070370  1609 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:14.077795  3095 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:14.078519  3095 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.080915  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:14.083227  3318 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:14.083629  3318 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.083956  3318 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.057965  3344 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.097388  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:14.073946   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:14.098567  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:14.099822  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:14.103431  1614 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.115612  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:14.131733  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 102078us
I20260709 13:00:14.125723  3257 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:14.129451  3260 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:14.137082  3260 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.117777  3285 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.139094  3325 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.140818  3228 raft_consensus.cc:3055] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:14.137503  3260 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.145082  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:14.115051  3302 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.156747  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:14.114859  3253 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:14.162978  1657 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:14.126123  3277 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:14.141245  3228 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:14.137110  3347 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.141284  3325 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:14.129549  3245 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
W20260709 13:00:14.190021  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 74485.8us
I20260709 13:00:14.183226  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 252) took 1036 ms. Trace:
I20260709 13:00:14.191133  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:14.164255  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:14.161648  3288 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:00:14.193998  1616 rpcz_store.cc:276] 0709 13:00:13.147525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.148966 (+  1441us) service_pool.cc:224] Handling call
0709 13:00:13.592495 (+443529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:13.642045 (+ 49550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:14.157523 (+515478us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b5400190850. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:14.175463 (+ 17940us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1010695552}
I20260709 13:00:14.205633  3228 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.211669  3189 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.212260  3189 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.214445  3189 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:14.214658  3343 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:14.214980  3260 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.217837  1535 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.198486  3325 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:14.223436  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:14.224289  1616 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.226485  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.174108  3253 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.227407  1541 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:14.222808  3260 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:14.240183  1609 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.235308  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:14.243536  3189 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:14.243995  3189 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.244364  3189 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.244419  3268 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.245080  3268 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:14.247113  3268 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:14.202800  2231 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:14.251354  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:14.252019  2235 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.266520  1616 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:14.244541  2228 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.276269  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:14.276973  1609 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:14.277483  3189 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.269354  3318 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.278697  3273 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:00:14.294664  3189 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:14.297752  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:14.298555  3168 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:14.298694  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:14.299561  1691 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:14.299156  3168 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.234403  3162 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 13:00:14.303889  3303 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.304011  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:14.304656  1688 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:14.305670  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:14.306306  1609 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.307920  3287 raft_consensus.cc:3055] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:14.308292  3287 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:14.308770  3251 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.309051  3287 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:14.301255  3316 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:14.310387  3316 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:14.310762  3316 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.279428  3115 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:00:14.282969  3095 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.321841  1582 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:14.333130  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.334990  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.335173  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.335773  1542 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:14.336925  1541 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:14.338065  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.338775  1541 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:14.236949  3344 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.342545  1584 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:14.287849  3268 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:14.352448  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:14.358294  1539 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.361241  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.365991  2235 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:14.367491  1657 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:14.368171  1543 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.368449  1540 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:14.379496  1539 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:14.339172  3322 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.382606  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.340796  3241 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:14.390208  1538 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.281572  3308 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.391687  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:14.394779  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:14.407202  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.369118  3277 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.420760  1615 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.414115  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:14.396378  3115 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:14.422894  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:14.382128  3268 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.423930  2235 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 2: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:14.402377  3288 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:00:14.425829  1610 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:14.415341  3241 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.433496  3318 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:14.433583  3241 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.419322  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:14.404258  3095 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:14.389882  3253 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:14.431607  1615 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:14.420832  3168 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:14.464012  1657 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:14.488521  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:14.426103  3294 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:14.424176  3251 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:14.431046  3115 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:14.435766  3308 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:14.467751  3300 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.557701  2228 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.487092  3261 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:14.501574  3347 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:14.500680  3291 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:14.543111  3168 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.547850  3243 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.544885  3294 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:14.555267  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:14.555980  1534 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:14.629782  1536 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.580355  3261 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.638522  3230 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:14.643077  3230 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.590478  3347 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:14.644558  3261 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:14.638724  1535 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:14.652369  3230 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:14.618327  3243 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:14.654480  3294 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.598348  3168 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:14.644835  3308 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.621735  3350 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.656648  3277 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:14.656836  2237 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.575610  3300 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.704260  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:14.704865  2234 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:14.705744  3300 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:14.680873  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.669646  3303 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.710907  3303 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.709141  1536 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.712153  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.712605  2229 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:14.712783  1538 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:14.698297  3261 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.713852  3261 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:14.716490  3261 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:14.718425  3228 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.669162  3364 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.724095  3364 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.727689  3364 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:14.731823  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:14.732939  3300 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:14.733474  3300 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.669114  3115 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.733889  3300 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:14.734643  3302 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.558s	user 0.011s	sys 0.002s
I20260709 13:00:14.682171  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:14.736524  2235 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 2: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:14.685580  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:14.687868  2201 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:00:14.738152  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 50419.1us
W20260709 13:00:14.689103  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.640463  3253 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:14.645668  3288 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.657737  3180 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.741904  3180 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.743964  3268 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:14.744112  3180 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:14.744349  3189 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.744597  3268 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.702576  1691 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.700996  3344 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:14.713300  3303 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:14.741048  3345 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:14.747725  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.749812  3288 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:14.767647  3366 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:00:14.748466  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:14.664646  3095 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 13:00:14.769392  1691 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.698868  3230 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.765973  3355 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.750743  3253 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.762818  3345 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:14.771360  1688 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:00:14.753876  3321 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.744850  3189 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.767407  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:14.765151  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:14.751425  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:14.752911  3261 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:14.711071  3347 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.777304  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:14.763610  3277 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.787956  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.771018  2234 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:14.793139  3302 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.778017  3345 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.702474  3308 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.779402  1683 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:00:14.786059  3271 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.064s	sys 0.097s
I20260709 13:00:14.754954  3268 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.828876  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.798132  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.831862  3189 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:14.837615  1534 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 13:00:14.807046  3302 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:14.839704  3302 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:14.840273  3287 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:14.799506  3277 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:14.777132  3230 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:14.783383  3321 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:14.828233  3271 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.065s	sys 0.098s
W20260709 13:00:14.849334  1507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.009s	sys 0.021s
I20260709 13:00:14.828038  3288 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:14.851867  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:14.862003  1691 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:14.804503  1609 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:14.840904  3241 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:14.864141  1507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.009s	sys 0.021s
I20260709 13:00:14.834339  3260 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.875218  1538 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:14.798905  3261 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:14.875975  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 684900us
I20260709 13:00:14.831835  3253 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:14.879042  1613 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.023s	sys 0.037s
W20260709 13:00:14.885980  1613 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.023s	sys 0.037s
I20260709 13:00:14.845314  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:14.779680  3355 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:14.886986  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 44078.2us
I20260709 13:00:14.889403  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.890436  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:14.890753  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:14.891391  1691 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:14.890540  1539 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:14.892429  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:14.894240  1687 raft_consensus.cc:2368] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:14.893616  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.878527  2231 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 2: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:00:14.905007  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.904053  1543 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:14.874675  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.874773  3189 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:14.915668  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:14.915763  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:14.885290  3257 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:14.918025  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:14.904412  3288 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:14.917950  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:14.916209  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:14.917502  1692 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:14.935535  1584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.014s	sys 0.018s
W20260709 13:00:14.946053  1584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.014s	sys 0.018s
W20260709 13:00:14.941164  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:14.914997  1583 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:00:14.961632  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 53170.3us
W20260709 13:00:14.969712  1542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.008s	sys 0.024s
W20260709 13:00:14.970209  1542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.008s	sys 0.024s
I20260709 13:00:14.973097  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:14.946336  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 593960us
I20260709 13:00:14.965354  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:14.975907  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:14.976722  1613 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:14.976658  1615 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:14.978127  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:14.879159  3260 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:14.903774  1685 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:14.986313  1539 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:14.989216  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:14.987041  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:14.991180  1541 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:14.984191  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:14.991884  1617 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:14.980765  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:14.982179  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:14.994222  1613 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
W20260709 13:00:14.949528  3314 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.007s	sys 0.058s
I20260709 13:00:14.971674  3288 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.001549  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:15.004609  1692 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
W20260709 13:00:14.989396  3359 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:00:14.971534  3243 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:14.981861  3095 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.018628  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:15.021790  3382 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:14.999641  3314 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.008s	sys 0.058s
I20260709 13:00:15.026747  1692 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 2: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:14.980546  3381 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:14.921618  3189 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.014072  3288 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.033260  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:15.014758  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.013667  3254 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:14.996264  3379 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.045926  1688 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:14.981112  3257 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.052847  3387 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.029634  3251 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:15.040352  3341 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:15.040169  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.049522  3259 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.079281  3397 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:15.082358  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:15.083412  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:15.069000  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:15.092990  3115 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.093940  3115 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.098428  3261 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.099231  3379 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.054322  3243 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:15.101825  3243 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.103526  3321 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.104642  3243 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.109691  3277 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.111943  3345 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.063098  2229 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:15.031996  3095 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.142575  1690 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:15.144140  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:15.088994  1688 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
W20260709 13:00:15.116382  3291 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.529s	user 0.008s	sys 0.002s
I20260709 13:00:15.070084  1535 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:15.164263  1689 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.164887  1540 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:15.117689  3350 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.118248  3268 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:14.996168  3376 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 13:00:15.040673  3381 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:15.136058  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 52748.4us
I20260709 13:00:15.124182  1692 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.097134  3115 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:15.172749  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.160203  1686 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:15.088348  3322 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.080309  3266 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.142650  3277 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.171739  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:15.152252  3243 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:00:15.194415  1688 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:15.195976  1688 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.196382  3243 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.179956  3381 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.167454  3291 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:15.173854  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.198545  2237 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:15.176816  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.181233  3230 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:15.181962  3352 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:00:15.181663  1690 raft_consensus.cc:2955] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:15.081190  3251 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:15.183482  3316 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.872s	user 0.009s	sys 0.004s
I20260709 13:00:15.222229  3316 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.224094  3251 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:15.183773  3383 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:00:15.184283  3379 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.226063  3379 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.226356  3308 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.226641  3379 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.144196  3321 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:00:15.198823  3345 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.060606  3382 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.232161  3382 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.233748  3316 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.189263  3355 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.189499  3350 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.190028  1684 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:15.190237  3268 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.246244  3268 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.247447  3240 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:15.248620  3268 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.168128  3349 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.190433  3384 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.249735  3384 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.251472  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.141891  3339 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:15.063562  3254 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.192824  2203 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:15.256296  2235 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:15.257263  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.257887  2235 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:15.257697  3349 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.183319  3257 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:15.258765  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 66032.7us
I20260709 13:00:15.254900  3253 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.259865  3253 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.261888  3180 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:15.262027  3253 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.210007  3376 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.216637  1687 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:15.218063  3189 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.151175  3387 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.220911  3095 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.156178  3241 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.279135  3339 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:00:15.136672  3259 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.228961  3379 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.187927  3300 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.287739  2234 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 2: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:15.231953  1437 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40172 (request call id 228) took 2144 ms. Trace:
I20260709 13:00:15.245122  3350 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.252168  2236 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:15.088410  3341 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.254560  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.294899  2237 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:15.171159  1689 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.259234  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.294761  3355 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.296546  2232 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:15.296634  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:15.297047  3355 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.297716  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.262430  3180 raft_consensus.cc:697] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:15.298310  3387 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.298978  3341 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.299269  1612 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:15.252043  3384 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.299041  3180 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.300130  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.263616  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.300889  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.301100  2235 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:15.263881  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.264570  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.265739  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.193222  3322 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.270403  1542 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.271252  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.272083  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:15.272862  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.305727  1539 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:15.203401  1688 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.264668  1438 catalog_manager.cc:5697] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 2, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:15.292016  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.306792  2236 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:15.292115  3259 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.276999  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.292608  3241 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.197125  3243 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:15.302270  1616 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 2: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:15.302500  1617 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:15.301503  2237 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:15.305461  1541 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.304430  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:15.270483  3316 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:00:15.304836  1543 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:15.309832  3345 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:15.311470  3322 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.205745  3266 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.308146  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.302793  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.317821  3241 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.307477  1438 catalog_manager.cc:5697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 2 to 3, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:00:15.314435  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 349153us
I20260709 13:00:15.332135  1692 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.283736  3288 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.308146  2203 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:15.318547  1689 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:15.313946  2233 raft_consensus.cc:2368] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3.
I20260709 13:00:15.321309  1614 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:15.335837  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:15.298341  2232 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:15.331449  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:15.338758  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.339035  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.339429  1539 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:15.339474  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:15.298869  3376 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.288920  1437 rpcz_store.cc:276] 0709 13:00:13.087094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.087284 (+   190us) service_pool.cc:224] Handling call
0709 13:00:15.231857 (+2144573us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:13.630320 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:13.630905 (+   585us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:13.630937 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:00:13.630963 (+    26us) tablet.cc:662] Decoding operations
0709 13:00:13.666288 (+ 35325us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:13.666364 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:13.666390 (+    26us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:00:13.669957 (+  3567us) tablet.cc:701] Row locks acquired
0709 13:00:13.669983 (+    26us) write_op.cc:260] PREPARE: finished
0709 13:00:13.670146 (+   163us) write_op.cc:270] Start()
0709 13:00:13.670290 (+   144us) write_op.cc:276] Timestamp: P: 1783602013670088 usec, L: 0
0709 13:00:13.670323 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 13:00:13.681300 (+ 10977us) log.cc:844] Serialized 1365 byte log entry
0709 13:00:14.187855 (+506555us) op_driver.cc:464] REPLICATION: finished
0709 13:00:14.832411 (+644556us) write_op.cc:301] APPLY: starting
0709 13:00:14.832511 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:14.832657 (+   146us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:14.832795 (+   138us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:14.918019 (+ 85224us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:14.989181 (+ 71162us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:00:15.002862 (+ 13681us) write_op.cc:312] APPLY: finished
0709 13:00:15.029731 (+ 26869us) log.cc:844] Serialized 35 byte log entry
0709 13:00:15.046569 (+ 16838us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:15.055028 (+  8459us) write_op.cc:454] Released schema lock
0709 13:00:15.058918 (+  3890us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:15.151212 (+ 92294us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:15.153778 (+  2566us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":619512,"apply.run_cpu_time_us":3589,"apply.run_wall_time_us":249480,"num_ops":3,"prepare.queue_time_us":131774,"prepare.run_cpu_time_us":105840,"prepare.run_wall_time_us":345573,"raft.queue_time_us":171785,"raft.run_cpu_time_us":70986,"raft.run_wall_time_us":441321,"replication_time_us":517385,"thread_start_us":941262,"threads_started":4,"wal-append.queue_time_us":316993}]]}
I20260709 13:00:15.337212  1689 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:15.342548  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:15.343257  1689 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.243995  3277 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.351668  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:15.353587  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.354287  1542 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:15.355970  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.357189  1539 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:15.358482  1507 leader_election.cc:400] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:15.354293  1541 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:15.358814  1507 leader_election.cc:403] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:15.362382  1507 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:15.331427  3341 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.363827  1691 raft_consensus.cc:2368] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3.
I20260709 13:00:15.353749  3349 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.364404  3095 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.353310  3400 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:15.364986  3350 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.360774  3117 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.379945  3399 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.379806  3117 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.380462  3399 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.382038  3117 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.382417  3399 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.384212  3376 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.384502  3300 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.374711  3341 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.385963  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:15.386268  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:15.386610  1688 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.386766  1685 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.387687  3341 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.392264  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.369892  3318 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:15.374709  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 1892) took 2123 ms. Trace:
I20260709 13:00:15.376297  3400 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.376670  3095 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.366497  1613 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:15.361119  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.369086  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:15.338996  1535 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:15.357395  3345 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.374325  3277 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.356688  3396 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:15.359422  3259 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.367645  3241 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.397751  1537 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:15.385982  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:15.403616  1618 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:15.398424  1507 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:15.426803  1507 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:15.393020  1610 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:15.428416  1507 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:15.435305  1507 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:15.436873  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:15.421118  1688 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:15.440723  1582 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:15.441089  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:15.443483  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:15.444675  1688 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:15.395511  3095 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.405030  3243 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.406358  3318 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:15.438320  3251 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:15.396634  3384 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.402516  3350 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:15.399967  3345 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.399209  3266 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.394886  3288 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.401058  3254 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.437230  3341 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:15.395083  3400 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.451331  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:15.394629  1683 rpcz_store.cc:276] 0709 13:00:13.251683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.251886 (+   203us) service_pool.cc:224] Handling call
0709 13:00:13.252187 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:14.809111 (+1556924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:15.356690 (+547579us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b540003fa54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:15.369740 (+ 13050us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3411859712}
I20260709 13:00:15.424801  3246 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.439054  3230 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:15.448052  3228 consensus_queue.cc:1059] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:00:15.442278  3303 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:15.442384  3300 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.475647  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.444337  3257 raft_consensus.cc:2764] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:00:15.446498  3322 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:15.446241  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:15.452317  3287 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:00:15.479099  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.479807  1537 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:15.446942  3316 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:15.457444  3318 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:15.458097  3251 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 3 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:15.458034  3384 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.400635  3259 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.456897  1689 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
W20260709 13:00:15.461685  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.453619  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:15.494905  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:15.466375  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.499397  3266 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:15.442319  3364 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:15.459225  3345 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.499785  3266 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:15.498790  1685 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:15.393258  3381 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.500233  3266 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.502146  3381 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.452869  3261 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 13:00:15.400389  3396 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.504460  3396 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.468576  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.505218  1616 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:15.506208  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.506225  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:15.506861  1616 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:15.507726  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.508292  1616 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:15.477697  1688 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:15.467979  3243 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.509666  3243 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.510035  3243 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.483489  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:15.509102  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.500007  3364 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.508956  3345 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.513622  3364 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.514508  2237 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:15.513906  1616 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:15.518712  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.524572  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:15.526165  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 259329us
I20260709 13:00:15.525265  1616 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:15.528970  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47379.1us
I20260709 13:00:15.471737  3303 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:15.530822  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.532526  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:15.534626  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.479104  3322 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.528322  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:15.533252  3241 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:00:15.469949  3246 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.570648  1616 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:15.534484  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:15.479409  3300 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.568970  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:15.527776  3240 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:00:15.580603  3254 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:15.584041  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.621997  1543 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:15.584739  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.585018  1683 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:15.585901  3328 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.525918  3316 raft_consensus.cc:697] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:00:15.588737  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 133940us
I20260709 13:00:15.591138  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.526137  3302 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.538024  1610 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:00:15.540807  3322 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.592463  2233 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:15.538305  3291 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:15.527043  3288 consensus_queue.cc:1059] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:00:15.577471  3384 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.651276  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.623387  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:15.623692  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:15.621881  1535 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 3
W20260709 13:00:15.625456  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 92988.4us
I20260709 13:00:15.587064  3180 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.632993  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.571746  3162 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:15.638912  3291 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.642144  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.646209  3261 consensus_queue.cc:1059] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:15.649358  3168 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.624085  1542 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:15.674511  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.653025  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:15.654939  1543 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:15.655882  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:15.656947  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.679095  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.679965  1539 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:15.659006  1616 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:15.682864  1616 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:15.662973  1507 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:15.631946  3328 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:15.665330  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 137104us
I20260709 13:00:15.652582  3381 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.671968  3300 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.665381  3291 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.662357  3240 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:15.681149  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:15.676122  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:15.689153  1542 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:15.699080  1539 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:15.594347  3251 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.663882  3241 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.686992  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.704434  1543 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:15.705901  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:15.701639  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.658546  3321 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.699827  3230 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.708793  3321 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.709286  3230 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:15.709724  3230 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.711014  3321 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.711165  3243 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.639298  3302 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.713611  3384 raft_consensus.cc:3055] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:15.668812  3162 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.714078  3384 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:15.714958  3384 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.699803  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.635156  3254 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.718088  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:15.719374  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 63541us
I20260709 13:00:15.700883  3240 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.715730  3162 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.720562  3408 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.690380  3328 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.749570  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.665241  3180 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.748529  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:15.707716  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.708334  3349 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:15.654032  3303 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.752339  1539 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:15.750874  3168 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:15.747112  3384 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.751139  3243 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.764956  1543 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:15.705343  3381 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.697803  3266 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.774703  1542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88cbab659ea041148748114fae549ae0"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:53670
I20260709 13:00:15.774962  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:00:15.828382  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 53474.8us
I20260709 13:00:15.778862  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:15.761215  3300 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:15.759724  3240 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.779680  1543 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.716607  3241 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.721894  3322 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:15.832226  3416 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.832772  3416 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.833132  3416 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.835093  3241 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.835276  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:15.835717  3241 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.836169  3241 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.836421  3376 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:15.837050  3376 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.837535  3376 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.770803  3259 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:15.806422  1688 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:15.793341  2229 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:15.716466  3316 consensus_queue.cc:237] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.775921  3168 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.844689  3168 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.774367  3408 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.846845  3408 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.776059  3349 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:15.793751  3162 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:15.771823  3364 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.800293  3277 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:15.858328  3277 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:15.829701  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50898.6us
I20260709 13:00:15.861841  1537 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:15.862713  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:15.837812  3230 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.836520  3241 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.801136  1689 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:15.820182  3341 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:15.855008  3417 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:15.831085  1542 raft_consensus.cc:606] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:15.880240  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.891345  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:15.861119  3277 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.882954  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.894080  1616 raft_consensus.cc:2410] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:15.896940  1542 raft_consensus.cc:614] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:00:15.897157  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 211406us
I20260709 13:00:15.896991  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:15.899263  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.900693  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.903453  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.906404  1538 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.906807  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.907642  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:15.908455  1616 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:15.865312  3162 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:15.883147  3243 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:15.883980  3259 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:15.848212  3300 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:15.853346  3397 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.921188  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:15.925855   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:00:15.931892   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:15.877957  3399 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:00:15.877583  3414 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260709 13:00:15.905506  3288 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:15.952589  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:15.953539  1542 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:15.946156  3397 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:15.911788  3180 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:15.918829  3162 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:15.905413  1610 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:15.919391  3417 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:15.926980  3243 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:15.960762  3243 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:15.965873  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:15.971866  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:15.930569  3259 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:15.974856  3250 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:00:15.992889  3266 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:15.963548  3349 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.009294  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:16.029609  3254 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:16.039548  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:16.041956  1538 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:16.048624  3230 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:16.058611  3397 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.061010  3397 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:16.066887  3095 consensus_meta.cc:287] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.612s	user 0.008s	sys 0.001s
I20260709 13:00:16.113238  1542 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:16.120775  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1135) took 1229 ms. Trace:
I20260709 13:00:16.121285  1684 rpcz_store.cc:276] 0709 13:00:14.891707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.088500 (+196793us) service_pool.cc:224] Handling call
0709 13:00:15.189652 (+101152us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:15.245794 (+ 56142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.120511 (+874717us) spinlock_profiling.cc:243] Waited 8.57 ms on lock 0x7b54000aaad0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:16.120699 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19719040}
I20260709 13:00:16.036859  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 269) took 1062 ms. Trace:
I20260709 13:00:16.135684  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:16.134014  1612 rpcz_store.cc:276] 0709 13:00:14.974690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.980515 (+  5825us) service_pool.cc:224] Handling call
0709 13:00:16.036593 (+1056078us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b5400389854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:16.036772 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":485055232}
I20260709 13:00:16.139099  1611 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:16.139813  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1152) took 1164 ms. Trace:
I20260709 13:00:16.140141  1611 rpcz_store.cc:276] 0709 13:00:14.975764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.981924 (+  6160us) service_pool.cc:224] Handling call
0709 13:00:16.138490 (+1156566us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:00:16.138728 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:16.139547 (+   819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.139757 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10107648}
I20260709 13:00:16.142875  3168 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.144410  3241 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.116096  3095 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.051100  3322 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:16.145066  3168 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:16.025275  3415 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:16.152513  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:16.165766  1508 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.519s (SENT) [suppressed 64 similar messages]
I20260709 13:00:16.067994  3291 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.015923  3300 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:16.153605  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 477557us
I20260709 13:00:16.180449  3291 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:16.001322  3416 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.034749  3417 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:16.187561  3430 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:00:16.185546  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:16.189229  2237 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:16.189855  3259 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:16.191735  3350 raft_consensus.cc:3055] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:16.192210  3350 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:00:16.193774  1687 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1146) took 1286 ms (client timeout 1519 ms). Trace:
W20260709 13:00:16.194170  1687 rpcz_store.cc:269] 0709 13:00:14.907620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.915954 (+  8334us) service_pool.cc:224] Handling call
0709 13:00:16.193608 (+1277654us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b54000aa5d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:16.193718 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624131968}
I20260709 13:00:16.197005  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1586) took 1451 ms. Trace:
I20260709 13:00:16.197412  1540 rpcz_store.cc:276] 0709 13:00:14.745448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.746064 (+   616us) service_pool.cc:224] Handling call
0709 13:00:15.164658 (+418594us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:15.165408 (+   750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.196809 (+1031401us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b54001fc7d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:16.196946 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":953272704}
I20260709 13:00:16.176687  3322 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:16.203178  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:16.203905  1539 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.178057  3095 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:16.182374  3424 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:00:16.206415  3241 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:16.153301  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:16.187788  3408 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.233667  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:16.221776  1542 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.234593  1538 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:16.235589  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:16.251353  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 352839us
I20260709 13:00:16.206709  3322 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:16.193127  3350 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.262164  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:16.227344  3400 consensus_meta.cc:287] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.757s	user 0.008s	sys 0.004s
I20260709 13:00:16.207542  3376 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.245931  3396 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:16.265739  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:16.270798  1616 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.265247  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:16.274560  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:16.258308  3382 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:16.268244  3259 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:16.269368  3349 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:16.290450  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:16.282737  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 117011us
I20260709 13:00:16.275193  2201 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:00:16.275522  1535 consensus_meta.cc:287] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.609s	user 0.011s	sys 0.000s
I20260709 13:00:16.287632  1539 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:16.287632  1616 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:16.273533  3396 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.276907  3424 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:16.296260  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:16.281844  3350 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.288048  3384 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:16.299983  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:16.298550  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:16.301499  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:16.301635  1583 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:16.304020  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:16.305269  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:16.292721  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:16.327489  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:16.330087  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:16.330554  1689 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:00:16.339779  1538 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:16.293079  1535 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:16.344658  1508 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:16.295778  3410 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:16.334095  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:16.344895  1688 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:00:16.347025  1688 pending_rounds.cc:85] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 5
I20260709 13:00:16.345382  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:16.343619  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:16.268363  3400 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.344726  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:16.347653  1616 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.347046  1508 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:16.352603  3300 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:16.353070  3341 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:16.353529  3341 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:16.353230  1508 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:16.353941  3341 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.355435  3318 raft_consensus.cc:2764] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:00:16.355741  1508 leader_election.cc:400] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3.
I20260709 13:00:16.356891  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:16.349014  3421 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.350464  2236 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:16.325501  3259 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.375624  1616 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:16.351687  1542 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:16.356205  1508 leader_election.cc:403] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:16.296905  3436 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.306933  3316 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:16.350176  3230 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.370733  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:16.350346  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:16.384234  2236 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:16.361712  3328 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.588s	user 0.014s	sys 0.006s
I20260709 13:00:16.296550  3424 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:16.294868  3349 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.325606  3254 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:00:16.378912  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:00:16.430493  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 51648.1us
I20260709 13:00:16.358434  3384 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:16.379591  3426 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.379880  3287 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.437541  3426 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.439399  2237 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:16.380126  3406 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:16.349872  3415 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.359236  1688 pending_rounds.cc:107] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:00:16.377372  3436 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:16.381645  1617 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:16.346385  3291 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:16.293881  3382 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.295818  3243 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.358613  3261 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.373865  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:16.429915  3251 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.457082  1692 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:16.325834  3375 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:00:16.362468  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:16.356990  3095 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
W20260709 13:00:16.483033  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:16.434413  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:16.375931  3259 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.503383  3259 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.503551  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:16.478633  3426 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:16.384013  3230 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.381057  3400 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:16.448854  1688 raft_consensus.cc:2889] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:16.377888  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:16.384720  3421 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.446489  2230 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:16.442073  3328 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.388602  3115 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:16.518329  3421 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.344478  3322 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.358513  3288 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.386219  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:16.538108  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 151965us
I20260709 13:00:16.538604  3115 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:16.388788  3168 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:16.436899  3424 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:16.460059  2201 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:16.445393  3266 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:16.543167  3450 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:16.547716  3266 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:16.440097  3308 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:00:16.545949  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 85981.6us
W20260709 13:00:16.537083  3162 consensus_meta.cc:287] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.570s	user 0.008s	sys 0.002s
I20260709 13:00:16.447968  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:16.483585  1687 raft_consensus.cc:2410] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:16.484875  2236 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:16.463030  3251 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.459364  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:16.463644  3408 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:16.448783  3415 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.551265  3300 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:16.561584  3435 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:00:16.454263  3384 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:16.559511  3428 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:16.557677  3308 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.475824  3375 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:16.460680  3291 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:16.456719  3261 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.472438  3243 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:16.474282  3382 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:16.504390  1613 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:16.574517  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:16.575471  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:16.575996  1613 raft_consensus.cc:2368] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4.
I20260709 13:00:16.576486  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:16.440729  1538 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:00:16.505728  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 66345.6us
I20260709 13:00:16.511425  1616 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:16.577625  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:16.577972  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:16.456080  3436 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.428557  3416 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:16.539183  3115 raft_consensus.cc:697] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:16.572093  3261 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.541004  3288 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.362651  3410 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.517125  3376 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:16.535285  3287 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3.
W20260709 13:00:16.558980  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64260us
I20260709 13:00:16.547851  3450 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.557025  3162 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.553532  3426 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:00:16.562728  3428 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.561775  3415 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.561911  3318 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:00:16.565907  3266 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.566372  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:16.547256  3254 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:16.561815  3300 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.562692  3384 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.578146  1538 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.504853  3230 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.578702  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 495) took 1318 ms. Trace:
I20260709 13:00:16.607980  2235 rpcz_store.cc:276] 0709 13:00:15.260003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.304067 (+ 44064us) service_pool.cc:224] Handling call
0709 13:00:15.304452 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:15.304882 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.304921 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:15.305478 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:00:16.038976 (+733498us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b180031be20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:00:16.271729 (+232753us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:16.271773 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.271863 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:16.571334 (+299471us) raft_consensus.cc:1685] finished
0709 13:00:16.571356 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:16.578640 (+  7284us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":911,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"spinlock_wait_cycles":4416896,"thread_start_us":232549,"threads_started":1,"wal-append.queue_time_us":1037716}
I20260709 13:00:16.609840  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:16.610381  2235 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:16.611035  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:16.559742  3396 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:16.570904  3434 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:00:16.563990  2231 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:16.550732  3168 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.588177  3115 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.600477  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:16.621872  1541 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.603161  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:16.584609  3408 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:16.605862  2230 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:16.607494  3300 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:16.616688  3431 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:00:16.607604  3291 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.584854  3375 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.618029  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 720) took 1146 ms. Trace:
I20260709 13:00:16.626323  1613 raft_consensus.cc:2368] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:16.635643  3162 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:16.613725  3349 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:16.618614  1684 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:16.605896  3254 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.646808  1538 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:16.649248  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:16.657155  3288 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:16.657476  2230 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:16.658114  3254 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:16.660283  1691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1132) took 1769 ms (client timeout 1950 ms). Trace:
W20260709 13:00:16.660724  1691 rpcz_store.cc:269] 0709 13:00:14.890973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.892223 (+  1250us) service_pool.cc:224] Handling call
0709 13:00:16.660108 (+1767885us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b540028e6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:16.660230 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1082398592}
I20260709 13:00:16.666260  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:16.667507  1538 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.668987  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:16.669802  1613 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:16.637938  3408 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.662173  3328 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:16.700708  1541 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:16.701853  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:16.701973  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:16.702692  1541 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:16.707852  1538 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:16.708603  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:16.710713  1613 raft_consensus.cc:2368] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:16.713716  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:16.725556  3428 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:16.728746  3240 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.876s	user 0.003s	sys 0.009s
I20260709 13:00:16.721232  3288 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:16.729256  3240 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.731592  3240 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:16.732350  1616 raft_consensus.cc:2368] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:16.734100  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:16.734719  1616 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.732270  3168 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:16.709524  3251 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:16.716372  3384 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:16.794588  3266 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:16.736935  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:16.747231  3288 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:16.820683  3411 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:16.820953  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:16.821180  3411 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:16.821630  1691 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:16.733984  3355 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:16.825747  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:16.827632  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:16.801187  3266 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.827059  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:16.837407  1688 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:16.838441  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:16.842643  3345 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:16.844422  1618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 263) took 1871 ms (client timeout 1727 ms). Trace:
W20260709 13:00:16.844960  1618 rpcz_store.cc:269] 0709 13:00:14.972691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.972831 (+   140us) service_pool.cc:224] Handling call
0709 13:00:16.844195 (+1871364us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b540008cfd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:16.844343 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":988462336}
I20260709 13:00:16.846362  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:16.847333  1618 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:16.848316  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:16.848984  1618 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.850564  3347 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:16.851115  3347 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:16.782646  3240 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:16.827527  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:16.813692  3246 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:16.734102  3341 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.855997  3347 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.830755  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:16.823314  2203 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:16.857887  1690 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:16.834089  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:16.828419  1692 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:16.838507  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:16.866858  1688 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:16.815956  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:16.868100  1614 raft_consensus.cc:2376] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3. Re-sending same reply.
I20260709 13:00:16.868737  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1637) took 1405 ms. Trace:
I20260709 13:00:16.869131  1614 rpcz_store.cc:276] 0709 13:00:15.464000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.464251 (+   251us) service_pool.cc:224] Handling call
0709 13:00:16.815796 (+1351545us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:16.868667 (+ 52871us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9081088}
W20260709 13:00:16.877493  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 54310us
W20260709 13:00:16.878079  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 44083us
I20260709 13:00:16.854802  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:16.883517  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:16.857184  3347 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.885335  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2087) took 1403 ms. Trace:
I20260709 13:00:16.888893  1537 rpcz_store.cc:276] 0709 13:00:15.481618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.481753 (+   135us) service_pool.cc:224] Handling call
0709 13:00:16.885084 (+1403331us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b540012cfd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:16.885257 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":747624320}
I20260709 13:00:16.886026  1582 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:16.835870  3384 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:16.893975  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:16.879812  1535 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.893270  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:16.896720  3418 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:16.896944  3456 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:00:16.897205  3418 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.897701  3418 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.889111  3246 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.891029  3450 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:16.901549  1688 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:16.901710  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:16.891124  3202 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:16.906981  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:16.905279  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:16.906786  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:16.908417  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:16.914685  1688 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:16.914762  1690 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:16.915791  1692 raft_consensus.cc:2393] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:16.916734  1582 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:16.917634  1582 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:16.918838  1582 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:16.923031  1616 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:16.882403  3240 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.930823  1690 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:16.886754  3261 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.822054  3411 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.903785  3246 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.642400  2232 rpcz_store.cc:276] 0709 13:00:15.471650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.478840 (+  7190us) service_pool.cc:224] Handling call
0709 13:00:15.594544 (+115704us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:15.911803 (+317259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.918370 (+  6567us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:15.965239 (+ 46869us) log.cc:844] Serialized 26 byte log entry
0709 13:00:16.278231 (+312992us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:16.289145 (+ 10914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.295198 (+  6053us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:16.607734 (+312536us) raft_consensus.cc:1685] finished
0709 13:00:16.607760 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:16.617962 (+ 10202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":505,"raft.queue_time_us":67988,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":16367,"thread_start_us":507051,"threads_started":2,"wal-append.queue_time_us":326826}
I20260709 13:00:16.924983  3376 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:16.918709  3336 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:16.967281  3259 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:16.968215  3423 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:16.907301  3426 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:16.974902  1618 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
W20260709 13:00:16.975600  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 56825.2us
I20260709 13:00:16.928716  3241 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:16.979871  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:16.981304  1618 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:16.928093  3345 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:16.983336  3345 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.984444  3436 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.970230  3246 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:16.972427  3376 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:16.937523  3341 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:16.934725  3375 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:16.977887  3426 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:17.005504  3254 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:16.981572  3202 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:16.980852  3355 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:16.950101  3230 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:16.935473  3288 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:17.025692  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 154761us
I20260709 13:00:16.978361  3408 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:16.985885  3345 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:16.977877  1609 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:17.022506  3411 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:17.022814  3376 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:17.045984  1539 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:17.027936  3426 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:17.033111  3202 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:17.032203  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:17.023002  3241 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
W20260709 13:00:17.054656  3457 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 13:00:17.055243  3463 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:00:17.008881  3384 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:17.046326  3266 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:17.054212  3421 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.519s	user 0.013s	sys 0.000s
I20260709 13:00:17.057749  3189 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:17.066788  3260 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:17.068219  2232 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:17.068535  3230 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:17.051761  3202 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:17.068759  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 311) took 1800 ms. Trace:
I20260709 13:00:17.066474  3271 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:17.037139  3355 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:17.049161  3300 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:17.076103  3454 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:00:17.050611  3424 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:17.084239  2234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 718) took 1907 ms (client timeout 1683 ms). Trace:
I20260709 13:00:17.071331  3189 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:17.096872  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 102291us
I20260709 13:00:17.069286  3415 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:17.095283  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 121276us
W20260709 13:00:17.094659  2234 rpcz_store.cc:269] 0709 13:00:15.176353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.176517 (+   164us) service_pool.cc:224] Handling call
0709 13:00:17.083942 (+1907425us) spinlock_profiling.cc:243] Waited 69.9 ms on lock 0x7b5400599cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:17.084150 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160676736}
I20260709 13:00:17.098079  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:17.101765  2235 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:17.102208  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:17.079881  3409 log.cc:927] Time spent T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.597s	user 0.008s	sys 0.000s
I20260709 13:00:17.103015  1690 raft_consensus.cc:2393] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:17.103533  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:17.103870  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:17.106464  1688 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:17.106472  1692 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:17.118249  1689 raft_consensus.cc:2368] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:17.118535  2201 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:17.090314  3271 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:17.072981  3421 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:17.129945  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:17.107487  3254 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:17.131379  1618 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:17.131779  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:17.132735  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:17.107499  3400 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:17.137354  1610 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:17.137693  3400 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:17.138521  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:17.138788  3271 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:17.139173  1610 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:17.140081  3400 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:17.077786  1534 rpcz_store.cc:276] 0709 13:00:15.268361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.270869 (+  2508us) service_pool.cc:224] Handling call
0709 13:00:17.036728 (+1765859us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5800003998. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7eee 0000000000375939 00007fe094a3b1e0 00007fe094b040c4 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:00:17.036889 (+   161us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:17.057596 (+ 20707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:17.058339 (+   743us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:17.060672 (+  2333us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:17.068501 (+  7829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.068580 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.068701 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3809,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":4685,"spinlock_wait_cycles":23877376}
I20260709 13:00:17.127497  3266 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:17.142575  3415 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:17.134157  3261 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:17.142858  3266 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:17.127946  3189 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:17.143939  3189 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:17.144310  3421 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:17.144939  3266 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:17.145747  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:17.107672  3314 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:17.149109  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:17.118495  3240 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:17.126204  3349 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:17.153458  1618 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:17.156805  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:17.154402  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:17.127701  3436 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:17.131085  3259 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:17.178817  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:17.140094  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:17.143966  1535 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:17.170603  3400 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:17.233870  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:17.234793  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:17.235222  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:17.218438  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:17.220769  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:17.157618  3314 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:17.221019  3418 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:17.237653  2231 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:17.238126  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:17.222735  3246 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:17.238582  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:17.238900  3314 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:17.239298  1616 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:17.239566  3418 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:17.240072  1615 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:17.223989  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:17.223606  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:17.241567  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:17.242357  2232 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:17.242231  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:17.242520  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:17.157848  3254 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:17.233762  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:17.225634  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:17.183444  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 1909) took 2105 ms. Trace:
I20260709 13:00:17.184044  3260 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:17.249830  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:17.250530  1691 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
W20260709 13:00:17.229251  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 75482.1us
I20260709 13:00:17.251071  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:17.251271  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:17.251880  1618 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:17.267416  1583 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:17.150565  3271 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:17.248157  3257 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:17.230141  3408 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:17.240255  3246 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:17.380162  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:17.462050  1509 leader_election.cc:400] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4.
W20260709 13:00:17.463021  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 176957us
I20260709 13:00:17.464095  1509 leader_election.cc:400] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
W20260709 13:00:17.484011  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:17.491698  1692 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:00:17.245615  3451 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:17.501662  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:17.309324  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:17.232614  3400 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:17.307111  3411 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:17.377061  3300 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:17.378674  3468 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:17.575217  3424 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:17.307224  3257 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:17.581241  3257 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:17.582482  3257 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:17.584553  3468 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:17.572429  3328 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:17.585307  3328 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:17.585624  3468 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:17.587567  3328 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:17.538046  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:17.536751  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:17.592777  1692 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:17.377033  3471 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:17.593623  3471 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:17.382921  3408 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:17.594055  3471 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:17.594347  3408 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:17.491248  3260 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:17.595570  3260 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:17.508822  3349 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:17.597085  3349 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:17.573336  3254 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:17.599800  3349 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:17.601704  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:17.614555  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 362425us
I20260709 13:00:17.499262  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:17.309669  3230 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.262s
I20260709 13:00:17.516009  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:17.380373  3271 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:17.671996  1618 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:17.245663  1685 rpcz_store.cc:276] 0709 13:00:15.078403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.583304 (+504901us) service_pool.cc:224] Handling call
0709 13:00:15.583681 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:15.721030 (+137349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.721066 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:15.721112 (+    46us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:15.721153 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.155336 (+1434183us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b540003eb54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:17.158197 (+  2861us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.181909 (+ 23712us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303982976}
W20260709 13:00:17.913754  1583 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.010s	sys 0.007s
W20260709 13:00:17.914567  1583 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.010s	sys 0.008s
I20260709 13:00:17.914304  1610 raft_consensus.cc:2376] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3. Re-sending same reply.
W20260709 13:00:17.915169  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 628589us
I20260709 13:00:17.465059  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:00:17.918656  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 453536us
I20260709 13:00:17.919004  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 304) took 1613 ms. Trace:
W20260709 13:00:17.919899  2229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 721) took 2447 ms (client timeout 2988 ms). Trace:
I20260709 13:00:17.920275  1509 leader_election.cc:400] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:17.920794  1509 leader_election.cc:403] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:00:17.719115  1692 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:17.683924  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:17.614091  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:17.637372  2232 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:17.696928  3202 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.612s	user 0.004s	sys 0.005s
I20260709 13:00:17.936805  3261 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:17.938943  3261 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:17.939568  3261 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:17.932209  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:17.942166  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1170) took 1339 ms. Trace:
I20260709 13:00:17.942667  1613 rpcz_store.cc:276] 0709 13:00:16.602686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.711434 (+108748us) service_pool.cc:224] Handling call
0709 13:00:17.931729 (+1220295us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:17.942078 (+ 10349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3538176}
I20260709 13:00:17.944181  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:17.931244  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:17.735365  3189 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.591s	user 0.012s	sys 0.000s
W20260709 13:00:17.921403  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 302853us
W20260709 13:00:17.945787  2228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 497) took 1601 ms (client timeout 1674 ms). Trace:
I20260709 13:00:17.946095  3202 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:17.946393  2228 rpcz_store.cc:269] 0709 13:00:16.344720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.346781 (+  2061us) service_pool.cc:224] Handling call
0709 13:00:17.930664 (+1583883us) spinlock_profiling.cc:243] Waited 43.8 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:17.945674 (+ 15010us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100684672}
I20260709 13:00:17.947692  3228 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:17.933666  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:17.948413  3228 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:17.950006  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:17.951026  1611 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:17.953002  1539 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2109) took 1654 ms (client timeout 1500 ms). Trace:
W20260709 13:00:17.953544  1539 rpcz_store.cc:269] 0709 13:00:16.298836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.326922 (+ 28086us) service_pool.cc:224] Handling call
0709 13:00:17.952387 (+1625465us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b54003444d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:17.952687 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1582166272}
I20260709 13:00:17.919687  1684 rpcz_store.cc:276] 0709 13:00:16.305074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.306925 (+  1851us) service_pool.cc:224] Handling call
0709 13:00:17.918486 (+1611561us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54000744d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:17.918741 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":314461056}
I20260709 13:00:17.927176  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:17.936697  3254 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:17.950337  3228 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:17.941157  1536 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.290s	user 0.008s	sys 0.031s
I20260709 13:00:17.957545  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1670) took 1140 ms. Trace:
I20260709 13:00:17.964833  1618 rpcz_store.cc:276] 0709 13:00:16.817358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.955891 (+1138533us) service_pool.cc:224] Handling call
0709 13:00:17.956417 (+   526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:17.956900 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:17.956949 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:17.957032 (+    83us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:17.957087 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.957170 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.957312 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:17.966560  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 50053.4us
W20260709 13:00:17.918656  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:17.968607  1583 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:17.958811  3254 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:17.937882  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:17.970552  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:17.970857  3230 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:17.970989  1507 connection.cc:256] RPC call timeout handler was delayed by -1.6897344930000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260709 13:00:17.971422  1507 connection.cc:256] RPC call timeout handler was delayed by -1.3018344930000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
I20260709 13:00:17.971968  3254 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:17.971666  3230 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:17.959867  2235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 503) took 1613 ms (client timeout 2046 ms). Trace:
W20260709 13:00:17.972949  2235 rpcz_store.cc:269] 0709 13:00:16.346396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.610861 (+264465us) service_pool.cc:224] Handling call
0709 13:00:17.959465 (+1348604us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b540065a9d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:17.959685 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1127381376}
I20260709 13:00:17.946575  3189 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:17.973948  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:17.974682  3230 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:17.974833  1542 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:17.926152  2234 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:17.668365  3328 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:17.633950  3345 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.000s
I20260709 13:00:17.822170  3271 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:17.628556  3376 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.003s
I20260709 13:00:18.070705  3376 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.072397  1614 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:17.668077  3396 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:00:18.073565  3347 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:18.074386  3347 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:17.847841  3408 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.076948  3347 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:17.698949  3426 consensus_meta.cc:287] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.001s
I20260709 13:00:17.514420  3451 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.077396  3408 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:17.782533  3469 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 13:00:17.865091  3471 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:17.804467  3472 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:00:18.083194  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:17.920539  2229 rpcz_store.cc:269] 0709 13:00:15.471974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.472087 (+   113us) service_pool.cc:224] Handling call
0709 13:00:17.919350 (+2447263us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b540023d7d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:17.919618 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361390336}
I20260709 13:00:17.691244  3459 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:00:17.958287  2233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 502) took 1612 ms (client timeout 1580 ms). Trace:
I20260709 13:00:17.960074  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2103) took 2306 ms. Trace:
W20260709 13:00:17.963435  1617 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1639) took 2498 ms (client timeout 1644 ms). Trace:
I20260709 13:00:17.961066  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1640) took 2495 ms. Trace:
W20260709 13:00:18.151074  1617 rpcz_store.cc:269] 0709 13:00:15.465012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.465117 (+   105us) service_pool.cc:224] Handling call
0709 13:00:16.039326 (+574209us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:17.962893 (+1923567us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b5400102554. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:17.963192 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":780202624}
I20260709 13:00:18.151587  1615 rpcz_store.cc:276] 0709 13:00:15.465288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.465387 (+    99us) service_pool.cc:224] Handling call
0709 13:00:17.235082 (+1769695us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:17.960675 (+725593us) spinlock_profiling.cc:243] Waited 4.2 ms on lock 0x7b540008b6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:17.960860 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16152320}
W20260709 13:00:17.962354  1536 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.312s	user 0.009s	sys 0.031s
I20260709 13:00:18.154047  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 1920) took 1324 ms. Trace:
I20260709 13:00:18.154556  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1588) took 3379 ms. Trace:
I20260709 13:00:18.154413  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 290) took 1023 ms. Trace:
W20260709 13:00:18.024745  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:17.962035  3228 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:17.916841  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:18.049480  3411 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:17.961509  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:18.052139  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:18.059190  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 139371us
I20260709 13:00:18.058920  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.057804  3202 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.027346  3230 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:00:18.067065  2237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 427) took 1700 ms (client timeout 1774 ms). Trace:
I20260709 13:00:18.067575  3300 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:18.068938  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:18.074824  1614 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:17.575135  3400 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.065261  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:18.212040  1534 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:17.962235  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 1919) took 1134 ms. Trace:
I20260709 13:00:17.972340  1540 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:18.103408  3364 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:18.090672  3345 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.079562  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1172) took 1456 ms. Trace:
I20260709 13:00:18.085487  3257 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:18.098852  3402 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 2.467s	user 0.016s	sys 0.016s
I20260709 13:00:18.094017  3475 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.086318  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:18.078210  3426 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.130043  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:18.131614  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.128057  1611 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:18.135264  3376 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.135528  3308 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:18.140406  1542 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:16.984193   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:18.147668  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 1914) took 1326 ms. Trace:
I20260709 13:00:18.157495  1615 rpcz_store.cc:276] 0709 13:00:17.131007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.153243 (+1022236us) service_pool.cc:224] Handling call
0709 13:00:18.153743 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.154097 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:18.154119 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:18.154174 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:18.154202 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.154251 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.154361 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.155573  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:18.160840  3459 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:18.150668  2233 rpcz_store.cc:269] 0709 13:00:16.346103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.438919 (+ 92816us) service_pool.cc:224] Handling call
0709 13:00:17.926519 (+1487600us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:17.958178 (+ 31659us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3818240}
I20260709 13:00:18.149580  3347 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:18.149010  3408 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:18.185824  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 117897us
W20260709 13:00:18.189409  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 221012us
I20260709 13:00:18.189508  1541 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:00:18.184762  2237 rpcz_store.cc:269] 0709 13:00:16.344385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.344498 (+   113us) service_pool.cc:224] Handling call
0709 13:00:18.027038 (+1682540us) spinlock_profiling.cc:243] Waited 87.1 ms on lock 0x7b5400472a54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:18.044935 (+ 17897us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":200254336}
I20260709 13:00:18.150774  1543 rpcz_store.cc:276] 0709 13:00:15.653821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:15.840332 (+186511us) service_pool.cc:224] Handling call
0709 13:00:15.840696 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:15.899840 (+ 59144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:15.899884 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:15.907710 (+  7826us) log.cc:844] Serialized 26 byte log entry
0709 13:00:16.244065 (+336355us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:16.244132 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.959340 (+1715208us) spinlock_profiling.cc:243] Waited 56.7 ms on lock 0x7b540030bbd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:17.959426 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:17.959521 (+    95us) raft_consensus.cc:1685] finished
0709 13:00:17.959535 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.959850 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2716,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":12978,"spinlock_wait_cycles":130441088,"thread_start_us":288183,"threads_started":1,"wal-append.queue_time_us":312333,"wal-append.run_cpu_time_us":2019,"wal-append.run_wall_time_us":1004181}
W20260709 13:00:18.193647  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 131546us
I20260709 13:00:18.193724  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1672) took 1115 ms. Trace:
I20260709 13:00:18.162611  3261 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.256273  1684 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:18.252547  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1678) took 1092 ms. Trace:
I20260709 13:00:18.257469  1611 rpcz_store.cc:276] 0709 13:00:17.157960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.949463 (+791503us) service_pool.cc:224] Handling call
0709 13:00:18.250743 (+301280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.205611  3241 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:18.258508  2228 raft_consensus.cc:2368] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 3.
I20260709 13:00:18.258854  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:18.259086  3241 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.259968  1611 raft_consensus.cc:2368] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4.
I20260709 13:00:18.260325  1692 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.260648  3257 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.205880  3300 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.206221  3468 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:00:18.207614  1612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1173) took 1397 ms (client timeout 1805 ms). Trace:
I20260709 13:00:18.208565  1613 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:18.209638  3230 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:18.210850  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 526912us
I20260709 13:00:18.213568  1686 rpcz_store.cc:276] 0709 13:00:16.827600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.828173 (+   573us) service_pool.cc:224] Handling call
0709 13:00:17.108990 (+280817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:17.109227 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:17.109263 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:17.109320 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:17.109363 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.109463 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:17.961746 (+852283us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54002ccc50. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:17.962026 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645140352}
I20260709 13:00:18.227703  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.228001  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 500) took 1882 ms. Trace:
I20260709 13:00:18.228886  3345 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:18.230084  2236 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:18.284672  3426 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:18.284812  3241 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.285676  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.286540  1541 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:18.287268  3471 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.287982  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.288947  1541 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:18.289680  3468 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:18.293030  3468 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:18.235321  2229 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:18.237893  3364 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:00:18.239346  1609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1669) took 1503 ms (client timeout 1742 ms). Trace:
I20260709 13:00:18.155140  1536 rpcz_store.cc:276] 0709 13:00:14.774768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.778719 (+  3951us) service_pool.cc:224] Handling call
0709 13:00:16.440364 (+1661645us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b5800003998. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:00:16.446550 (+  6186us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:16.571807 (+125257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.573367 (+  1560us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.614428 (+ 41061us) log.cc:844] Serialized 26 byte log entry
0709 13:00:18.153895 (+1539467us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:18.154025 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.154139 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:18.154192 (+    53us) raft_consensus.cc:1685] finished
0709 13:00:18.154210 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.154498 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":211,"raft.queue_time_us":59379,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":16567,"spinlock_wait_cycles":7326976,"thread_start_us":1610334,"threads_started":2,"wal-append.queue_time_us":439744,"wal-append.run_cpu_time_us":1899,"wal-append.run_wall_time_us":1002946}
W20260709 13:00:18.305711  1609 rpcz_store.cc:269] 0709 13:00:16.736327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.858551 (+122224us) service_pool.cc:224] Handling call
0709 13:00:18.238936 (+1380385us) spinlock_profiling.cc:243] Waited 72.5 ms on lock 0x7b5400190854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:18.239150 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166710016}
I20260709 13:00:18.306380  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.306648  3347 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:18.307164  3364 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:18.307235  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:18.307785  3364 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:18.308429  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:18.309191  1543 raft_consensus.cc:2368] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5.
I20260709 13:00:18.220719  1616 rpcz_store.cc:276] 0709 13:00:16.622845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.989769 (+366924us) service_pool.cc:224] Handling call
0709 13:00:17.020540 (+ 30771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:17.239737 (+219197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.078916 (+839179us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b5400190fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:18.079313 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502617344}
I20260709 13:00:18.308907  3364 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:18.310125  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.310285  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:18.310494  3364 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.310905  1543 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:18.311494  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:18.311754  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:18.312672  1543 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:18.312788  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:18.251273  1689 rpcz_store.cc:276] 0709 13:00:16.820889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.838192 (+ 17303us) service_pool.cc:224] Handling call
0709 13:00:18.090002 (+1251810us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54000f6c54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:18.147440 (+ 57438us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641447808}
I20260709 13:00:18.309012  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:18.316128  1609 raft_consensus.cc:2368] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3.
I20260709 13:00:18.317198  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:18.319494  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:18.255851  1614 rpcz_store.cc:276] 0709 13:00:17.072034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.073883 (+1001849us) service_pool.cc:224] Handling call
0709 13:00:18.074411 (+   528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.186864 (+112453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.187159 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.265635  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:18.321195  1611 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:18.321092  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:18.321936  1614 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.322324  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:18.322458  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:18.248714  2235 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:18.323844  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 735) took 1068 ms. Trace:
I20260709 13:00:18.324225  2235 rpcz_store.cc:276] 0709 13:00:17.255645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.085916 (+830271us) service_pool.cc:224] Handling call
0709 13:00:18.323790 (+237874us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.225054  3202 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.284215  1538 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:18.229960  3410 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:18.325651  3410 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.325275  3202 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.326179  3410 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.328207  3410 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:18.293550  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:18.328828  1616 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.282117  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:18.330155  1615 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:18.331180  1542 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:18.233536  3411 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.293823  1686 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 4. Ops: []
I20260709 13:00:18.334151  1615 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:18.334329  1540 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:18.296573  3302 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:18.298091  3230 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.335233  3411 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.335781  1615 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:18.154580  1683 rpcz_store.cc:276] 0709 13:00:16.829888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.835305 (+  5417us) service_pool.cc:224] Handling call
0709 13:00:16.901909 (+ 66604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:16.902330 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.902369 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.902835 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 13:00:17.078947 (+176112us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:17.148106 (+ 69159us) log.cc:844] Serialized 14 byte log entry
0709 13:00:17.181949 (+ 33843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:17.185175 (+  3226us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:18.153475 (+968300us) raft_consensus.cc:1685] finished
0709 13:00:18.153512 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.153776 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1553,"prepare.queue_time_us":142797,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":5173,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":42705,"thread_start_us":218427,"threads_started":2,"wal-append.queue_time_us":227105}
I20260709 13:00:18.299220  3261 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.282864  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 283) took 1153 ms. Trace:
I20260709 13:00:18.337610  1613 rpcz_store.cc:276] 0709 13:00:17.129227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.944122 (+814895us) service_pool.cc:224] Handling call
0709 13:00:18.282802 (+338680us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.338109  1609 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.342474  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2111) took 1908 ms. Trace:
I20260709 13:00:18.342993  1535 rpcz_store.cc:276] 0709 13:00:16.433712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.440509 (+  6797us) service_pool.cc:224] Handling call
0709 13:00:18.342080 (+1901571us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54000e92d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:18.342275 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675701632}
I20260709 13:00:18.347029  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:18.347980  2237 raft_consensus.cc:2368] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3.
I20260709 13:00:18.348919  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:18.349037  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:18.349864  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:18.302449  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:18.284015  2230 rpcz_store.cc:276] 0709 13:00:16.345609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:16.658147 (+312538us) service_pool.cc:224] Handling call
0709 13:00:16.658517 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:16.658877 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:16.658908 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:16.659407 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 13:00:16.688416 (+ 29009us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:16.688905 (+   489us) log.cc:844] Serialized 14 byte log entry
0709 13:00:16.692438 (+  3533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:16.692578 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:18.210032 (+1517454us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b18003e2fc0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:00:18.227395 (+ 17363us) raft_consensus.cc:1685] finished
0709 13:00:18.227516 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.227774 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64184,"prepare.queue_time_us":264,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":32169,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":8318,"spinlock_wait_cycles":27727232,"thread_start_us":31590,"threads_started":2,"wal-append.queue_time_us":63476}
I20260709 13:00:18.307526  1536 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:18.242080  1509 leader_election.cc:400] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:18.351415  1509 leader_election.cc:403] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:18.319696  3459 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:18.280385  1612 rpcz_store.cc:269] 0709 13:00:16.806606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.914769 (+1108163us) service_pool.cc:224] Handling call
0709 13:00:18.203520 (+288751us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.205623  3489 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 13:00:18.207262  1539 raft_consensus.cc:2410] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 13:00:18.336771  1542 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:18.317852  3300 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:18.300686  3469 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:00:18.339069  3484 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.340857  1614 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:18.340651  3408 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.347697  1616 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:18.355188  1692 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:18.362833  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:18.363405  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 159657us
I20260709 13:00:18.364236  3202 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.364989  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:18.366628  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:18.366920  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:18.367175  1534 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:18.368101  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:18.369041  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 66749.5us
I20260709 13:00:18.370460  3261 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:18.370993  1609 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:18.371152  1610 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:18.406108  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.401669  1686 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.299527  3451 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.306391  1507 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:18.401683  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:18.411227  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 105030us
I20260709 13:00:18.411401  1692 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.412215  1507 leader_election.cc:400] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3.
I20260709 13:00:18.412518  1507 leader_election.cc:403] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:18.371349  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.414611  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.414891  3257 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.415520  1691 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:18.372990  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:18.358165  3473 log.cc:927] Time spent T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.774s	user 0.007s	sys 0.012s
I20260709 13:00:18.386200  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.415832  1534 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:18.417963  3451 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3.
I20260709 13:00:18.394627  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.420919  1541 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:18.422772  1541 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:18.398672  1536 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.398763  3414 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:18.416476  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.399868  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:18.400849  3328 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.401647  2232 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:18.402179  1543 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:18.402823  2230 raft_consensus.cc:2368] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4.
I20260709 13:00:18.403477  1537 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.411463  3484 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.398347  1685 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:18.437763  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:18.421105  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1671) took 1585 ms. Trace:
I20260709 13:00:18.403824  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.400841  3303 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:18.476387  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.490176  1618 rpcz_store.cc:276] 0709 13:00:16.835403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.966603 (+1131200us) service_pool.cc:224] Handling call
0709 13:00:17.967134 (+   531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.059424 (+ 92290us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:18.098541 (+ 39117us) log.cc:844] Serialized 14 byte log entry
0709 13:00:18.245819 (+147278us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:18.245878 (+    59us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:18.252029 (+  6151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.420738 (+168709us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b5400389354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:18.420821 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.420990 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20771,"prepare.queue_time_us":34629,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":11862,"raft.queue_time_us":42802,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":49014,"spinlock_wait_cycles":31198336,"thread_start_us":19423,"threads_started":1,"wal-append.queue_time_us":254243}
I20260709 13:00:18.539824  2233 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:18.544667  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 509) took 1994 ms. Trace:
I20260709 13:00:18.548435  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:18.545307  2234 rpcz_store.cc:276] 0709 13:00:16.549964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.108513 (+558549us) service_pool.cc:224] Handling call
0709 13:00:17.925221 (+816708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:18.039379 (+114158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.536542 (+497163us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b540023d7d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:18.544587 (+  8045us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1833481984}
I20260709 13:00:18.534143  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 730) took 1284 ms. Trace:
W20260709 13:00:18.555965  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 160487us
I20260709 13:00:18.561806  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:18.573215  1689 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:18.579265  3484 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.580325  3495 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:00:18.579370  3494 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.145s	user 0.002s	sys 0.000s
I20260709 13:00:18.581286  3495 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:18.583832  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 187178us
I20260709 13:00:18.584401  1687 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:00:18.589509  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 70362.4us
I20260709 13:00:18.558251  2231 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:18.599182  1537 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:18.559661  1683 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:18.610939  1689 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:18.612910  1657 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:18.594139  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:18.614265  3347 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.604820  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.602834  3414 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.557631  2232 rpcz_store.cc:276] 0709 13:00:17.248918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.276034 (+ 27116us) service_pool.cc:224] Handling call
0709 13:00:18.503287 (+1227253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:18.642694  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:18.649299  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:18.619027  3482 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.651760  1692 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:18.652966  1582 leader_election.cc:400] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:18.654068  1582 leader_election.cc:403] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:00:18.653990  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 44815.3us
I20260709 13:00:18.625701  3408 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.638646  3495 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.612463  3489 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:00:18.657789  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:18.658262  3489 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:18.658733  3489 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:18.659946  3489 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:18.661732  3489 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.662627  2230 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.626623  3162 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:18.649367  3451 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.631984  3381 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:18.656980  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.663669  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:18.667766  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 309341us
I20260709 13:00:18.667479  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.671407  2234 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:18.672454  2233 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:18.634407  3400 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.677690  3259 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:18.678390  3408 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
W20260709 13:00:18.678500  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.678426  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:18.687460  1584 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:18.684752  3451 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.689141  3384 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:18.673678  3414 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:18.669287  3482 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:18.679493  3308 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:18.693104  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.689760  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:18.679335  3347 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.697917  3288 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:18.718163  2234 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:18.698609  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.699748  3381 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.701117  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:18.702088  3468 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:18.704460  1686 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:18.714804  1613 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:18.715575  2236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 515) took 1567 ms (client timeout 1674 ms). Trace:
I20260709 13:00:18.722272  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:18.729859  1508 connection.cc:256] RPC call timeout handler was delayed by -1.3129824719999874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
I20260709 13:00:18.714608  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:18.715541  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.715965  2229 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 13:00:18.719396  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 54038.5us
I20260709 13:00:18.720671  1687 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:18.734241  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.704491  3261 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:18.683504  3408 consensus_queue.cc:1059] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 13:00:18.726302  2236 rpcz_store.cc:269] 0709 13:00:17.148586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.233520 (+ 84934us) service_pool.cc:224] Handling call
0709 13:00:18.715241 (+1481721us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x7b54004bbed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:18.715457 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2287826816}
I20260709 13:00:18.689721  3384 raft_consensus.cc:697] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:18.737884  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:18.737691  3288 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:18.727931  3189 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:18.745366  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 87728.6us
I20260709 13:00:18.747408  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:18.748221  1686 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:18.738045  3411 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.742218  3499 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:18.758543  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.759253  1509 leader_election.cc:400] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4.
I20260709 13:00:18.762669  1509 leader_election.cc:403] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:18.760300  3240 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:18.755600  1542 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:18.764102  2235 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.742761  3288 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:18.746955  3495 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:18.774577  3495 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.775455  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 737) took 1387 ms. Trace:
I20260709 13:00:18.775965  2231 rpcz_store.cc:276] 0709 13:00:17.388124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.228341 (+840217us) service_pool.cc:224] Handling call
0709 13:00:18.477543 (+249202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:18.604360 (+126817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.775209 (+170849us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54007102d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:18.775365 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":499066752}
I20260709 13:00:18.761246  3202 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:18.778913  3202 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:18.780441  3202 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:18.782055  2228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 734) took 1532 ms. Trace:
I20260709 13:00:18.782557  2228 rpcz_store.cc:276] 0709 13:00:17.250010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.082650 (+832640us) service_pool.cc:224] Handling call
0709 13:00:18.781817 (+699167us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b5400599cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:18.781985 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303770496}
I20260709 13:00:18.783872  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.784540  1538 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:18.785741  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.787108  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:18.788307  1508 leader_election.cc:400] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3.
I20260709 13:00:18.788906  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.789803  1691 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:18.791013  1691 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:18.790841  3240 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:18.757668  3347 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.730688  3308 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:18.796973  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.739342  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.691985  3162 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:18.768422  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:18.771068  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.763973  3228 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:18.797449  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.785501  3411 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:18.799736  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 85415.7us
I20260709 13:00:18.804263  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.766182  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:18.794324  3288 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.792677  3240 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.820971  1688 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 13:00:18.823680  1539 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
W20260709 13:00:18.824059  1583 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.674s (SENT)
I20260709 13:00:18.824616  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:18.821131  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:18.826944  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:18.827243  3364 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:18.828634  3228 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.812060  3347 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4.
I20260709 13:00:18.758038  3189 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.794852  3408 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:18.819309  3254 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:18.813366  3261 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:00:18.840129  1508 leader_election.cc:400] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4.
I20260709 13:00:18.805155  3499 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:18.832268  3468 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:18.855360  1543 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:18.855953  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.856945  1539 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:18.857498  3408 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.835105  3364 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:18.863204  1685 raft_consensus.cc:2410] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:18.863989  1616 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:18.860759  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:18.868172  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:18.868477  1691 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:18.869330  1686 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:18.870061  1538 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:18.847909  3495 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.866977  3364 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:18.876873  3504 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:00:18.877753  3500 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:00:18.865222  3451 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.879755  3308 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:18.890396  2235 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:18.890636  1616 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:18.891953  1686 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:18.866137  3261 consensus_queue.cc:1243] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer 8a6a5ae1b7f240408b3ddd30d8a18262 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:00:18.870203  3411 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:18.902339  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 81298.8us
I20260709 13:00:18.904660  1686 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:18.906283  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 45611.8us
I20260709 13:00:18.887778  3400 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.909215  1686 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:18.911603  1618 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:18.913993  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 289) took 1779 ms. Trace:
I20260709 13:00:18.913684  3355 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:18.899962  3451 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:18.901386  3482 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:18.900604  3254 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:18.928066  3228 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.893220  3384 consensus_queue.cc:237] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:18.915534  1617 rpcz_store.cc:276] 0709 13:00:17.130754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:18.153052 (+1022298us) service_pool.cc:224] Handling call
0709 13:00:18.190035 (+ 36983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:18.416355 (+226320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:18.416402 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:18.417033 (+   631us) log.cc:844] Serialized 26 byte log entry
0709 13:00:18.605473 (+188440us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:18.605517 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:18.897882 (+292365us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b540014afd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:18.897932 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:18.897980 (+    48us) raft_consensus.cc:1685] finished
0709 13:00:18.897993 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:18.910406 (+ 12413us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508507520,"thread_start_us":187958,"threads_started":1,"wal-append.queue_time_us":304053}
I20260709 13:00:18.930325  2230 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:18.935405  3355 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:18.935760  1685 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:00:18.936127  3355 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.937876  3408 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.945498  1536 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:18.946189  3257 raft_consensus.cc:2764] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:00:18.946460  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:18.946633  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.947345  1542 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:18.948314  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.948972  1542 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:18.950464  3499 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:18.950702  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:18.952751  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:18.958145  1438 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 339) took 5291 ms. Trace:
I20260709 13:00:18.958842  1535 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:18.935491  3259 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:18.955540  1582 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:18.959781  3259 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:18.969352  3230 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:18.970063  3230 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:18.970674  3259 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.971290  3230 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:18.972859  3230 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.973335  3259 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:18.976328  3414 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:18.980515  3400 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:18.983248  1440 catalog_manager.cc:5697] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 2 to 3, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:18.986392  3381 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:18.986840  3381 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:18.987484  3355 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:18.987756  3381 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:18.990669  3162 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:18.994832  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:18.994527  1440 catalog_manager.cc:5697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 2 to 3, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:18.997452  1440 catalog_manager.cc:5697] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 2 to 4, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:18.999653  1440 catalog_manager.cc:5697] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 2 to 3, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 3 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:19.003185  1507 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:18.996253  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:18.996057  3414 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:19.007306  3499 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.015000  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:19.016103  3303 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.016248  2203 leader_election.cc:400] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5.
I20260709 13:00:19.016655  2203 leader_election.cc:403] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:00:19.016567  3303 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.017158  3303 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.019465  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:19.019884  3349 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5.
I20260709 13:00:19.021587  3257 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:00:19.022289  1610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 284) took 1892 ms (client timeout 2164 ms). Trace:
I20260709 13:00:19.022168  3257 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:19.022775  1610 rpcz_store.cc:269] 0709 13:00:17.129482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:17.916199 (+786717us) service_pool.cc:224] Handling call
0709 13:00:19.022118 (+1105919us) spinlock_profiling.cc:243] Waited 81.7 ms on lock 0x7b54001a63d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:19.022227 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187990784}
I20260709 13:00:19.024276  3257 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:19.025796  3499 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:19.026219  3303 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.027969  3499 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:19.013226  3347 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:19.029922  3347 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:19.030887  3347 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:19.032183  3347 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.034727  3189 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:19.035887  3162 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:19.036514  3162 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:19.037163  2203 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:19.037582  3162 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:19.038676  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:19.038767  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.039498  1536 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:19.039848  3189 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.040414  3418 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.041157  2230 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:19.041522  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:19.041810  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:19.042428  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:19.042689  2229 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:18.989255  3381 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.043670  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:19.044035  2237 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:19.045583  3259 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.046362  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:19.046226  3259 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:19.047048  1692 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 2: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:19.041128  3418 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:00:19.048173  3499 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.048290  3259 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:19.048791  3499 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:19.048906  3259 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:19.049836  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:19.050159  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:19.050601  3400 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:19.051507  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.052255  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:19.052454  3499 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:19.051708  3400 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.052826  2235 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:19.053201  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.053846  1536 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:19.055871  1541 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:19.057739  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:19.058131  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55052.7us
I20260709 13:00:19.060117  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:19.057852  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:19.060976  1541 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:19.061049  2228 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:19.045679  3308 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.061914  3308 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.062286  3308 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.060997  3254 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:19.060165  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:19.050227  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:19.067010  3257 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:19.067497  2229 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:19.067667  3257 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.067996  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:19.068306  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:19.070363  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 58580.1us
I20260709 13:00:19.073068  2235 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:19.068789  3257 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.074774  3257 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.075012  1583 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:19.075232  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:19.076052  1541 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:19.075367  3303 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:19.077615  3468 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:19.069111  3228 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:19.083345  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:19.088236  1507 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:19.080704  2203 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:19.092730  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:19.093284  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:19.115163  3314 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:19.092401  3475 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:00:19.111930  3303 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:19.120357  3303 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:19.120769  3303 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:18.979689  3288 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:19.100472  3499 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:19.146201  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 48826.9us
W20260709 13:00:19.150116  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 57491.6us
I20260709 13:00:19.150477  2203 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:19.125717  3260 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:19.146221  1584 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.500s (SENT)
I20260709 13:00:19.151924  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:19.156090  1692 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:19.152907  1691 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 4. Ops: []
I20260709 13:00:19.096700  3202 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:00:19.160414  2203 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:19.161239  3355 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.132962  3228 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:19.163098  1543 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:19.165201  1539 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:19.112172  3364 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.167244  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:19.171427  3346 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.172111  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.172955  1535 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:19.163609  3355 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:19.173869  3355 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.174088  3259 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:19.175822  3259 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:19.173895  3421 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:00:19.177853  3349 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:19.178330  3349 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:19.178685  3349 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:19.180940  3364 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:19.184311  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:19.186235  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.186945  1541 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:19.193542  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:18.958998  1438 rpcz_store.cc:276] 0709 13:00:13.666160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:13.666300 (+   140us) service_pool.cc:224] Handling call
0709 13:00:14.010210 (+343910us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:00:18.957867 (+4947657us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:15.624910 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:15.714320 (+ 89410us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:15.714358 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:00:15.714474 (+   116us) tablet.cc:662] Decoding operations
0709 13:00:15.714918 (+   444us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:15.714986 (+    68us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:15.715018 (+    32us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:00:15.715335 (+   317us) tablet.cc:701] Row locks acquired
0709 13:00:15.715348 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:00:15.715492 (+   144us) write_op.cc:270] Start()
0709 13:00:15.715604 (+   112us) write_op.cc:276] Timestamp: P: 1783602015715446 usec, L: 0
0709 13:00:15.715620 (+    16us) op_driver.cc:348] REPLICATION: starting
0709 13:00:15.721247 (+  5627us) log.cc:844] Serialized 929 byte log entry
0709 13:00:16.206779 (+485532us) op_driver.cc:464] REPLICATION: finished
0709 13:00:16.587157 (+380378us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08c9e20ac 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe093522ef0
0709 13:00:16.905971 (+318814us) write_op.cc:301] APPLY: starting
0709 13:00:16.908031 (+  2060us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:16.934468 (+ 26437us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:16.935955 (+  1487us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:16.962734 (+ 26779us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:16.963012 (+   278us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:00:16.963082 (+    70us) write_op.cc:312] APPLY: finished
0709 13:00:16.963372 (+   290us) log.cc:844] Serialized 29 byte log entry
0709 13:00:16.963602 (+   230us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:16.963719 (+   117us) write_op.cc:454] Released schema lock
0709 13:00:16.963774 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:16.964278 (+   504us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:16.964323 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":118,"reported_tablets":6,"spinlock_wait_cycles":673864448,"child_traces":[["op",{"apply.queue_time_us":676403,"apply.run_cpu_time_us":2885,"apply.run_wall_time_us":65253,"num_ops":2,"prepare.queue_time_us":215986,"prepare.run_cpu_time_us":4194,"prepare.run_wall_time_us":129885,"replication_time_us":470499,"spinlock_wait_cycles":3894656,"thread_start_us":608999,"threads_started":2}]]}
I20260709 13:00:19.150315  3418 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:00:19.157757  3414 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:00:19.199138  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:19.201078  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:19.208312  3291 raft_consensus.cc:2764] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:00:19.208102  3308 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.214720  3384 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.215269  3384 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.215641  3384 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.195665  3408 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:19.233059  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:19.250249  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:19.243409  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.257603  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.197824  3259 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.218534  3288 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:19.264966  3189 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:19.268162  3349 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.271564  3303 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.208145  3364 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.293291  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:19.306190  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 154989us
I20260709 13:00:19.306999  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.261926  3259 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.311870  1541 raft_consensus.cc:2410] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:19.220249  3300 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
W20260709 13:00:19.316159  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:19.314991  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 57753.2us
W20260709 13:00:19.308633  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 212112us
I20260709 13:00:19.320714  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:19.324465  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.325353  1539 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:00:19.329036  3489 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.667s	user 0.008s	sys 0.001s
I20260709 13:00:19.329232  3308 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:19.332338  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:19.332767  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.300475  3189 raft_consensus.cc:697] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:19.261639  3408 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:19.355576  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:19.357249  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 61064us
I20260709 13:00:19.348080  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:19.348759  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:19.358271  1691 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:19.310382  3259 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.355127  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.344763  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:19.347191  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:19.292379  3355 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.362713  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:19.320034  3288 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:19.375171  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:19.377064  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.339323  3240 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.381557  1541 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:19.358125  3303 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:19.314810  3364 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:19.396900  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.397528  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:19.397759  1538 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:19.393958  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:19.394655  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:19.395083  1617 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 13:00:19.395802  1618 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 4. Ops: []
I20260709 13:00:19.386436  1534 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:00:19.396150  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.396330  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:19.412992  1543 raft_consensus.cc:2376] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5. Re-sending same reply.
I20260709 13:00:19.409873  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:19.407580  1534 pending_rounds.cc:85] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 1
I20260709 13:00:19.414037  1618 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.407934  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:19.408753  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:19.411484  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:19.419703  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 105708us
I20260709 13:00:19.407733  1616 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:19.416742  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:19.384963  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:19.421571  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:19.428251  1542 raft_consensus.cc:2376] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4. Re-sending same reply.
I20260709 13:00:19.425168  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.351368  3516 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 13:00:19.399780  1683 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:19.421517  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:19.430511  2201 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:19.410311  3512 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 13:00:19.430869  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.431684  1538 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:19.417349  1534 pending_rounds.cc:107] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:00:19.433488  1541 raft_consensus.cc:2368] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:19.431483  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:19.433808  1583 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:19.401635  3288 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:19.432374  1611 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:19.437760  2235 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:19.438004  1612 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:19.460103  3260 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.460659  3260 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:19.460945  3421 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:19.461063  3260 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.411412  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:19.462831  1685 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:19.438745  2229 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:19.439311  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.440855  3308 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.443120  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:19.443602  3355 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:19.471891  1684 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:19.444741  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:19.480026  1617 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:19.432519  1534 raft_consensus.cc:2889] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:19.417804  3303 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.457034  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:19.461169  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:19.438570  3364 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:19.467319  1618 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:19.471975  1616 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:19.470044  1583 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:19.474197  3515 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:00:19.478224  3308 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.471865  3468 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.481580  3189 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.405922  3240 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:19.435899  3257 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.522082  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:19.626159  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 279038us
I20260709 13:00:19.547168  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.628144  1543 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:19.516172  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:19.424149  1535 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:19.583207  2236 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:19.582445  1617 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:19.636217  2235 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.446220  3349 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:19.482923  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:19.641830  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:19.649282  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 179308us
W20260709 13:00:19.631268  3347 consensus_meta.cc:287] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.599s	user 0.013s	sys 0.000s
W20260709 13:00:19.550300  3500 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:00:19.525851  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:19.630959  3308 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.560768  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:19.658017  3325 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:19.658035  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:19.663049  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:19.663971  2234 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:19.582578  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:19.525175  3254 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:19.657835  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:19.668107  1583 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:19.530761  3414 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.126s
I20260709 13:00:19.677354  2203 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:19.680462  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:19.679915  3349 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.654524  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:19.655256  3400 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:19.688555  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:19.530550  3162 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.129s
I20260709 13:00:19.524879  3260 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:19.701539  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.676236  1583 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:19.702661  2203 leader_election.cc:400] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3.
I20260709 13:00:19.703486  2203 leader_election.cc:403] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:00:19.666404  2229 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:19.680356  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:19.680462  3288 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:19.707201  3468 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:19.717397  3468 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:19.717939  3468 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:19.686545  3202 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:19.721649  3202 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:19.686343  3355 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.691144  3421 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.708214  1583 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:19.716228  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.722239  3259 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.722651  3257 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:19.705633  1536 raft_consensus.cc:2410] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:19.726189  2235 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:19.743978  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:19.682035  3254 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:19.682807  3384 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.723165  3421 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:19.700652  3520 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:19.709152  3408 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:19.703297  3349 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.725874  2232 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
W20260709 13:00:19.748286  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 57979.7us
W20260709 13:00:19.779193  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:19.762193  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 108863us
W20260709 13:00:19.744905  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44925.8us
I20260709 13:00:19.790975  2230 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:19.791356  3257 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:00:19.778370  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 68631.9us
I20260709 13:00:19.792778  1584 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:19.793691  3162 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:19.794014  3414 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:19.794744  3162 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:19.795212  3162 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:19.795110  3347 consensus_queue.cc:1059] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:19.796444  3364 raft_consensus.cc:3055] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:19.796247  3162 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.796885  3364 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:19.797483  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.798237  1543 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:19.798120  3364 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.800213  3364 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:19.800429  3421 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:19.801928  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.802692  3384 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:19.802619  1543 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:19.804590  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:19.805394  3230 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:19.805480  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.805913  3230 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:19.806308  3230 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:19.806727  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:19.793936  3257 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:19.723124  3260 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:19.808017  1508 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:19.811910  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.790302  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:19.817080  3408 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.820602  3529 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:00:19.822650  1691 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:00:19.823210  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:19.825615  3523 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.145s	user 0.001s	sys 0.001s
I20260709 13:00:19.826568  1687 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:19.748801  3400 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:19.746415  3355 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3.
I20260709 13:00:19.834283  2237 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:19.835490  1541 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:19.837145  3418 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:19.754746  3254 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:19.749229  3381 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.845489  3489 raft_consensus.cc:3055] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:19.846777  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:19.848444  1543 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:19.846772  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 56559.6us
I20260709 13:00:19.834899  3162 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:19.776029  3471 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 13:00:19.771859  3517 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:00:19.869338  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:19.870443  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:19.875288  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 82584.2us
I20260709 13:00:19.846223  3400 raft_consensus.cc:697] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:19.886761  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:19.848112  3489 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:19.882493  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:19.882207  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 74290us
I20260709 13:00:19.895749  3384 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.896323  3384 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.896713  3384 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.897583  3528 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.898134  3528 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.898517  3528 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.902076  3471 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:19.902164  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:19.902706  3471 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:19.903992  3355 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:19.903708  3471 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.904592  3355 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:19.904985  3471 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.905619  3355 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.907526  1584 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:19.908872  1584 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:19.910248  3408 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:19.910573  1584 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:19.912019  3408 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:19.912389  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:19.912292  3308 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.913175  3529 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:19.913740  3475 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:19.915323  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:19.916425  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:19.914253  3381 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:00:19.915253  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:19.920279  1537 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:19.921331  1508 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:19.932452  1543 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:19.933576  1508 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:19.924548  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:19.881075  3240 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:19.941010  1540 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:19.853859  3300 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:00:19.935717  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:19.944077  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:19.944862  1687 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:19.945675  2236 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:19.946189  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:19.945652  1543 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
W20260709 13:00:19.947597  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:19.887722  3254 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:19.951931  1540 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:00:19.952220  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 131804us
I20260709 13:00:19.912411  3408 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:19.958635  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:19.889021  3325 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:00:19.964648  1537 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:19.919070  3259 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:19.940062  3291 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.941322  2229 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:19.952392  3516 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:19.892299  3257 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:00:19.956689  3408 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:19.987759  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:19.990756  1689 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:19.974431  3291 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.975394  3347 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:19.980330  3414 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:19.994307  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:20.003626  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:20.005152  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:19.995676  1508 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:00:20.003825  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 91486.2us
I20260709 13:00:19.990919  3529 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:20.018603  3529 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:20.019047  3529 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:19.990728  3516 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:19.975500  3254 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.913894  3400 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:19.998422  3291 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.002808  3418 raft_consensus.cc:2764] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:00:19.998550  3347 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:19.995688  3536 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:19.992501  3300 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
W20260709 13:00:20.028530  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 110262us
I20260709 13:00:19.999248  3230 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:20.022964  3349 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:20.040395  3541 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:20.019387  3537 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:20.031064  3347 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:20.014618  3414 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:20.032749  3536 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.049302  3531 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.051548  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:20.052512  3308 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:20.057456  3489 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.061964  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:20.130611  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 68784.7us
W20260709 13:00:20.125183  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 137540us
I20260709 13:00:20.131639  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:20.048502  3254 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:20.066134  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:20.079981  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 84356.9us
I20260709 13:00:20.088258  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:20.035077  3539 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:20.133358  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:20.140991  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:20.145236  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:20.130307  3364 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:20.124202  3536 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.159369  1508 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:20.124285  3543 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:20.147315  3482 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:20.138561  3539 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:00:20.169093  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 164011us
I20260709 13:00:20.146574  3325 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:20.171089  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 124911us
I20260709 13:00:20.187955  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:20.156742  3537 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:20.154707  3260 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:20.158135  3519 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:20.162250  3257 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:20.162376  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:20.138517  3468 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.175745  3482 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.153966  3259 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 13:00:20.175339  3543 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.156281  3475 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:00:20.198431  1509 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:20.158674  3531 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:20.200105  3325 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:20.187044  3539 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.194665  3544 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:20.196113  3489 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:20.246130  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 58315.5us
I20260709 13:00:20.168617  3516 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.179813  3189 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:20.253213  3384 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:20.256686  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 58399.5us
I20260709 13:00:20.308435  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:20.310125  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:20.257526  3538 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.094s	user 0.001s	sys 0.004s
I20260709 13:00:20.200574  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:20.249691  3543 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.260694  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:20.324760  1540 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
W20260709 13:00:20.260156  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 110209us
W20260709 13:00:20.325874  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:20.246469  1685 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.284507  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:20.250933  3381 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:20.250761  3482 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.301034  3230 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:20.300824  3325 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:20.302927  1684 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:20.303505  3554 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:20.306960  1584 leader_election.cc:400] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:20.362797  1584 leader_election.cc:403] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:20.363555  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 76675.1us
W20260709 13:00:20.298609  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 257252us
I20260709 13:00:20.337128  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:20.334739  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:20.338218  3547 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:20.306525  3537 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:20.335249  3381 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:20.325351  3418 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:20.334548  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:20.352321  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:20.364298  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:20.364331  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:20.366364  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 294965us
I20260709 13:00:20.330485  3544 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:20.367651  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:00:20.375456  1657 leader_election.cc:336] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.683s (SENT)
I20260709 13:00:20.347112  3468 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:20.378679  2235 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:20.362588  3554 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:20.387683  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:20.321316  3189 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.387624  3364 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:20.390728  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:20.392603  3468 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:20.374126  3384 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:20.377405  3381 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.373534  3346 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:20.394676  2233 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:20.396067  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:20.386850  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:20.385026  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:20.435321  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 68256.8us
I20260709 13:00:20.369036  3230 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:20.436399  1509 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:20.363957  3551 raft_consensus.cc:2764] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:00:20.435352  2237 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:20.437565  3551 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.438225  3551 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:20.440928  3551 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:20.415128  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 255775us
I20260709 13:00:20.377802  3418 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:20.442382  3418 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.443622  3418 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.436502  3230 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:20.368705  3537 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:00:20.458196  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 93923.5us
I20260709 13:00:20.446041  3551 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:20.388557  3202 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:20.380986  3541 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:00:20.382942  3561 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.447196  3346 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:20.414031  3303 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:00:20.446123  3537 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:20.492661  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:20.467798  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:20.398083  3554 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.467798  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:20.414211  3414 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:20.479373  1618 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:20.465191  3551 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:20.479820  1657 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:20.364408  3528 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:20.428674  3260 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:00:20.491912  3545 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.144s	user 0.003s	sys 0.005s
I20260709 13:00:20.516253  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:20.489764  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:20.496701  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:20.485519  3471 consensus_meta.cc:287] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.568s	user 0.004s	sys 0.008s
I20260709 13:00:20.467905  3240 raft_consensus.cc:3055] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:20.526938  3240 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:20.528227  3240 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.398793  1535 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:20.530457  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:20.531059  3553 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:00:20.519160  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:20.532483  1612 raft_consensus.cc:2410] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:20.532325  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:20.470641  3230 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.535245  3230 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.535754  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:20.431037  3468 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.479091  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:20.489056  2232 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:20.446751  3288 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:00:20.499660  3540 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:20.518642  3308 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:20.558135  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:20.559074  1615 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:20.525532  3259 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:20.526156  1613 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.492265  3555 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.525527  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:20.536790  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:20.540774  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:20.541335  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:20.505318  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:20.510128  3551 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:20.536190  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:20.553146  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:20.509878  2231 raft_consensus.cc:2376] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5. Re-sending same reply.
I20260709 13:00:20.552270  3189 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:20.557492  3573 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:20.521072  3408 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.005s
I20260709 13:00:20.561554  3308 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:20.558086  3528 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:20.527758  3564 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:20.568436  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 88728.7us
I20260709 13:00:20.522852  3202 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:20.585749  3557 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 13:00:20.585748  3562 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:00:20.587082  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:20.588358  3560 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:00:20.589972  2237 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
W20260709 13:00:20.591696  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55684.8us
I20260709 13:00:20.515182  3291 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:20.533296  1535 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:20.610707  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:20.567071  3240 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:20.612672  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:20.562052  3515 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:20.619926  1540 raft_consensus.cc:2368] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:20.626307  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:20.579713  3547 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:00:20.631700  3549 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.384s	user 0.002s	sys 0.003s
I20260709 13:00:20.632146  3349 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:20.632018  3547 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.632928  3517 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.633307  3349 raft_consensus.cc:3055] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:20.633807  3349 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:20.634195  3570 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:00:20.635133  3541 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:20.636436  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:20.636509  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:20.637132  1538 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:20.611294  3529 consensus_meta.cc:287] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.585s	user 0.010s	sys 0.000s
I20260709 13:00:20.634696  3349 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.642316  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:20.643186  1541 raft_consensus.cc:2410] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:00:20.642897  3515 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.644109  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:20.644258  3202 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:20.645568  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:20.646170  1537 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.590304  1609 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:20.631479  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:20.566517  3531 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:20.648558  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:20.655498  2237 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:20.649442  1692 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:20.628038  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:20.659588  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:20.661218  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:20.673565  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:20.620600  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:20.674481  1540 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.645246  3515 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:20.629524  3300 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:20.613620  3408 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:20.659142  1611 raft_consensus.cc:2410] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:20.549667  3544 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.566362  3555 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:20.654035  3529 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:20.564464  3346 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.762698  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:20.659329  3257 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.639186  3517 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:20.804471  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 174053us
W20260709 13:00:20.809520  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 408068us
I20260709 13:00:20.836010  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:20.837513  1509 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:20.693632  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:20.838652  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:20.618341  3564 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.840480  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:20.841413  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:20.845327  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 217405us
I20260709 13:00:20.846129  2203 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:20.838652  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:20.787426  3349 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:20.809237  3259 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.851884  3346 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:20.855695  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:20.857131  1613 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:20.858385  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:20.851531  1537 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:20.762368  3468 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:20.750499  1612 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:20.868844  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:20.875936  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:20.877326  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:20.878121  1612 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:20.858579  1583 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:20.814396  3347 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.664s	user 0.010s	sys 0.000s
I20260709 13:00:20.750227  3515 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:20.841253  3257 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:20.834515  3528 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 13:00:20.697573  3568 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.409s
I20260709 13:00:20.689167  3260 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:20.910153  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 51658.8us
I20260709 13:00:21.358560  3564 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:21.051834  1611 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:20.883282  3259 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.865670  2232 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:00:21.121762  3291 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:21.808398  3589 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.321s	user 0.001s	sys 0.000s
I20260709 13:00:21.787715  3588 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:20.952005  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:21.839701  3536 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.663s	user 0.013s	sys 0.000s
I20260709 13:00:20.815003  3482 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:21.225107  1614 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:21.711318  3555 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:20.935853  3347 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:20.909379  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:20.952266  3400 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:21.120215  1538 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
W20260709 13:00:21.082511  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 213778us
I20260709 13:00:20.905542  3547 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:21.633467  2236 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.000s	sys 0.008s
I20260709 13:00:21.977631  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:22.018659  2236 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.216s	user 0.001s	sys 0.008s
W20260709 13:00:21.905042  1655 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.001s	sys 0.007s
W20260709 13:00:21.103765  3489 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.792s	user 0.011s	sys 0.000s
I20260709 13:00:21.995568  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:21.325135  3590 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
W20260709 13:00:21.377851  3537 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.873s	user 0.010s	sys 0.000s
W20260709 13:00:21.806317  3583 log.cc:927] Time spent T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
W20260709 13:00:21.324121  3418 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.880s	user 0.004s	sys 0.007s
W20260709 13:00:21.513217  1535 consensus_meta.cc:287] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.853s	user 0.007s	sys 0.004s
W20260709 13:00:21.360307  3539 consensus_meta.cc:287] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.062s	user 0.000s	sys 0.011s
W20260709 13:00:21.662765  3516 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.942s	user 0.003s	sys 0.012s
W20260709 13:00:21.633215  2203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.006s	sys 0.002s
I20260709 13:00:22.013252  1615 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:21.370018  3384 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:00:21.854379  3346 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.981s	user 0.009s	sys 0.001s
W20260709 13:00:22.026602  2203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.158s	user 0.007s	sys 0.003s
I20260709 13:00:21.098498  3230 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:21.093117  3582 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:21.675552  3559 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:21.092621  3571 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:21.392136  3381 consensus_meta.cc:287] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.924s	user 0.010s	sys 0.000s
I20260709 13:00:21.488981  1584 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:21.283363  1685 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.945s	user 0.008s	sys 0.003s
I20260709 13:00:20.844857  3573 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:00:21.989220  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 1.12694e+06us
I20260709 13:00:21.981855  3257 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:21.633574  1507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.004s	sys 0.004s
W20260709 13:00:21.992460  1540 consensus_meta.cc:287] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.305s	user 0.005s	sys 0.006s
W20260709 13:00:22.019677  1655 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.125s	user 0.002s	sys 0.007s
W20260709 13:00:21.979331  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 992933us
I20260709 13:00:22.010437  1613 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:22.015480  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1671) took 1425 ms. Trace:
W20260709 13:00:22.014518  1610 consensus_meta.cc:287] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.738s	user 0.010s	sys 0.000s
I20260709 13:00:22.020954  1537 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:22.021474  1686 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:20.983407  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:22.014737  3482 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:22.021050  3588 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.027051  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1693) took 1229 ms. Trace:
W20260709 13:00:21.877012  1612 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.998s	user 0.010s	sys 0.002s
W20260709 13:00:21.878851  3554 consensus_meta.cc:287] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.365s	user 0.012s	sys 0.000s
W20260709 13:00:21.881711  3543 consensus_meta.cc:287] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.538s	user 0.004s	sys 0.009s
W20260709 13:00:21.884516  3551 consensus_meta.cc:287] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.269s	user 0.008s	sys 0.003s
W20260709 13:00:22.030740  1506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.007s	sys 0.001s
W20260709 13:00:22.028270  1657 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.091s	user 0.002s	sys 0.009s
W20260709 13:00:22.027468  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 1.18131e+06us
W20260709 13:00:21.865398  3349 consensus_meta.cc:287] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.992s	user 0.007s	sys 0.000s
I20260709 13:00:22.003381  3475 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.038739  2235 raft_consensus.cc:2410] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:21.987149  3529 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:22.022109  3400 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:22.039162  3571 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:21.868368  3561 consensus_meta.cc:287] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.371s	user 0.002s	sys 0.007s
I20260709 13:00:22.003975  3584 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:21.086251  3572 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.006724  3364 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:21.978765  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1698) took 1179 ms. Trace:
I20260709 13:00:21.378707  3517 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.027012  3489 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.129590  3529 raft_consensus.cc:2764] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:00:22.130075  3602 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:00:22.129726  3489 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.130362  3529 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:22.130903  3529 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.127280  3572 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.132097  3489 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.013495  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 451) took 1364 ms. Trace:
I20260709 13:00:21.996302  3259 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.003098  3564 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.132959  3529 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.133466  3564 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.127646  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:22.101686  3543 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:22.074251  1507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.005s	sys 0.009s
I20260709 13:00:22.015301  3555 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.013686  3595 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.073657  3559 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:22.077342  1508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.023s	user 0.000s	sys 0.014s
W20260709 13:00:22.029173  3516 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.335s	user 0.004s	sys 0.012s
I20260709 13:00:22.075273  1540 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:22.026700  1535 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
W20260709 13:00:22.076725  1613 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 313) took 1229 ms (client timeout 1598 ms). Trace:
W20260709 13:00:22.043259  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.082872  1615 rpcz_store.cc:276] 0709 13:00:20.798119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.866560 (+ 68441us) service_pool.cc:224] Handling call
0709 13:00:22.026989 (+1160429us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.077142  1538 rpcz_store.cc:276] 0709 13:00:20.590580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.636155 (+ 45575us) service_pool.cc:224] Handling call
0709 13:00:22.015403 (+1379248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.023097  3600 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:22.101019  1657 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.164s	user 0.002s	sys 0.009s
W20260709 13:00:22.100788  1506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.008s	sys 0.001s
I20260709 13:00:22.013051  3536 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.103024  2203 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:22.075593  1534 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.000s	sys 0.014s
W20260709 13:00:22.052016  3581 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.981s	user 0.004s	sys 0.008s
I20260709 13:00:22.103822  3554 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.019380  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 746) took 1686 ms. Trace:
I20260709 13:00:22.038713  3596 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:22.041702  1685 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:22.026177  3547 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.024858  3537 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:22.099377  3162 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.008s	sys 0.001s
I20260709 13:00:22.029251  3539 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.076316  1610 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:22.082962  3598 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.123991  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:22.083645  3384 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:22.076823  3528 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:22.084239  1612 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
W20260709 13:00:22.099776  3468 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.000s	user 0.009s	sys 0.009s
I20260709 13:00:22.123412  3347 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:22.298657  3581 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.345s	user 0.005s	sys 0.019s
I20260709 13:00:22.351313  1543 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:22.076742  3582 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.298635  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 349) took 2021 ms. Trace:
I20260709 13:00:22.123755  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 1971) took 1279 ms. Trace:
I20260709 13:00:22.104861  3551 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:22.299662  1508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.246s	user 0.000s	sys 0.016s
I20260709 13:00:22.100668  3588 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.305400  3349 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:22.186730  1542 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:22.137512  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:22.306202  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 203253us
I20260709 13:00:22.225865  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:22.307199  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:22.125970  3364 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.308707  1689 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
W20260709 13:00:22.311285  1656 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.137s	user 0.003s	sys 0.012s
I20260709 13:00:22.114531  3571 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.126369  3561 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:22.041730  3381 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.367692  3482 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.298799  2236 rpcz_store.cc:276] 0709 13:00:20.332798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.332930 (+   132us) service_pool.cc:224] Handling call
0709 13:00:20.346181 (+ 13251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:20.346414 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:20.380149 (+ 33735us) log.cc:844] Serialized 14 byte log entry
0709 13:00:22.019041 (+1638892us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:22.019089 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:22.019148 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.019211 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:22.019329 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13767,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":7619,"raft.queue_time_us":356362,"raft.run_wall_time_us":140009,"spinlock_wait_cycles":22248192,"thread_start_us":1549058,"threads_started":2,"wal-append.queue_time_us":538548}
I20260709 13:00:22.353958  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1694) took 1555 ms. Trace:
I20260709 13:00:22.108039  3475 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.184418  3257 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.163058  3489 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.184149  3573 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:22.137900  1537 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:22.298055  3564 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.358290  1541 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:00:22.353016  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 864063us
I20260709 13:00:22.351003  3529 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:22.300071  1613 rpcz_store.cc:269] 0709 13:00:20.847702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.883759 (+ 36057us) service_pool.cc:224] Handling call
0709 13:00:22.076659 (+1192900us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:22.313302  3468 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.213s	user 0.009s	sys 0.009s
W20260709 13:00:22.357064  1657 connection.cc:256] RPC call timeout handler was delayed by -1.2099028319999547s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
I20260709 13:00:22.125603  1611 rpcz_store.cc:276] 0709 13:00:20.799307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.838376 (+ 39069us) service_pool.cc:224] Handling call
0709 13:00:21.978699 (+1140323us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:22.370338  1658 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.580s (SENT) [suppressed 21 similar messages]
I20260709 13:00:22.311264  3596 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.353235  1540 rpcz_store.cc:276] 0709 13:00:20.276790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.638913 (+362123us) service_pool.cc:224] Handling call
0709 13:00:22.298562 (+1659649us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:22.352578  1685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 347) took 2236 ms (client timeout 1709 ms). Trace:
W20260709 13:00:22.388734  1656 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.452s	user 0.003s	sys 0.012s
I20260709 13:00:22.459770  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:22.125530  3584 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.390293  1691 raft_consensus.cc:2368] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:22.388568  1615 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:22.391211  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:22.335933  3555 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.389905  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:22.352355  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:22.132757  2232 rpcz_store.cc:276] 0709 13:00:20.649614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.650712 (+  1098us) service_pool.cc:224] Handling call
0709 13:00:20.802408 (+151696us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:22.011029 (+1208621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.013407 (+  2378us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.351351  3517 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:22.306396  1534 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.235s	user 0.000s	sys 0.023s
W20260709 13:00:22.313170  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 237198us
I20260709 13:00:22.351850  3607 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:22.372038  3347 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:22.367766  3603 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:22.354125  3608 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:22.352486  3162 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:22.458189  1657 connection.cc:256] RPC call timeout handler was delayed by -1.1528019709999739s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
I20260709 13:00:22.209225  3230 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:22.400269  1616 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:22.313007  1535 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 350) took 2034 ms (client timeout 2237 ms). Trace:
I20260709 13:00:22.391870  1692 raft_consensus.cc:2368] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
W20260709 13:00:22.501518  1657 connection.cc:256] RPC call timeout handler was delayed by -1.0291471619999584s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
W20260709 13:00:22.462832  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 71697.3us
I20260709 13:00:22.468894  3349 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:22.355257  1684 rpcz_store.cc:276] 0709 13:00:20.841230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.844730 (+  3500us) service_pool.cc:224] Handling call
0709 13:00:22.120941 (+1276211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2944}
I20260709 13:00:22.458338  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1676) took 1599 ms. Trace:
I20260709 13:00:22.354331  3528 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.467418  1540 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:22.507480  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 352) took 2222 ms. Trace:
I20260709 13:00:22.507989  1540 rpcz_store.cc:276] 0709 13:00:20.284997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.466642 (+2181645us) service_pool.cc:224] Handling call
0709 13:00:22.467101 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:22.507092 (+ 39991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.507404 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.508834  3528 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.509699  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:22.394266  3421 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.698s	user 0.029s	sys 0.036s
I20260709 13:00:22.470566  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:22.470670  3604 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.511593  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1710) took 1451 ms. Trace:
I20260709 13:00:22.472222  3599 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.512197  1611 rpcz_store.cc:276] 0709 13:00:21.060109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.470249 (+1410140us) service_pool.cc:224] Handling call
0709 13:00:22.511493 (+ 41244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.512416  3599 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.513253  3566 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:22.513875  3566 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.474234  3384 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.488204  3517 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.516247  3566 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.516695  3517 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.472743  3596 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:22.472911  3597 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:22.395202  3543 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:22.490259  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 119966us
I20260709 13:00:22.459889  1612 rpcz_store.cc:276] 0709 13:00:20.798381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.876987 (+ 78606us) service_pool.cc:224] Handling call
0709 13:00:22.351192 (+1474205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.314227  3598 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.529297  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:22.494720  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 104929us
I20260709 13:00:22.311362  3595 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.504834  3349 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:22.542531  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:22.496927  3529 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.489691  3573 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.504452  3561 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:22.468796  3260 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.570842  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:22.368891  3259 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:22.470623  3547 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.234246  3400 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.517169  3559 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:22.511444  3604 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.471724  3555 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.492326  3475 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:22.526400  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 66734.2us
W20260709 13:00:22.459443  1685 rpcz_store.cc:269] 0709 13:00:20.070967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.073296 (+  2329us) service_pool.cc:224] Handling call
0709 13:00:22.307096 (+2233800us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:22.501672  1535 rpcz_store.cc:269] 0709 13:00:20.277131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.531968 (+254837us) service_pool.cc:224] Handling call
0709 13:00:22.311503 (+1779535us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.455688  3482 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.500449  3564 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:22.561336  3584 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.531024  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:22.642266  1689 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:22.556264  3529 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:22.540180  1657 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.731s (SENT)
W20260709 13:00:22.643317  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 106764us
I20260709 13:00:22.628837  3596 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:22.493814  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:22.504352  3608 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.523233  1537 rpcz_store.cc:276] 0709 13:00:20.859314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.882975 (+ 23661us) service_pool.cc:224] Handling call
0709 13:00:22.458271 (+1575296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.544317  1616 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:22.504503  3347 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.469204  3257 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.497792  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:22.549397  1582 leader_election.cc:400] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:22.502766  3603 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.552824  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:22.651600  1582 leader_election.cc:403] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:22.553408  1615 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:22.652163  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1712) took 1359 ms. Trace:
I20260709 13:00:22.652546  1615 rpcz_store.cc:276] 0709 13:00:21.312369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.306907 (+994538us) service_pool.cc:224] Handling call
0709 13:00:22.652098 (+345191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.355767  3600 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.499612  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:22.654989  1613 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:22.655431  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1702) took 1855 ms. Trace:
I20260709 13:00:22.655716  1613 rpcz_store.cc:276] 0709 13:00:20.800252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.476349 (+1676097us) service_pool.cc:224] Handling call
0709 13:00:22.655378 (+179029us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.502614  2203 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:22.598359  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 79055.5us
I20260709 13:00:22.475769  3489 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.553936  3349 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.510663  1540 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:22.553720  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:22.556977  3606 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
W20260709 13:00:22.565680  3605 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
I20260709 13:00:22.555251  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:22.555723  3611 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:00:22.629922  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 346) took 2354 ms. Trace:
W20260709 13:00:22.558494  3616 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 13:00:22.556388  3609 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:00:22.530299  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:22.632722  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:22.594830  3346 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.702538  1691 raft_consensus.cc:2410] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:00:22.518880  3162 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.555634  3626 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:22.553973  3624 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:22.554584  3614 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.594525  3418 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.556458  3543 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.726379  3543 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.725914  3582 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:22.708745  3346 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.727759  3603 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:22.518635  3230 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.528687  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:22.568656  3572 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:22.554083  3618 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.734229  3618 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.734735  3482 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.735131  3608 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:22.592371  3561 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.599932  3516 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.628989  3581 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.618269  3612 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.537246  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:22.644819  1657 leader_election.cc:336] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.112s (SENT)
I20260709 13:00:22.646281  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:22.743769  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 97558.5us
I20260709 13:00:22.744781  1584 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:22.522995  3421 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.827s	user 0.030s	sys 0.036s
I20260709 13:00:22.647181  2237 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:22.742980  1690 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:22.549958  3595 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.634725  3619 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.501958  3607 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.640086  3564 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:22.638383  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:22.647398  1683 raft_consensus.cc:2410] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:22.651515  2233 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:22.658713  1689 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:22.642387  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1695) took 1842 ms. Trace:
I20260709 13:00:22.625952  3259 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.648317  3547 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:22.538764  3597 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.646413  3539 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:22.677017  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 127738us
I20260709 13:00:22.618498  3260 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.678215  1537 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:22.779429  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2146) took 2336 ms. Trace:
I20260709 13:00:22.779961  1537 rpcz_store.cc:276] 0709 13:00:20.442814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.648957 (+2206143us) service_pool.cc:224] Handling call
0709 13:00:22.665357 (+ 16400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.779060 (+113703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.779350 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.647624  3475 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.647293  3554 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.650041  3555 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.652585  3596 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:22.690402  2228 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
W20260709 13:00:22.696919  1609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 301) took 2236 ms (client timeout 2072 ms). Trace:
I20260709 13:00:22.653855  3517 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:22.697573  1535 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:22.697409  3468 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:22.698160  3620 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:22.699280  3604 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:22.707296  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:22.707834  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:22.673171  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:22.708379  1615 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:22.706944  3300 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:22.690774  1538 rpcz_store.cc:276] 0709 13:00:20.275827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.311392 (+2035565us) service_pool.cc:224] Handling call
0709 13:00:22.311715 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.311934 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:22.495467 (+183533us) log.cc:844] Serialized 14 byte log entry
0709 13:00:22.629474 (+134007us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:22.629547 (+    73us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:22.629618 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.629705 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:22.629842 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":329885,"raft.queue_time_us":4263,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":5937,"spinlock_wait_cycles":4355328,"thread_start_us":208839,"threads_started":2,"wal-append.queue_time_us":149277}
I20260709 13:00:22.726125  3418 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.640465  2201 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:22.731230  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:22.728741  3543 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:22.732931  3346 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:22.733349  2229 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:22.733668  3230 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:22.731261  1540 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
W20260709 13:00:22.737964  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 235507us
I20260709 13:00:22.751961  1584 leader_election.cc:400] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
W20260709 13:00:22.762616  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 117825us
I20260709 13:00:22.711084  3588 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:22.740762  3561 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:22.761054  3564 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:22.765341  1611 rpcz_store.cc:276] 0709 13:00:20.798629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.513085 (+1714456us) service_pool.cc:224] Handling call
0709 13:00:22.521128 (+  8043us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.574634 (+ 53506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:22.593327 (+ 18693us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:22.597589 (+  4262us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:22.601905 (+  4316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.632116 (+ 30211us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:22.640652 (+  8536us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:22.768679  3259 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.770846  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:22.774545  1582 leader_election.cc:400] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:22.778118  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:22.740988  3624 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.758085  3582 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:22.737712  3626 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.763597  3625 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:22.795475  3475 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.750648  3619 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.746961  3581 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.751164  3612 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.731215  3614 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:22.747927  3516 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.162390  1584 leader_election.cc:403] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:23.163305  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 411511us
I20260709 13:00:23.163646  3625 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:23.164134  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:23.164361  3625 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.165144  3624 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.932646  3596 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:23.126322  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:23.168097  3619 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:23.171308  3619 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:23.168603  1582 leader_election.cc:403] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:23.169734  2236 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:23.172662  3612 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:23.173028  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 398602us
I20260709 13:00:23.125489  3584 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:23.173548  3614 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:23.174222  3614 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
W20260709 13:00:22.966843  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 252369us
I20260709 13:00:23.174090  3584 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.175645  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1697) took 2376 ms. Trace:
I20260709 13:00:23.176479  3584 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:23.169799  3582 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
W20260709 13:00:23.176739  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:23.176051  1615 rpcz_store.cc:276] 0709 13:00:20.799076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.667098 (+1868022us) service_pool.cc:224] Handling call
0709 13:00:22.707960 (+ 40862us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:23.175393 (+467433us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:23.175577 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:23.168613  2230 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 754) took 2699 ms (client timeout 1580 ms). Trace:
I20260709 13:00:23.177811  3517 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:23.178372  2230 rpcz_store.cc:269] 0709 13:00:20.469365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.480708 (+ 11343us) service_pool.cc:224] Handling call
0709 13:00:22.958598 (+2477890us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7eee 0000000000375939 00007fe094a3b1e0 00007fe094b040c4 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:23.168456 (+209858us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6712576}
I20260709 13:00:22.801857  3600 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:23.163295  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:23.180282  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:23.179171  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:23.181875  1684 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:22.929209  3537 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:23.180006  3517 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:22.815578  1688 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
W20260709 13:00:22.931051  1541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 354) took 2604 ms (client timeout 3270 ms). Trace:
W20260709 13:00:23.180025  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:23.155998  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:23.219200  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 511949us
I20260709 13:00:23.219595  2233 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:22.774626  3547 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.757799  3618 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:23.426723  1543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 344) took 3164 ms (client timeout 2113 ms). Trace:
I20260709 13:00:23.228108  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:23.462786  1543 rpcz_store.cc:269] 0709 13:00:20.262546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.262737 (+   191us) service_pool.cc:224] Handling call
0709 13:00:23.426467 (+3163730us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b54003449d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:23.426633 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4648357248}
I20260709 13:00:23.463306  1540 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
W20260709 13:00:23.425796  1542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 347) took 3149 ms (client timeout 2380 ms). Trace:
I20260709 13:00:23.283639  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:23.465790  1692 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:23.159466  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1652) took 3320 ms. Trace:
I20260709 13:00:23.466905  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:23.467818  1692 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:23.248570  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:23.220091  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 579666us
I20260709 13:00:23.360518  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:23.464802  1542 rpcz_store.cc:269] 0709 13:00:20.276167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.631192 (+355025us) service_pool.cc:224] Handling call
0709 13:00:23.425480 (+2794288us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54001e4954. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:23.425690 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3527671040}
I20260709 13:00:23.479724  1689 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:23.482625  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:23.461007  1541 rpcz_store.cc:269] 0709 13:00:20.326209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.643824 (+317615us) service_pool.cc:224] Handling call
0709 13:00:22.930726 (+2286902us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54003458d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:22.930940 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3930030336}
I20260709 13:00:23.478489  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:23.488655  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:22.797196  3347 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:23.182673  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:23.294646  3563 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 2.243s	user 0.006s	sys 0.011s
I20260709 13:00:23.315464  3594 raft_consensus.cc:3055] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:23.499279  3418 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:23.499704  3489 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:22.928013  3408 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:23.302724  1684 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:22.803694  3607 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:23.217968  3631 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:22.799232  3597 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:23.516028  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:23.626353  1540 raft_consensus.cc:2368] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:22.924172  3349 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.514881  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:23.446619  3257 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:23.516886  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:23.636215  1543 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:23.503816  3594 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:22.814469  3555 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:22.922250  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 345) took 2647 ms. Trace:
I20260709 13:00:23.125247  3554 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.409552  3162 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:23.446585  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:23.167939  3581 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:23.304695  3613 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:23.460721  3260 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:23.495131  3346 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:23.224324  3230 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:23.500370  3300 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:23.621776  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:23.622848  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:23.640786  3537 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:22.800684  3571 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:23.504114  3618 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:23.634054  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 119273us
I20260709 13:00:23.653935  3547 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:23.678000  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:23.651394  3418 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:23.637264  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:23.262665  3588 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:23.641474  1689 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 13:00:23.645555  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 155644us
W20260709 13:00:23.673548  3580 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.034s	sys 0.028s
I20260709 13:00:23.525566  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 353) took 3204 ms. Trace:
I20260709 13:00:23.485575  3620 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.618024  3489 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:23.661705  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 134927us
I20260709 13:00:23.698204  1583 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:23.661784  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:23.631652  3347 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:23.636621  3516 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:23.414048  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:23.674412  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 51678.5us
I20260709 13:00:23.689978  3189 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.632108  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:23.679783  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:23.681629  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:23.690815  3300 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:23.691162  3618 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:23.713474  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1700) took 2913 ms. Trace:
I20260709 13:00:23.732900  1615 rpcz_store.cc:276] 0709 13:00:20.799778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.177668 (+2377890us) service_pool.cc:224] Handling call
0709 13:00:23.284249 (+106581us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:23.285095 (+   846us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:23.285158 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:23.286068 (+   910us) log.cc:844] Serialized 26 byte log entry
0709 13:00:23.636756 (+350688us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:23.636797 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:23.636883 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:23.713214 (+ 76331us) raft_consensus.cc:1685] finished
0709 13:00:23.713235 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:23.713384 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":124,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":129,"thread_start_us":349969,"threads_started":1,"wal-append.queue_time_us":399955}
I20260709 13:00:23.740617  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:23.632535  3631 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:23.693311  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:23.671988  3550 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:23.678963  3581 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:23.743817  3581 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:23.744024  3468 consensus_meta.cc:287] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.562s	user 0.008s	sys 0.000s
I20260709 13:00:23.744300  3581 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:23.744572  3468 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:23.698823  2237 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:23.744117  3630 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.044s	user 0.001s	sys 0.000s
I20260709 13:00:23.748718  3519 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.676388  3230 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:23.691465  3418 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.672389  2231 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:23.667847  3607 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:00:23.662415  3554 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:23.681248  3613 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:23.681056  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:23.647119  3408 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:23.691334  3537 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.706717  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:23.709611  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1703) took 2902 ms. Trace:
W20260709 13:00:23.710027  3637 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:00:23.710594  3626 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:23.829586  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:23.710999  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1696) took 2912 ms. Trace:
W20260709 13:00:23.763726  3564 consensus_meta.cc:287] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.595s	user 0.003s	sys 0.006s
I20260709 13:00:23.702740  3489 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.695606  3594 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.712337  3347 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:23.734297  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 52732.4us
I20260709 13:00:23.693461  3260 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:23.736953  3571 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:23.466667  1536 rpcz_store.cc:276] 0709 13:00:19.839384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.839550 (+   166us) service_pool.cc:224] Handling call
0709 13:00:19.839877 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:19.888563 (+ 48686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:19.891055 (+  2492us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:19.977661 (+ 86606us) log.cc:844] Serialized 26 byte log entry
0709 13:00:20.051398 (+ 73737us) spinlock_profiling.cc:243] Waited 7.69 ms on lock 0x7b480030c858. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:00:20.147247 (+ 95849us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:20.154215 (+  6968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.158329 (+  4114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:20.631344 (+473015us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b2400175b68. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c8859c4 00007fe08c883acb 00007fe08ca3edc9 00007fe08ca3ea55 00007fe08ca3e09d 00007fe08f67010e 00007fe08f68788c 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221
0709 13:00:22.970130 (+2338786us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b2400175ad8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c8859c4 00007fe08c883acb 00007fe08ca3edc9 00007fe08ca3ea55 00007fe08ca3dfb0 00007fe08f6b4488 00007fe08f6b4be8 00007fe08f68c398 00007fe08f68776a 00007fe08f686bb6
0709 13:00:23.159105 (+188975us) raft_consensus.cc:1685] finished
0709 13:00:23.159151 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:23.159367 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37485,"raft.queue_time_us":69607,"raft.run_cpu_time_us":3321,"raft.run_wall_time_us":141686,"spinlock_wait_cycles":107338752,"thread_start_us":115388,"threads_started":1,"wal-append.queue_time_us":175996}
I20260709 13:00:23.738719  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:23.712393  3598 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:23.759831  3381 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:23.671416  3400 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.934s	user 0.017s	sys 0.035s
W20260709 13:00:23.217839  1609 rpcz_store.cc:269] 0709 13:00:20.460129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.462192 (+  2063us) service_pool.cc:224] Handling call
0709 13:00:22.696698 (+2234506us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b5400087ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:22.696835 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43634048}
I20260709 13:00:23.755394  1691 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:00:23.831432  3635 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:23.882270  1691 pending_rounds.cc:85] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 4
I20260709 13:00:23.882602  1691 pending_rounds.cc:107] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:00:23.882871  1691 raft_consensus.cc:2889] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:23.785880  2237 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
W20260709 13:00:23.821792  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 115170us
I20260709 13:00:23.765101  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:23.830469  1692 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:23.834368  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 545) took 1354 ms. Trace:
W20260709 13:00:23.754403  3364 consensus_meta.cc:287] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.359s	user 0.009s	sys 0.004s
I20260709 13:00:23.754267  3568 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:23.732736  3619 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:23.731982  1542 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:23.749375  3519 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.831295  3564 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.750521  3555 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:23.751351  3349 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:23.752032  1683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 1973) took 1648 ms (client timeout 2103 ms). Trace:
I20260709 13:00:23.851059  1537 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:23.830003  3547 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:23.752897  3475 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:23.782131  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1701) took 2982 ms. Trace:
I20260709 13:00:23.971616  1618 rpcz_store.cc:276] 0709 13:00:20.800006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.288848 (+2488842us) service_pool.cc:224] Handling call
0709 13:00:23.289318 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:23.494557 (+205239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:23.494591 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:23.499543 (+  4952us) log.cc:844] Serialized 26 byte log entry
0709 13:00:23.672032 (+172489us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:23.672072 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:23.672139 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:23.781831 (+109692us) raft_consensus.cc:1685] finished
0709 13:00:23.781860 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:23.782067 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":211811,"raft.queue_time_us":224329,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":111366,"thread_start_us":172037,"threads_started":1,"wal-append.queue_time_us":235928}
I20260709 13:00:23.693656  1535 rpcz_store.cc:276] 0709 13:00:20.320841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.643804 (+2322963us) service_pool.cc:224] Handling call
0709 13:00:22.673523 (+ 29719us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:23.482483 (+808960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:23.521845 (+ 39362us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:23.892380  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:23.887106  1541 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:23.700300  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:23.897936  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:23.898897  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:23.977054  2237 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:23.899103  3468 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
W20260709 13:00:23.899269  2203 connection.cc:256] RPC call timeout handler was delayed by -2.1371077779999723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1577s for a response
W20260709 13:00:23.881948  3400 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.145s	user 0.017s	sys 0.036s
W20260709 13:00:23.905151  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:23.899765  2201 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:23.889767  2232 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:23.854019  1543 raft_consensus.cc:2368] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:23.831940  3408 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.884647  3635 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:23.885110  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:23.921006  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 549) took 1440 ms. Trace:
I20260709 13:00:23.884307  3346 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:23.763131  3308 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.785450  3189 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.765291  3300 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.899089  3260 raft_consensus.cc:697] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:23.910934  3568 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:23.918990  3619 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:23.969825  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 116859us
W20260709 13:00:23.974651  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 82412.6us
I20260709 13:00:23.975098  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:23.981107  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 81434.1us
W20260709 13:00:23.978412  2203 connection.cc:256] RPC call timeout handler was delayed by -1.6967315470000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
I20260709 13:00:23.829452  1611 rpcz_store.cc:276] 0709 13:00:20.800514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.517825 (+2717311us) service_pool.cc:224] Handling call
0709 13:00:23.663831 (+146006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:23.674097 (+ 10266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:23.675709 (+  1612us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:23.678763 (+  3054us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:23.687200 (+  8437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:23.693951 (+  6751us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:23.700883 (+  6932us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:23.983171  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:23.890810  3588 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:23.897944  3598 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 13:00:23.909920  3364 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.920416  3613 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:23.983502  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:24.689800  1542 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:24.470904  3537 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:23.998346  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:24.691622  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1699) took 3892 ms. Trace:
I20260709 13:00:24.064445  3555 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:24.123307  3519 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:24.692262  1613 rpcz_store.cc:276] 0709 13:00:20.799544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.667192 (+1867648us) service_pool.cc:224] Handling call
0709 13:00:22.691852 (+ 24660us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:22.695351 (+  3499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:22.695393 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:23.739270 (+1043877us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b2c0005b3b8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:00:23.739749 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 13:00:23.974248 (+234499us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:23.974288 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:23.974356 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:24.691282 (+716926us) raft_consensus.cc:1685] finished
0709 13:00:24.691300 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:24.691554 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":38151,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":8085,"spinlock_wait_cycles":5052288,"thread_start_us":234100,"threads_started":1,"wal-append.queue_time_us":949792}
I20260709 13:00:24.694502  3189 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:24.385025  1583 leader_election.cc:336] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.681s (SENT)
I20260709 13:00:24.694733  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:24.695526  1583 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:24.696280  3651 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:20.797465   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:23.981348  3349 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.708704  3568 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.226956  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:24.710098  1536 raft_consensus.cc:2368] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:24.697129  3400 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:23.992041  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:24.710371  3346 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.647224  1534 rpcz_store.cc:276] 0709 13:00:20.275432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.761390 (+485958us) service_pool.cc:224] Handling call
0709 13:00:20.871206 (+109816us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:22.493461 (+1622255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:22.499613 (+  6152us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:22.626496 (+126883us) log.cc:844] Serialized 26 byte log entry
0709 13:00:22.708860 (+ 82364us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:22.817615 (+108755us) log.cc:844] Serialized 14 byte log entry
0709 13:00:22.921873 (+104258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.921962 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:22.922011 (+    49us) raft_consensus.cc:1685] finished
0709 13:00:22.922027 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:22.922180 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2443,"prepare.queue_time_us":898928,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":3675,"raft.queue_time_us":1038550,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":10102,"thread_start_us":1489735,"threads_started":3,"wal-append.queue_time_us":117291}
I20260709 13:00:24.712302  3619 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.728458  3260 consensus_queue.cc:237] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:24.712617  3308 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.712815  1618 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
W20260709 13:00:24.696660  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 311675us
I20260709 13:00:23.999050  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:23.893644  2231 rpcz_store.cc:276] 0709 13:00:22.478905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.488472 (+  9567us) service_pool.cc:224] Handling call
0709 13:00:23.446334 (+957862us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:23.832789 (+386455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24241536}
W20260709 13:00:24.713549  2203 connection.cc:256] RPC call timeout handler was delayed by -1.691607777999991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
W20260709 13:00:24.244164  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:24.734674  2203 connection.cc:256] RPC call timeout handler was delayed by -1.5431077779999782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2237s for a response
W20260709 13:00:23.977505  3561 consensus_meta.cc:287] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.797s	user 0.011s	sys 0.001s
I20260709 13:00:23.976277  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:23.984643  2233 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.507s	user 0.005s	sys 0.004s
W20260709 13:00:23.989933  1616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1707) took 3188 ms (client timeout 3163 ms). Trace:
I20260709 13:00:23.992439  3600 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:23.987076  3620 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:23.990569  3418 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:24.589134  3581 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.844s	user 0.005s	sys 0.006s
I20260709 13:00:24.716374  1541 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:24.758524  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2166) took 1241 ms. Trace:
I20260709 13:00:24.758865  1541 rpcz_store.cc:276] 0709 13:00:23.517598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.974849 (+457251us) service_pool.cc:224] Handling call
0709 13:00:24.758453 (+783604us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:24.602792  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:24.692327  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 457) took 1046 ms. Trace:
W20260709 13:00:24.684930  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:23.995079  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:24.763509  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1716) took 1255 ms. Trace:
I20260709 13:00:24.763424  2230 rpcz_store.cc:276] 0709 13:00:23.645882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.647288 (+  1406us) service_pool.cc:224] Handling call
0709 13:00:23.998188 (+350900us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:24.692265 (+694077us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23591680}
I20260709 13:00:24.763947  1617 rpcz_store.cc:276] 0709 13:00:23.507712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.994771 (+487059us) service_pool.cc:224] Handling call
0709 13:00:24.763448 (+768677us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.764868  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:24.695726  1542 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:24.765980  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1687) took 1303 ms. Trace:
I20260709 13:00:24.766391  1542 rpcz_store.cc:276] 0709 13:00:23.462151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.690956 (+1228805us) service_pool.cc:224] Handling call
0709 13:00:24.765910 (+ 74954us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:24.134809  3644 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 13:00:24.713912  3189 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:24.767735  3189 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.768795  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1718) took 1260 ms. Trace:
I20260709 13:00:24.692689  3555 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.695875  3650 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:24.732666  2237 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:24.796000  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 460) took 1147 ms. Trace:
I20260709 13:00:24.795859  3650 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.796391  2237 rpcz_store.cc:276] 0709 13:00:23.648584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.998741 (+350157us) service_pool.cc:224] Handling call
0709 13:00:24.795938 (+797197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.797384  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:23.992967  1684 raft_consensus.cc:2410] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:24.798094  3650 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:24.798343  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 361) took 1151 ms. Trace:
I20260709 13:00:24.798774  1684 rpcz_store.cc:276] 0709 13:00:23.647003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.683526 (+ 36523us) service_pool.cc:224] Handling call
0709 13:00:24.798278 (+1114752us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.738863  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:24.800081  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:24.717545  3325 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.021s	sys 0.021s
W20260709 13:00:24.801620  3325 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.022s	sys 0.021s
W20260709 13:00:24.735119  2203 connection.cc:256] RPC call timeout handler was delayed by -1.4144077779999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.238s for a response
I20260709 13:00:24.803771  3189 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:24.804003  3347 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:24.731840  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:24.730651  3519 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:24.735697  2233 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
W20260709 13:00:24.814273  2233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 453) took 2185 ms (client timeout 2176 ms). Trace:
I20260709 13:00:23.977919  3547 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.741544  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:24.744596  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:24.817044  1689 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:24.695844  3364 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:24.749753  3561 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.127777  3544 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:23.983613  3571 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:24.714068  2234 rpcz_store.cc:276] 0709 13:00:22.480311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.480745 (+   434us) service_pool.cc:224] Handling call
0709 13:00:22.496882 (+ 16137us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:22.497371 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:22.497412 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:22.526014 (+ 28602us) log.cc:844] Serialized 26 byte log entry
0709 13:00:22.793735 (+267721us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:22.794486 (+   751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:22.794585 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:23.920780 (+1126195us) raft_consensus.cc:1685] finished
0709 13:00:23.920803 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:23.920944 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":305,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":17401,"thread_start_us":163064,"threads_started":1,"wal-append.queue_time_us":900052}
I20260709 13:00:24.726220  3537 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.725009  3489 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:24.805066  3346 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.764652  3581 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:23.831039  1612 rpcz_store.cc:276] 0709 13:00:20.798853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.593390 (+1794537us) service_pool.cc:224] Handling call
0709 13:00:22.638725 (+ 45335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:22.792130 (+153405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:22.795288 (+  3158us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:23.446817 (+651529us) log.cc:844] Serialized 26 byte log entry
0709 13:00:23.646945 (+200128us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:23.657858 (+ 10913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:23.666968 (+  9110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:23.710739 (+ 43771us) raft_consensus.cc:1685] finished
0709 13:00:23.710759 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:23.710932 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1150,"raft.queue_time_us":28443,"raft.run_cpu_time_us":514,"raft.run_wall_time_us":20871,"thread_start_us":293916,"threads_started":2,"wal-append.queue_time_us":261812}
W20260709 13:00:23.969219  1683 rpcz_store.cc:269] 0709 13:00:22.103300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.104276 (+   976us) service_pool.cc:224] Handling call
0709 13:00:23.751647 (+1647371us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b54000744d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:23.751934 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1068857216}
I20260709 13:00:24.720556  3635 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.772090  1584 leader_election.cc:400] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:24.769215  1617 rpcz_store.cc:276] 0709 13:00:23.508390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.764616 (+1256226us) service_pool.cc:224] Handling call
0709 13:00:24.768742 (+  4126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.770381  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:24.848387  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 76432.1us
I20260709 13:00:24.848472  1540 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:24.848577  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:24.849058  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 359) took 1156 ms. Trace:
I20260709 13:00:24.849368  1584 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:24.730548  3349 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.793855  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1728) took 1121 ms. Trace:
I20260709 13:00:24.795305  3555 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:24.798879  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:24.855656  1541 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:00:24.735826  1616 rpcz_store.cc:269] 0709 13:00:20.802021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.951740 (+149719us) service_pool.cc:224] Handling call
0709 13:00:23.989740 (+3038000us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b5400398ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:23.989859 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":876996992}
I20260709 13:00:24.805190  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1724) took 1181 ms. Trace:
W20260709 13:00:24.815021  2233 rpcz_store.cc:269] 0709 13:00:22.628478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.761737 (+133259us) service_pool.cc:224] Handling call
0709 13:00:24.814197 (+2052460us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.712100  3588 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.734869  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 455) took 1088 ms. Trace:
I20260709 13:00:24.756000  3600 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:24.735909  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2164) took 1272 ms. Trace:
I20260709 13:00:23.966929  3618 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.757572  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 350) took 4364 ms. Trace:
I20260709 13:00:24.769556  3661 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.770951  3620 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.803812  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:24.843456  3544 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.884974  2230 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
W20260709 13:00:24.885165  1688 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 1980) took 2334 ms (client timeout 1907 ms). Trace:
I20260709 13:00:24.885601  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 559) took 1198 ms. Trace:
I20260709 13:00:24.851851  1535 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:24.696861  3651 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.886368  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:24.853750  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:24.849393  1540 rpcz_store.cc:276] 0709 13:00:23.692204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.730007 (+1037803us) service_pool.cc:224] Handling call
0709 13:00:24.849006 (+118999us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.846802  3571 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:24.830971  3564 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:24.738395  3598 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.856325  1618 rpcz_store.cc:276] 0709 13:00:23.624158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.731573 (+1107415us) service_pool.cc:224] Handling call
0709 13:00:24.805130 (+ 73557us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.892633  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:24.831219  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:24.893741  1611 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:24.894260  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1723) took 1274 ms. Trace:
I20260709 13:00:24.857857  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2169) took 1163 ms. Trace:
I20260709 13:00:24.894869  3349 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:24.895129  1541 rpcz_store.cc:276] 0709 13:00:23.694754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.798645 (+1103891us) service_pool.cc:224] Handling call
0709 13:00:24.857802 (+ 59157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.894361  2201 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:24.896420  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:24.858376  3541 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:24.873633  1615 raft_consensus.cc:2368] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5.
I20260709 13:00:24.814179  3622 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:00:24.882318  1686 rpcz_store.cc:276] 0709 13:00:20.388634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.389211 (+   577us) service_pool.cc:224] Handling call
0709 13:00:20.389747 (+   536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:22.075630 (+1685883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:24.752376 (+2676746us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b5400374454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:24.752595 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3751929088}
I20260709 13:00:24.869890  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:24.883831  3661 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.885325  3620 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:24.887040  1616 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:24.851020  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1730) took 1166 ms. Trace:
I20260709 13:00:24.887514  2234 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 13:00:24.908288  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:24.916260  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 462) took 1267 ms. Trace:
I20260709 13:00:24.916647  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1200) took 1030 ms. Trace:
I20260709 13:00:24.882472  3600 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:24.896523  3564 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.897145  3308 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:24.857846  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:24.883942  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:24.838054  3519 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:24.872066  3565 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 4.102s	user 0.007s	sys 0.020s
I20260709 13:00:24.842423  3619 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.916568  1540 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:24.932822  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2173) took 1226 ms. Trace:
I20260709 13:00:24.933172  1540 rpcz_store.cc:276] 0709 13:00:23.706302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.887659 (+1181357us) service_pool.cc:224] Handling call
0709 13:00:24.932765 (+ 45106us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:24.922400  3662 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:00:24.916939  2234 rpcz_store.cc:276] 0709 13:00:23.649306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.846856 (+1197550us) service_pool.cc:224] Handling call
0709 13:00:24.916177 (+ 69321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.917191  1616 rpcz_store.cc:276] 0709 13:00:23.886303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.868538 (+982235us) service_pool.cc:224] Handling call
0709 13:00:24.916590 (+ 48052us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.851011  1609 rpcz_store.cc:276] 0709 13:00:23.672888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.915060 (+242172us) service_pool.cc:224] Handling call
0709 13:00:24.793768 (+878708us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:24.885669  1688 rpcz_store.cc:269] 0709 13:00:22.550560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.552580 (+  2020us) service_pool.cc:224] Handling call
0709 13:00:24.884968 (+2332388us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54002ce054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:24.885089 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376006400}
W20260709 13:00:24.918147  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 67655.1us
I20260709 13:00:24.915933  1617 rpcz_store.cc:276] 0709 13:00:23.687936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.848343 (+1160407us) service_pool.cc:224] Handling call
0709 13:00:24.850922 (+  2579us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.938369  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:24.939858  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:24.886050  2230 rpcz_store.cc:276] 0709 13:00:23.687605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.803524 (+1115919us) service_pool.cc:224] Handling call
0709 13:00:24.885539 (+ 82015us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.894644  1611 rpcz_store.cc:276] 0709 13:00:23.619832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.732401 (+1112569us) service_pool.cc:224] Handling call
0709 13:00:24.894203 (+161802us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.908509  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1725) took 1257 ms. Trace:
I20260709 13:00:24.846032  3537 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.873191  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:24.898713  2233 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:24.898705  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2163) took 1439 ms. Trace:
W20260709 13:00:24.925586  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 67788.9us
I20260709 13:00:24.881909  1536 rpcz_store.cc:276] 0709 13:00:23.460869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.930210 (+469341us) service_pool.cc:224] Handling call
0709 13:00:24.733182 (+802972us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.886937  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2162) took 1428 ms. Trace:
I20260709 13:00:24.897948  3622 consensus_queue.cc:1243] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:00:24.855387  3346 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:24.894655  3547 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:24.936832  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:24.906638  3408 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:24.944003  3619 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:24.944748  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:24.940861  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:24.868942  2232 rpcz_store.cc:276] 0709 13:00:23.644498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.647203 (+  2705us) service_pool.cc:224] Handling call
0709 13:00:24.733076 (+1085873us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.946084  1618 rpcz_store.cc:276] 0709 13:00:23.636372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.892382 (+1256010us) service_pool.cc:224] Handling call
0709 13:00:24.893289 (+   907us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.922003  3400 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:24.908550  3598 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:25.137841  1541 rpcz_store.cc:276] 0709 13:00:23.459582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.896133 (+1436551us) service_pool.cc:224] Handling call
0709 13:00:24.898642 (+  2509us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.989494  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.140727  1540 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:25.142355  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 466) took 1252 ms. Trace:
W20260709 13:00:25.138404  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 197581us
I20260709 13:00:24.946847  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1727) took 1282 ms. Trace:
I20260709 13:00:24.956007  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:25.142884  1616 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:25.095649  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:00:25.093972  2236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 555) took 2523 ms (client timeout 3241 ms). Trace:
I20260709 13:00:25.142547  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2167) took 1622 ms. Trace:
I20260709 13:00:25.144229  2233 rpcz_store.cc:276] 0709 13:00:23.890187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.869520 (+979333us) service_pool.cc:224] Handling call
0709 13:00:25.142294 (+272774us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.148280  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:25.144620  1617 rpcz_store.cc:276] 0709 13:00:23.664771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.939570 (+1274799us) service_pool.cc:224] Handling call
0709 13:00:24.946775 (+  7205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.150841  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.151639  1617 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:25.152314  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1202) took 1257 ms. Trace:
I20260709 13:00:25.152730  1617 rpcz_store.cc:276] 0709 13:00:23.895164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.150496 (+1255332us) service_pool.cc:224] Handling call
0709 13:00:25.152234 (+  1738us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.145571  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1729) took 1468 ms. Trace:
I20260709 13:00:25.154008  1616 rpcz_store.cc:276] 0709 13:00:23.676813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.936801 (+1259988us) service_pool.cc:224] Handling call
0709 13:00:25.145479 (+208678us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.155578  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:25.156913  1541 raft_consensus.cc:2410] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:25.157593  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2170) took 1456 ms. Trace:
I20260709 13:00:25.158051  1541 rpcz_store.cc:276] 0709 13:00:23.701414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.139092 (+1437678us) service_pool.cc:224] Handling call
0709 13:00:25.157499 (+ 18407us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.159438  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:25.160677  1541 raft_consensus.cc:2410] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:00:25.146490  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50908.9us
I20260709 13:00:25.162704  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:24.926839  3308 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.164919  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.166546  1618 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:24.949920  1584 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:25.146899  1540 rpcz_store.cc:276] 0709 13:00:23.518660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.944486 (+1425826us) service_pool.cc:224] Handling call
0709 13:00:25.141305 (+196819us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.165673  2203 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:25.211702  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.212396  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1651) took 5328 ms. Trace:
W20260709 13:00:25.146759  2236 rpcz_store.cc:269] 0709 13:00:22.570412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.598037 (+ 27625us) service_pool.cc:224] Handling call
0709 13:00:25.093704 (+2495667us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54003bced4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:25.093866 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093552000}
I20260709 13:00:25.213150  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1201) took 1326 ms. Trace:
I20260709 13:00:25.213656  1616 rpcz_store.cc:276] 0709 13:00:23.886599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.154890 (+1268291us) service_pool.cc:224] Handling call
0709 13:00:25.213089 (+ 58199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.185952  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:24.955076  3573 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:25.036849  3536 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:25.028600  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:25.093835  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:24.991451  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 461) took 1342 ms. Trace:
I20260709 13:00:25.092326  3661 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:25.111043  3541 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:25.161226  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2157) took 1984 ms. Trace:
W20260709 13:00:25.231343  1583 leader_election.cc:336] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.106s (SENT)
I20260709 13:00:25.246773  1541 rpcz_store.cc:276] 0709 13:00:23.176692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.159029 (+1982337us) service_pool.cc:224] Handling call
0709 13:00:25.161149 (+  2120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:24.945235  1612 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:24.989784  3657 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.004436  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:25.025840  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:25.256338  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:25.213078  1539 rpcz_store.cc:276] 0709 13:00:19.838123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.838259 (+   136us) service_pool.cc:224] Handling call
0709 13:00:19.838573 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:20.167981 (+329408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:20.168013 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:20.190569 (+ 22556us) log.cc:844] Serialized 26 byte log entry
0709 13:00:20.386802 (+196233us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:20.386855 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:20.386950 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:24.733982 (+4347032us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:00:25.113682 (+379700us) raft_consensus.cc:1685] finished
0709 13:00:25.113702 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.166085 (+ 52383us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9646,"raft.queue_time_us":52320,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":23529,"spinlock_wait_cycles":55772928,"thread_start_us":478286,"threads_started":2,"wal-append.queue_time_us":409714}
I20260709 13:00:25.257192  1540 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:25.241832  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.257745  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2172) took 1555 ms. Trace:
I20260709 13:00:25.258004  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1198) took 1372 ms. Trace:
I20260709 13:00:25.258088  1540 rpcz_store.cc:276] 0709 13:00:23.702134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.246810 (+1544676us) service_pool.cc:224] Handling call
0709 13:00:25.257687 (+ 10877us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.258519  1616 rpcz_store.cc:276] 0709 13:00:23.885643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.214379 (+1328736us) service_pool.cc:224] Handling call
0709 13:00:25.257949 (+ 43570us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.163753  3555 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:24.990232  3668 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:00:25.244796  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 294937us
I20260709 13:00:25.149068  2233 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
W20260709 13:00:25.243551  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 71119.1us
I20260709 13:00:25.246023  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:24.931856  3564 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:25.168251  3408 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:25.168726  3349 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:25.246995  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:25.296998  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 560) took 1603 ms. Trace:
I20260709 13:00:25.255183  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:25.168628  3308 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:25.253847  1612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1722) took 1737 ms (client timeout 2220 ms). Trace:
I20260709 13:00:25.245620  3189 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:25.216939  3615 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.167030  3300 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:25.044415  3568 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:25.214464  3651 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:25.241577  3544 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:25.216535  1534 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:25.248356  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:25.249234  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 83626.8us
I20260709 13:00:25.092531  3571 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:25.266352  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:25.246445  2231 rpcz_store.cc:276] 0709 13:00:23.648977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.815408 (+1166431us) service_pool.cc:224] Handling call
0709 13:00:24.991371 (+175963us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.164499  3400 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:25.250710  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:25.293565  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:25.247869  2230 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:25.250486  1609 raft_consensus.cc:2376] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4. Re-sending same reply.
W20260709 13:00:25.346863  1537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1684) took 2652 ms (client timeout 1723 ms). Trace:
I20260709 13:00:25.361475  1541 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
W20260709 13:00:25.361626  1537 rpcz_store.cc:269] 0709 13:00:22.694738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.780924 (+ 86186us) service_pool.cc:224] Handling call
0709 13:00:25.346552 (+2565628us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b540012e3d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:25.346756 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":889039232}
I20260709 13:00:25.362931  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:25.363196  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:25.363814  1540 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:25.364339  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2171) took 1662 ms. Trace:
I20260709 13:00:25.251331  3670 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:25.316991  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:25.389324  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 123054us
I20260709 13:00:25.391170  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:25.323824  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.137490  1535 rpcz_store.cc:276] 0709 13:00:23.458840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.991709 (+532869us) service_pool.cc:224] Handling call
0709 13:00:24.886758 (+895049us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54000e97d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:24.886878 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312158464}
I20260709 13:00:25.363693  1541 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:25.297585  2236 rpcz_store.cc:276] 0709 13:00:23.693340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.228316 (+1534976us) service_pool.cc:224] Handling call
0709 13:00:25.296914 (+ 68598us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.364773  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:00:25.303344  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 72053.1us
I20260709 13:00:25.216856  1617 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:25.261008  3555 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:00:25.394472  1658 leader_election.cc:336] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.580s (SENT)
W20260709 13:00:25.302042  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.333040  1611 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:25.302433  3349 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.325022  3561 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:25.505901  1537 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:25.506875  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2147) took 5063 ms. Trace:
I20260709 13:00:25.486963  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1206) took 1516 ms. Trace:
I20260709 13:00:25.544510  1537 rpcz_store.cc:276] 0709 13:00:20.443142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.405903 (+4962761us) service_pool.cc:224] Handling call
0709 13:00:25.505523 (+ 99620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.506539 (+  1016us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.506803 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.544777  1617 rpcz_store.cc:276] 0709 13:00:23.970761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.185599 (+1214838us) service_pool.cc:224] Handling call
0709 13:00:25.486883 (+301284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.536230  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:25.539698  1535 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:24.716630  3516 consensus_queue.cc:1175] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "984c9089d9804652846f3caeb609d56e" responder_term: 4 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: [3.5-3.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:25.561126  2231 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:25.581923  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2150) took 5137 ms. Trace:
I20260709 13:00:25.582518  1535 rpcz_store.cc:276] 0709 13:00:20.444034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.505104 (+5061070us) service_pool.cc:224] Handling call
0709 13:00:25.537692 (+ 32588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.569903 (+ 32211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.581829 (+ 11926us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.584729  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2153) took 5118 ms. Trace:
I20260709 13:00:25.487794  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:25.493029  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 128333us
I20260709 13:00:25.486379  3555 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:25.335479  3189 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:25.301585  3408 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:25.510766  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:25.335839  3622 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:25.343660  1610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 308) took 4844 ms (client timeout 2237 ms). Trace:
I20260709 13:00:25.347066  3519 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:25.512740  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1689) took 2041 ms. Trace:
I20260709 13:00:25.260722  3657 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:25.292834  3668 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:25.488855  3300 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:25.672631  3300 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:25.663442  1536 rpcz_store.cc:276] 0709 13:00:23.471393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.227325 (+1755932us) service_pool.cc:224] Handling call
0709 13:00:25.512651 (+285326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.489339  3655 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:25.335151  1612 rpcz_store.cc:269] 0709 13:00:23.516394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.870132 (+1353738us) service_pool.cc:224] Handling call
0709 13:00:25.252304 (+382172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.676790  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:25.677270  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.677969  1612 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:25.678458  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1211) took 1253 ms. Trace:
I20260709 13:00:25.504942  3546 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:25.678747  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1665) took 5178 ms. Trace:
I20260709 13:00:25.679102  3546 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:25.679628  3546 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:25.679131  1536 rpcz_store.cc:276] 0709 13:00:20.499966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.677955 (+5177989us) service_pool.cc:224] Handling call
0709 13:00:25.678256 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.678442 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.678472 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.678520 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:25.678551 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.678607 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.678698 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.683255  3657 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:25.685884  3564 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:25.687228  3635 consensus_meta.cc:287] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.818s	user 0.011s	sys 0.000s
I20260709 13:00:25.678771  1612 rpcz_store.cc:276] 0709 13:00:24.433150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.677012 (+1243862us) service_pool.cc:224] Handling call
0709 13:00:25.678400 (+  1388us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.688900  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:25.689757  1582 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45356 dropped due to backpressure: service queue is full with 50 items
I20260709 13:00:25.689890  1612 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:25.690659  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1218) took 1009 ms. Trace:
I20260709 13:00:25.691048  1612 rpcz_store.cc:276] 0709 13:00:24.681464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.688632 (+1007168us) service_pool.cc:224] Handling call
0709 13:00:25.690598 (+  1966us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.504830  3349 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:25.486809  1609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1726) took 1841 ms (client timeout 2190 ms). Trace:
I20260709 13:00:25.476854  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 556) took 1803 ms. Trace:
I20260709 13:00:25.405485  3615 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:00:25.482180  1534 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1692) took 1961 ms (client timeout 2205 ms). Trace:
I20260709 13:00:25.522058  3620 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:25.530049  1684 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:25.526713  1611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1203) took 1616 ms (client timeout 1969 ms). Trace:
I20260709 13:00:25.546875  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:25.487622  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 93173.5us
I20260709 13:00:25.361534  1539 raft_consensus.cc:2368] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:25.744345  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1214) took 1083 ms. Trace:
I20260709 13:00:25.744976  1617 rpcz_store.cc:276] 0709 13:00:24.667758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.546613 (+878855us) service_pool.cc:224] Handling call
0709 13:00:25.744253 (+197640us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.364640  1540 rpcz_store.cc:276] 0709 13:00:23.701803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.259272 (+1557469us) service_pool.cc:224] Handling call
0709 13:00:25.364285 (+105013us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.486654  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 358) took 1795 ms. Trace:
I20260709 13:00:25.746647  1541 rpcz_store.cc:276] 0709 13:00:23.691144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.362584 (+1671440us) service_pool.cc:224] Handling call
0709 13:00:25.486586 (+124002us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.747859  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:25.652112  2236 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:25.541707  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:25.650518  3189 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:25.712476  1609 rpcz_store.cc:269] 0709 13:00:23.645983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.980976 (+1334993us) service_pool.cc:224] Handling call
0709 13:00:25.405377 (+424401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.477401  3675 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:25.389196  3619 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:00:25.654001  3600 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.720s	user 0.004s	sys 0.008s
I20260709 13:00:25.687680  3635 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:25.699616  1543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2161) took 2241 ms (client timeout 1773 ms). Trace:
W20260709 13:00:25.663774  1610 rpcz_store.cc:269] 0709 13:00:20.499654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.508034 (+  8380us) service_pool.cc:224] Handling call
0709 13:00:25.343341 (+4835307us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b54000bf4d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:25.343522 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1172263680}
I20260709 13:00:25.651324  3598 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:25.719166  1584 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:25.585196  1535 rpcz_store.cc:276] 0709 13:00:20.466557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.583593 (+5117036us) service_pool.cc:224] Handling call
0709 13:00:25.584030 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.584277 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.584323 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.584383 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:25.584452 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.584511 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.584655 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:25.768788  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 49683.1us
I20260709 13:00:25.769668  3622 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:25.770228  3622 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:25.770694  3622 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:25.767241  1543 rpcz_store.cc:269] 0709 13:00:23.458063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.636989 (+178926us) service_pool.cc:224] Handling call
0709 13:00:25.699372 (+2062383us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54001e4454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:25.699531 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393877248}
I20260709 13:00:25.775461  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:25.776356  1543 raft_consensus.cc:2410] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:25.778656  1584 leader_election.cc:400] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:25.779038  1584 leader_election.cc:403] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:25.779604  3674 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:25.687642  3555 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:25.784256  1541 raft_consensus.cc:2393] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 2: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:25.785495  3555 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:25.777108  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:25.789942  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:25.790019  1543 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
W20260709 13:00:25.759826  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 160840us
I20260709 13:00:25.792299  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:25.717501  2230 rpcz_store.cc:276] 0709 13:00:23.673891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.028284 (+1354393us) service_pool.cc:224] Handling call
0709 13:00:25.405178 (+376894us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.790730  1539 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:25.719079  3546 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:25.795614  1543 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:25.799033  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1669) took 5261 ms. Trace:
I20260709 13:00:25.799209  2203 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:25.799394  1543 rpcz_store.cc:276] 0709 13:00:20.537658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.795116 (+5257458us) service_pool.cc:224] Handling call
0709 13:00:25.795428 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.798722 (+  3294us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.798971 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.750568  3408 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:25.730366  1534 rpcz_store.cc:269] 0709 13:00:23.520806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.815035 (+1294229us) service_pool.cc:224] Handling call
0709 13:00:25.406118 (+591083us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.803586  1543 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:25.690080  1582 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45356 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1215)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1220)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1219)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1226)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1731)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1739)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1204)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 320)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1735)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 321)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1227)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 319)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1224)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 323)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 324)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 314)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1225)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 317)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1213)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1222)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1209)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 326)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 322)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1212)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1732)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1221)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1736)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1223)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1714)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1717)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1216)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 318)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1704)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1705)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1706)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1708)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 312)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1709)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1711)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1715)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1208)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1733)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1734)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1737)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1738)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 315)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 316)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 325)
W20260709 13:00:25.741971  1611 rpcz_store.cc:269] 0709 13:00:23.914207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.989182 (+1074975us) service_pool.cc:224] Handling call
0709 13:00:25.522416 (+533234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.807050  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:25.825038  2236 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:25.826119  1539 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:25.826786  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 357) took 5265 ms. Trace:
I20260709 13:00:25.827097  1539 rpcz_store.cc:276] 0709 13:00:20.561106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.797998 (+5236892us) service_pool.cc:224] Handling call
0709 13:00:25.825957 (+ 27959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.826539 (+   582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.826731 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.830241  1540 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:25.830906  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1672) took 5214 ms. Trace:
I20260709 13:00:25.831205  1540 rpcz_store.cc:276] 0709 13:00:20.616722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.802989 (+5186267us) service_pool.cc:224] Handling call
0709 13:00:25.829933 (+ 26944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.830647 (+   714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.830840 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:25.806767  3651 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:25.786614  3619 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:25.796085  3568 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:25.834431  3300 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:25.787159  3600 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:25.843391  1614 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1690) took 5194 ms (client timeout 2115 ms). Trace:
W20260709 13:00:25.845307  1614 rpcz_store.cc:269] 0709 13:00:20.648913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.649176 (+   263us) service_pool.cc:224] Handling call
0709 13:00:25.843201 (+5194025us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b54000befd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:25.843308 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":536279424}
I20260709 13:00:25.846547  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.847379  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1204) took 1882 ms. Trace:
I20260709 13:00:25.847793  1614 rpcz_store.cc:276] 0709 13:00:23.965413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.846301 (+1880888us) service_pool.cc:224] Handling call
0709 13:00:25.847307 (+  1006us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:25.849903  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:25.850833  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.879962  1616 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
W20260709 13:00:25.870532  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 128032us
I20260709 13:00:25.881392  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.863837  3349 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:25.843123  3300 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:25.853091  2233 raft_consensus.cc:2368] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5.
I20260709 13:00:25.852174  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.897815  3300 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:25.882483  1614 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:25.899173  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 356) took 2413 ms. Trace:
W20260709 13:00:25.881300  1658 leader_election.cc:336] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.190s (SENT)
I20260709 13:00:25.851954  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:25.882066  3578 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 4.783s	user 0.026s	sys 0.016s
I20260709 13:00:25.899569  1691 rpcz_store.cc:276] 0709 13:00:23.479801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.480059 (+   258us) service_pool.cc:224] Handling call
0709 13:00:23.754959 (+274900us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:23.894517 (+139558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.886860 (+1992343us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54000744d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:25.893173 (+  6313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581954944}
W20260709 13:00:25.898984  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 78142.5us
I20260709 13:00:25.906316  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 566) took 1090 ms. Trace:
I20260709 13:00:25.906401  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:25.914834  2203 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:25.906659  3349 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:25.910022  3594 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:25.923301  3308 consensus_meta.cc:287] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.580s	user 0.008s	sys 0.000s
I20260709 13:00:25.910518  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:25.910739  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.896080  1535 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:00:25.911412  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:25.914057  1542 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:25.865270  3599 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:25.915717  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.916419  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:25.852988  3551 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:25.916625  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:25.917344  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:25.920195  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:25.924091  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:25.911563  3681 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:00:25.841912  3651 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:25.996659  2235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 756) took 5480 ms (client timeout 2220 ms). Trace:
I20260709 13:00:25.990291  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 1969) took 5149 ms. Trace:
W20260709 13:00:26.004914  2235 rpcz_store.cc:269] 0709 13:00:20.515803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.515935 (+   132us) service_pool.cc:224] Handling call
0709 13:00:25.996341 (+5480406us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54003d3bd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:25.996537 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3498552192}
W20260709 13:00:25.991668  1615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1720) took 2482 ms (client timeout 1580 ms). Trace:
W20260709 13:00:26.006349  1615 rpcz_store.cc:269] 0709 13:00:23.509039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.740276 (+231237us) service_pool.cc:224] Handling call
0709 13:00:25.991415 (+2251139us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400190854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:25.991583 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2419146624}
I20260709 13:00:26.007334  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2154) took 5540 ms. Trace:
I20260709 13:00:26.006772  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1677) took 3531 ms. Trace:
I20260709 13:00:26.011066  1542 rpcz_store.cc:276] 0709 13:00:22.475424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.895490 (+3420066us) service_pool.cc:224] Handling call
0709 13:00:25.913609 (+ 18119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.006502 (+ 92893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.006717 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.005862  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.012581  2231 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:26.013285  3581 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:25.991714  3668 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:26.013185  3189 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:26.014173  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.015287  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1670) took 5438 ms. Trace:
W20260709 13:00:25.951918  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.021620  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.007807  1537 rpcz_store.cc:276] 0709 13:00:20.466805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.612191 (+5145386us) service_pool.cc:224] Handling call
0709 13:00:25.664756 (+ 52565us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:25.713650 (+ 48894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.713699 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:25.714329 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 13:00:25.843277 (+128948us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:25.843325 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.843428 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:25.994179 (+150751us) raft_consensus.cc:1685] finished
0709 13:00:25.994199 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.007273 (+ 13074us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6227,"raft.queue_time_us":6346,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":147,"thread_start_us":89755,"threads_started":1,"wal-append.queue_time_us":166641}
I20260709 13:00:25.990569  1611 raft_consensus.cc:2410] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:00:25.991140  3678 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:00:26.061071  1691 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:26.061156  2236 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:26.017560  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:25.998260  2229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 554) took 3457 ms (client timeout 1907 ms). Trace:
I20260709 13:00:26.025188  1689 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:26.005502  1687 rpcz_store.cc:276] 0709 13:00:20.840625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.841994 (+  1369us) service_pool.cc:224] Handling call
0709 13:00:20.842292 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:22.018459 (+1176167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:22.018490 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:22.021856 (+  3366us) log.cc:844] Serialized 26 byte log entry
0709 13:00:22.301889 (+280033us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:22.301948 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.989927 (+3687979us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002cd154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:25.989978 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:25.990023 (+    45us) raft_consensus.cc:1685] finished
0709 13:00:25.990033 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.990233 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60054,"raft.queue_time_us":29425,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":11320,"spinlock_wait_cycles":2649735552,"thread_start_us":224450,"threads_started":1,"wal-append.queue_time_us":328414,"wal-append.run_cpu_time_us":3604,"wal-append.run_wall_time_us":1455708}
I20260709 13:00:26.027031  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:25.989615  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.013616  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.062983  2235 raft_consensus.cc:2410] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:26.016016  3349 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:25.914846  2230 rpcz_store.cc:276] 0709 13:00:24.815769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.835408 (+1019639us) service_pool.cc:224] Handling call
0709 13:00:25.906246 (+ 70838us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.056458  3546 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:26.058405  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.079936  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.081118  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:26.062562  2229 rpcz_store.cc:269] 0709 13:00:22.541363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.554914 (+ 13551us) service_pool.cc:224] Handling call
0709 13:00:25.998032 (+3443118us) spinlock_profiling.cc:243] Waited 34.4 ms on lock 0x7b5400741154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:25.998165 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79145728}
I20260709 13:00:26.082422  1611 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:26.082996  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.083827  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 765) took 1279 ms. Trace:
I20260709 13:00:26.083889  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.084295  2229 rpcz_store.cc:276] 0709 13:00:24.804486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.082709 (+1278223us) service_pool.cc:224] Handling call
0709 13:00:26.083767 (+  1058us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.084863  1611 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:26.006541  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1673) took 5331 ms. Trace:
I20260709 13:00:26.026226  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1674) took 5328 ms. Trace:
I20260709 13:00:26.062127  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 470) took 1154 ms. Trace:
I20260709 13:00:26.086088  3622 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:26.086269  2236 rpcz_store.cc:276] 0709 13:00:24.907654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.788575 (+880921us) service_pool.cc:224] Handling call
0709 13:00:26.062068 (+273493us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.087522  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:26.088565  3622 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:26.088287  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 474) took 1150 ms. Trace:
I20260709 13:00:26.089207  2236 rpcz_store.cc:276] 0709 13:00:24.937817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.087288 (+1149471us) service_pool.cc:224] Handling call
0709 13:00:26.088224 (+   936us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.090160  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.090871  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 773) took 1208 ms. Trace:
I20260709 13:00:26.061506  1685 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:26.092269  2236 rpcz_store.cc:276] 0709 13:00:24.882741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.089913 (+1207172us) service_pool.cc:224] Handling call
0709 13:00:26.090819 (+   906us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:26.092553  1508 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.068s (SENT)
I20260709 13:00:26.062788  1614 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:26.078287  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.061743  3260 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.085330  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:26.094000  2230 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 563) took 2266 ms (client timeout 3011 ms). Trace:
I20260709 13:00:26.093889  3260 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:26.094250  2229 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
W20260709 13:00:26.094530  2230 rpcz_store.cc:269] 0709 13:00:23.828137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.077957 (+2249820us) service_pool.cc:224] Handling call
0709 13:00:26.093939 (+ 15982us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.095063  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 774) took 1204 ms. Trace:
I20260709 13:00:26.095644  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:26.074867  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 760) took 1271 ms. Trace:
I20260709 13:00:26.095970  2232 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:25.923768  3308 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:26.096271  2235 rpcz_store.cc:276] 0709 13:00:24.803276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.005998 (+1202722us) service_pool.cc:224] Handling call
0709 13:00:26.074796 (+ 68798us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.096650  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 569) took 1246 ms. Trace:
I20260709 13:00:26.097797  2234 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:26.097234  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.028350  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 574) took 1069 ms. Trace:
I20260709 13:00:26.098912  3635 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:26.098971  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 766) took 1294 ms. Trace:
I20260709 13:00:26.098918  2231 rpcz_store.cc:276] 0709 13:00:24.958710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.502108 (+543398us) service_pool.cc:224] Handling call
0709 13:00:26.027052 (+524944us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.104051  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:26.104580  3688 log.cc:927] Time spent T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.115s	user 0.000s	sys 0.003s
I20260709 13:00:26.104871  2231 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:26.105314  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 467) took 1207 ms. Trace:
I20260709 13:00:26.105706  2231 rpcz_store.cc:276] 0709 13:00:24.897545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.103802 (+1206257us) service_pool.cc:224] Handling call
0709 13:00:26.105259 (+  1457us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.106648  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.107324  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 472) took 1182 ms. Trace:
I20260709 13:00:26.107666  2231 rpcz_store.cc:276] 0709 13:00:24.924623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.106362 (+1181739us) service_pool.cc:224] Handling call
0709 13:00:26.107274 (+   912us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:26.108131  3670 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.619s	user 0.010s	sys 0.000s
I20260709 13:00:26.108246  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:26.108584  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:26.108867  1613 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.109352  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 572) took 1217 ms. Trace:
I20260709 13:00:26.109747  2231 rpcz_store.cc:276] 0709 13:00:24.892282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.108343 (+1216061us) service_pool.cc:224] Handling call
0709 13:00:26.109299 (+   956us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.110658  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.111404  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 777) took 1182 ms. Trace:
I20260709 13:00:26.111725  2231 rpcz_store.cc:276] 0709 13:00:24.929454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.110394 (+1180940us) service_pool.cc:224] Handling call
0709 13:00:26.111330 (+   936us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.112622  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:26.098307  3308 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:26.093096  1508 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:26.114163  3599 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:26.108610  3670 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:26.088122  1534 rpcz_store.cc:276] 0709 13:00:20.657570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.829184 (+5171614us) service_pool.cc:224] Handling call
0709 13:00:25.891210 (+ 62026us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.902577 (+ 11367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.907543 (+  4966us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:25.919580 (+ 12037us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:25.926206 (+  6626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.958422 (+ 32216us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:25.988586 (+ 30164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.097084  2232 rpcz_store.cc:276] 0709 13:00:24.849771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.013263 (+1163492us) service_pool.cc:224] Handling call
0709 13:00:26.096590 (+ 83327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.115706  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.116484  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 779) took 1170 ms. Trace:
I20260709 13:00:26.074442  1615 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
W20260709 13:00:26.095556  3686 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:00:26.116869  2232 rpcz_store.cc:276] 0709 13:00:24.946109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.115422 (+1169313us) service_pool.cc:224] Handling call
0709 13:00:26.116426 (+  1004us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.100814  2230 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:26.117987  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.100149  2235 rpcz_store.cc:276] 0709 13:00:24.804745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.096981 (+1292236us) service_pool.cc:224] Handling call
0709 13:00:26.098919 (+  1938us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.101374  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.118633  2237 raft_consensus.cc:2410] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:26.119076  1612 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:26.118554  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 459) took 2471 ms. Trace:
I20260709 13:00:26.119305  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.119222  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:26.119891  2237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 464) took 2454 ms (client timeout 2068 ms). Trace:
I20260709 13:00:26.120038  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.120620  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 562) took 2402 ms. Trace:
I20260709 13:00:26.091295  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.121048  1612 raft_consensus.cc:2393] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 3: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:26.121001  2235 rpcz_store.cc:276] 0709 13:00:23.718787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.118983 (+2400196us) service_pool.cc:224] Handling call
0709 13:00:26.120566 (+  1583us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.121562  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1213) took 1512 ms. Trace:
I20260709 13:00:26.121742  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1212) took 1661 ms. Trace:
I20260709 13:00:26.027900  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.122192  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.095547  2229 rpcz_store.cc:276] 0709 13:00:24.890837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.085067 (+1194230us) service_pool.cc:224] Handling call
0709 13:00:26.095000 (+  9933us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.122148  1612 rpcz_store.cc:276] 0709 13:00:24.460958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.119780 (+1658822us) service_pool.cc:224] Handling call
0709 13:00:26.121683 (+  1903us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.122949  2235 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:26.123443  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 775) took 1224 ms. Trace:
I20260709 13:00:26.123374  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.123670  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.076073  1543 rpcz_store.cc:276] 0709 13:00:20.577066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.800230 (+5223164us) service_pool.cc:224] Handling call
0709 13:00:25.800543 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.821607 (+ 21064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.015028 (+193421us) spinlock_profiling.cc:243] Waited 2.87 ms on lock 0x7b540012ded4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:26.015224 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6597376}
I20260709 13:00:26.124233  2229 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:26.124816  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 463) took 2462 ms. Trace:
I20260709 13:00:26.093250  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.102046  3554 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:26.121997  1611 rpcz_store.cc:276] 0709 13:00:24.615654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.085953 (+1470299us) service_pool.cc:224] Handling call
0709 13:00:26.121501 (+ 35548us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.102902  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.171541  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 473) took 1243 ms. Trace:
I20260709 13:00:26.161890  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:26.172191  1614 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:26.172276  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:25.977633  3689 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.123963  2235 rpcz_store.cc:276] 0709 13:00:24.899017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.121928 (+1222911us) service_pool.cc:224] Handling call
0709 13:00:26.123392 (+  1464us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.117122  1613 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:26.174543  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:26.159776  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.175227  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1209) took 1963 ms. Trace:
I20260709 13:00:26.125031  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:26.120151  1615 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:26.026175  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 356) took 5470 ms. Trace:
I20260709 13:00:26.159469  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:26.127297  3519 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:26.120462  2237 rpcz_store.cc:269] 0709 13:00:23.665876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.797138 (+1131262us) service_pool.cc:224] Handling call
0709 13:00:26.118252 (+1321114us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54004bba20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa63dc0 00007fe08faefe18 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6
0709 13:00:26.119709 (+  1457us) spinlock_profiling.cc:243] Waited 99.6 ms on lock 0x7b54004bb254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:26.119829 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1779811840}
I20260709 13:00:26.125142  2229 rpcz_store.cc:276] 0709 13:00:23.662030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.123083 (+2461053us) service_pool.cc:224] Handling call
0709 13:00:26.124758 (+  1675us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.167845  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1668) took 5649 ms. Trace:
I20260709 13:00:26.119925  2230 rpcz_store.cc:276] 0709 13:00:23.647738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.095358 (+2447620us) service_pool.cc:224] Handling call
0709 13:00:26.118496 (+ 23138us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:26.008540  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 93763.5us
I20260709 13:00:26.183106  2203 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:25.990466  3619 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:26.171263  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.117992  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.173172  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.172200  2236 rpcz_store.cc:276] 0709 13:00:24.927796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.092997 (+1165201us) service_pool.cc:224] Handling call
0709 13:00:26.171481 (+ 78484us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.161886  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:26.036700  3615 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:26.175676  1613 rpcz_store.cc:276] 0709 13:00:24.212440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.108010 (+1895570us) service_pool.cc:224] Handling call
0709 13:00:26.175130 (+ 67120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.176402  1543 raft_consensus.cc:2368] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:26.170857  1612 raft_consensus.cc:2376] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4. Re-sending same reply.
I20260709 13:00:26.178191  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:26.191310  2236 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:26.227991  1542 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:26.228426  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 758) took 1425 ms. Trace:
I20260709 13:00:26.227981  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1732) took 1303 ms. Trace:
I20260709 13:00:26.229312  1612 rpcz_store.cc:276] 0709 13:00:24.924294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.123298 (+1199004us) service_pool.cc:224] Handling call
0709 13:00:26.227906 (+104608us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.230098  3625 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:26.230433  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:26.231175  1612 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.232137  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:26.181559  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.234071  2237 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:26.181095  2235 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:26.180464  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:26.235036  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 465) took 2384 ms. Trace:
W20260709 13:00:26.235191  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 54974.3us
I20260709 13:00:26.236037  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:26.236424  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 557) took 2560 ms. Trace:
I20260709 13:00:26.180042  1536 rpcz_store.cc:276] 0709 13:00:20.518127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.680314 (+5162187us) service_pool.cc:224] Handling call
0709 13:00:25.680632 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:25.681012 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.681040 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:25.692511 (+ 11471us) log.cc:844] Serialized 26 byte log entry
0709 13:00:25.796582 (+104071us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:25.796647 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:25.796745 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:26.167528 (+370783us) raft_consensus.cc:1685] finished
0709 13:00:26.167559 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.167763 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":719,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":172,"thread_start_us":53956,"threads_started":1,"wal-append.queue_time_us":172197}
I20260709 13:00:26.236821  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.237078  1509 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:26.186295  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:26.184165  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:26.186587  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.190830  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.191028  2230 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:26.178635  3554 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:26.188390  1537 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:26.228852  2236 rpcz_store.cc:276] 0709 13:00:24.802747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.188177 (+1385430us) service_pool.cc:224] Handling call
0709 13:00:26.191053 (+  2876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.228160 (+ 37107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.228371 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.086191  1540 rpcz_store.cc:276] 0709 13:00:20.697918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.831936 (+5134018us) service_pool.cc:224] Handling call
0709 13:00:25.880925 (+ 48989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.881123 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:25.952361 (+ 71238us) log.cc:844] Serialized 14 byte log entry
0709 13:00:26.025879 (+ 73518us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:26.025943 (+    64us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:26.025999 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.026063 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.026175 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2907,"prepare.queue_time_us":179312,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":5080,"raft.run_wall_time_us":24264,"spinlock_wait_cycles":3482368,"thread_start_us":120590,"threads_started":2}
I20260709 13:00:26.233280  1613 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:26.180428  3519 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:26.236858  2235 rpcz_store.cc:276] 0709 13:00:23.676251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.176404 (+2500153us) service_pool.cc:224] Handling call
0709 13:00:26.180904 (+  4500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.236162 (+ 55258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.236359 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.180115  3689 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:26.238101  3620 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:26.239940  3555 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:26.240157  3571 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:26.240286  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.188759  3308 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:26.239784  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:26.185128  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.185971  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.246958  1536 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:26.270027  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1678) took 3776 ms. Trace:
I20260709 13:00:26.247558  2232 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:26.270130  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 565) took 1489 ms. Trace:
I20260709 13:00:26.250874  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.260160  3620 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:26.270371  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:26.260423  3555 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.277462  3555 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.280911  2236 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:26.286209  3693 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
W20260709 13:00:26.286589  1657 connection.cc:256] RPC call timeout handler was delayed by -2.1163822160000336s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2016s for a response
W20260709 13:00:26.287269  1657 connection.cc:256] RPC call timeout handler was delayed by -1.9338580040000011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
W20260709 13:00:26.288502  1657 connection.cc:256] RPC call timeout handler was delayed by -1.7312199969999824s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
I20260709 13:00:26.288733  3689 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:26.251924  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1221) took 1569 ms. Trace:
I20260709 13:00:26.259275  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:26.253871  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 69763.8us
I20260709 13:00:26.240525  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.289829  1613 raft_consensus.cc:2376] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4. Re-sending same reply.
I20260709 13:00:26.251852  3580 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:26.270634  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 761) took 1467 ms. Trace:
I20260709 13:00:26.270488  1536 rpcz_store.cc:276] 0709 13:00:22.493519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.246463 (+3752944us) service_pool.cc:224] Handling call
0709 13:00:26.246784 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.269699 (+ 22915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.269943 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.256683  3189 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:26.274227  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 573) took 1334 ms. Trace:
I20260709 13:00:26.249154  3300 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:26.262373  3571 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.275817  3620 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:26.288986  1657 connection.cc:256] RPC call timeout handler was delayed by -1.6898199969999723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
I20260709 13:00:26.290385  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1736) took 1365 ms. Trace:
I20260709 13:00:26.318516  1613 rpcz_store.cc:276] 0709 13:00:24.925336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.259055 (+1333719us) service_pool.cc:224] Handling call
0709 13:00:26.290329 (+ 31274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.319559  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:26.320363  1613 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:26.256971  3516 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:26.261184  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.330477  1692 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 1.
I20260709 13:00:26.235611  2237 rpcz_store.cc:276] 0709 13:00:23.850601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.179054 (+2328453us) service_pool.cc:224] Handling call
0709 13:00:26.189585 (+ 10531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.234691 (+ 45106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.234958 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.291023  1614 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:26.177268  1535 rpcz_store.cc:276] 0709 13:00:20.555398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.795986 (+5240588us) service_pool.cc:224] Handling call
0709 13:00:25.842136 (+ 46150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.006602 (+164466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.024095 (+ 17493us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.294827  2235 rpcz_store.cc:276] 0709 13:00:24.803507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.257246 (+1453739us) service_pool.cc:224] Handling call
0709 13:00:26.269546 (+ 12300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.269831 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.270221 (+   390us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:26.270290 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:26.270339 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.270412 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.270560 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.272635  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:26.274829  2230 rpcz_store.cc:276] 0709 13:00:24.774603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.190285 (+1415682us) service_pool.cc:224] Handling call
0709 13:00:26.190662 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.255168 (+ 64506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.264182 (+  9014us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.296051  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1682) took 3643 ms. Trace:
I20260709 13:00:26.296475  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 1982) took 3745 ms. Trace:
I20260709 13:00:26.259384  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:26.268030  3308 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.322155  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.296036  3349 consensus_queue.cc:1059] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:00:26.346273  1685 raft_consensus.cc:2410] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:00:26.328698  3571 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.332500  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:26.373889  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.374562  2237 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:26.347230  1507 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:26.303016  2232 rpcz_store.cc:276] 0709 13:00:24.939744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.117710 (+1177966us) service_pool.cc:224] Handling call
0709 13:00:26.274161 (+156451us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.358834  2229 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:26.358708  1536 rpcz_store.cc:276] 0709 13:00:22.653095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.295265 (+3642170us) service_pool.cc:224] Handling call
0709 13:00:26.295555 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.295728 (+   173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.295752 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:26.295795 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:26.295831 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.295884 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.295997 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.358996  1690 rpcz_store.cc:276] 0709 13:00:22.551052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:22.760039 (+208987us) service_pool.cc:224] Handling call
0709 13:00:22.760282 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:22.797787 (+ 37505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:22.797818 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:22.797855 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:22.797885 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.296256 (+3498371us) spinlock_profiling.cc:243] Waited 37.3 ms on lock 0x7b54002ccc54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:26.296301 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.296422 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":85780736}
I20260709 13:00:26.359016  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1675) took 5608 ms. Trace:
I20260709 13:00:26.289618  1611 rpcz_store.cc:276] 0709 13:00:24.682428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.188597 (+1506169us) service_pool.cc:224] Handling call
0709 13:00:26.251849 (+ 63252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.325933  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:26.371290  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.353078  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.376224  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:26.389619  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.349799  3516 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:26.363430  3554 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:26.391185  3594 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:00:26.393920  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.394255  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.394614  1683 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:26.394838  1618 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:26.395181  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2155) took 5928 ms. Trace:
W20260709 13:00:26.395303  1618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1215) took 1715 ms (client timeout 2095 ms). Trace:
I20260709 13:00:26.393126  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.362624  3308 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.353360  3300 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:26.397564  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 567) took 1571 ms. Trace:
I20260709 13:00:26.380728  1539 rpcz_store.cc:276] 0709 13:00:20.750775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.911367 (+5160592us) service_pool.cc:224] Handling call
0709 13:00:25.911684 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.911900 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:26.244709 (+332809us) log.cc:844] Serialized 14 byte log entry
0709 13:00:26.358663 (+113954us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:26.358718 (+    55us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:26.358767 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.358840 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.358959 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21126,"prepare.queue_time_us":173754,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"thread_start_us":373031,"threads_started":2}
W20260709 13:00:26.395735  1618 rpcz_store.cc:269] 0709 13:00:24.680421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.393993 (+1713572us) service_pool.cc:224] Handling call
0709 13:00:26.395256 (+  1263us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:26.398087  2233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 771) took 1527 ms (client timeout 1580 ms). Trace:
I20260709 13:00:26.385020  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:26.405792  2233 rpcz_store.cc:269] 0709 13:00:24.870863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.487452 (+616589us) service_pool.cc:224] Handling call
0709 13:00:26.397876 (+910424us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54003bced4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:26.398008 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":598568832}
I20260709 13:00:26.406369  1614 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:00:26.388582  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 62799.9us
I20260709 13:00:26.368160  1534 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:26.391872  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:26.392282  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2159) took 2936 ms. Trace:
I20260709 13:00:26.392359  2237 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:26.378350  3414 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:26.294754  3580 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:26.417778  1613 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:26.396169  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:26.421880  1692 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.398985  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:26.384770  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 145058us
I20260709 13:00:26.424860  1539 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:26.424975  1509 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:26.425519  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1690) took 2934 ms. Trace:
I20260709 13:00:26.425871  1539 rpcz_store.cc:276] 0709 13:00:23.490927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.404012 (+2913085us) service_pool.cc:224] Handling call
0709 13:00:26.424552 (+ 20540us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.425266 (+   714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.425470 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.391582  3349 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:26.324050  1657 connection.cc:256] RPC call timeout handler was delayed by -1.6446304589999841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22200000000000003s for a response
I20260709 13:00:26.395586  1538 rpcz_store.cc:276] 0709 13:00:20.467079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.648545 (+5181466us) service_pool.cc:224] Handling call
0709 13:00:25.648974 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:25.733652 (+ 84678us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:25.733701 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:25.734371 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 13:00:25.899478 (+165107us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:26.042396 (+142918us) log.cc:844] Serialized 14 byte log entry
0709 13:00:26.060798 (+ 18402us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.104010 (+ 43212us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:00:26.394854 (+290844us) spinlock_profiling.cc:243] Waited 68.6 ms on lock 0x7b54003444d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:26.394910 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:26.394968 (+    58us) raft_consensus.cc:1685] finished
0709 13:00:26.394980 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.395125 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3842,"prepare.queue_time_us":185845,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":18312,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":18156,"spinlock_wait_cycles":166451328,"thread_start_us":219475,"threads_started":2,"wal-append.queue_time_us":173890}
W20260709 13:00:26.439549  1657 connection.cc:256] RPC call timeout handler was delayed by -1.6286199969999871s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1648s for a response
W20260709 13:00:26.440018  1657 connection.cc:256] RPC call timeout handler was delayed by -1.3181199969999966s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19930000000000003s for a response
I20260709 13:00:26.398355  3516 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.404995  1688 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:26.443732  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.444938  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:26.409621  1536 rpcz_store.cc:276] 0709 13:00:23.456096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.387634 (+2931538us) service_pool.cc:224] Handling call
0709 13:00:26.387973 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.391947 (+  3974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.391978 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:26.392021 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:26.392055 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.392118 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.392229 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3646}
I20260709 13:00:26.446334  3551 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:26.446947  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 767) took 1633 ms. Trace:
I20260709 13:00:26.447367  2232 rpcz_store.cc:276] 0709 13:00:24.813726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.402399 (+1588673us) service_pool.cc:224] Handling call
0709 13:00:26.446332 (+ 43933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.446549 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.446591 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:26.446652 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:26.446690 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.446753 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.446886 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.410099  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 763) took 1606 ms. Trace:
I20260709 13:00:26.408670  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:26.403743  2235 rpcz_store.cc:276] 0709 13:00:24.826326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.396759 (+1570433us) service_pool.cc:224] Handling call
0709 13:00:26.397071 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.397254 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.397283 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:26.397321 (+    38us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:26.397354 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.397401 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.397512 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.449863  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.450732  1611 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:26.451090  2235 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:26.451638  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.451872  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 770) took 1591 ms. Trace:
I20260709 13:00:26.452422  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1223) took 1087 ms. Trace:
I20260709 13:00:26.381723  2231 raft_consensus.cc:2368] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5.
I20260709 13:00:26.435032  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.452450  2235 rpcz_store.cc:276] 0709 13:00:24.860684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.450420 (+1589736us) service_pool.cc:224] Handling call
0709 13:00:26.450828 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.451595 (+   767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.451803 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.454957  1688 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.436507  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:26.455158  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46576.1us
I20260709 13:00:26.437222  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.438076  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:26.440318  1657 connection.cc:256] RPC call timeout handler was delayed by -1.2452199969999924s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2074s for a response
I20260709 13:00:26.409442  1690 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:26.456785  1657 connection.cc:256] RPC call timeout handler was delayed by -1.1138199970000073s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22200000000000003s for a response
W20260709 13:00:26.438951  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 91815.4us
I20260709 13:00:26.434911  3620 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:26.457617  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:26.449584  2237 rpcz_store.cc:276] 0709 13:00:24.804024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.375263 (+1571239us) service_pool.cc:224] Handling call
0709 13:00:26.392163 (+ 16900us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.409780 (+ 17617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.410020 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.405606  3162 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:26.458280  1507 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:26.462980  1508 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:26.471555  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.471767  3696 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:26.471922  2235 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:00:26.472371  3696 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:26.472584  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 570) took 1600 ms. Trace:
I20260709 13:00:26.472782  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.449361  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 768) took 1605 ms. Trace:
I20260709 13:00:26.465636  1583 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:26.477939  1583 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:26.473033  2235 rpcz_store.cc:276] 0709 13:00:24.872704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.455269 (+1582565us) service_pool.cc:224] Handling call
0709 13:00:26.471571 (+ 16302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.472356 (+   785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.472530 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.479987  1539 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:26.480058  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 571) took 1591 ms. Trace:
I20260709 13:00:26.473409  1615 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:26.474123  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.477309  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:26.482020  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1691) took 2986 ms. Trace:
I20260709 13:00:26.482466  2235 rpcz_store.cc:276] 0709 13:00:24.888417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.478975 (+1590558us) service_pool.cc:224] Handling call
0709 13:00:26.479368 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.479608 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.479649 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:26.479710 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:26.479768 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.479842 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.479977 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.421202  3414 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:26.491906  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:26.492656  1614 raft_consensus.cc:2376] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5. Re-sending same reply.
I20260709 13:00:26.484236  1539 rpcz_store.cc:276] 0709 13:00:23.495781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.426641 (+2930860us) service_pool.cc:224] Handling call
0709 13:00:26.479684 (+ 53043us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.480474 (+   790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.481965 (+  1491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.475646  3620 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:26.474982  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:26.476593  3162 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:26.473543  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 776) took 1551 ms. Trace:
I20260709 13:00:26.531970  3580 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:26.660869  3189 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:26.661046  3162 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.661504  3189 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:26.663729  3189 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:26.647109  1612 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:26.664896  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:26.665499  1612 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.666144  3308 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:26.667289  2230 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 4. Ops: []
I20260709 13:00:26.667536  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 1994) took 1906 ms. Trace:
I20260709 13:00:26.667644  3571 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:26.668128  1684 rpcz_store.cc:276] 0709 13:00:24.761529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.801021 (+ 39492us) service_pool.cc:224] Handling call
0709 13:00:24.801346 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:25.745135 (+943789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.667265 (+922130us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b540003fa54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:26.667467 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1625867648}
I20260709 13:00:26.660674  2231 rpcz_store.cc:276] 0709 13:00:24.922293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.112356 (+1190063us) service_pool.cc:224] Handling call
0709 13:00:26.473316 (+360960us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54003d3bd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:26.473483 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564553984}
I20260709 13:00:26.653676  3325 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:26.661070  3580 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:26.504747  3414 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:26.470988  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 80845.1us
W20260709 13:00:26.610644  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 185736us
I20260709 13:00:26.678524  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:26.626459  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:26.408377  3568 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.076s
I20260709 13:00:26.629365  3555 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:26.629086  1692 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:26.494990  3670 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 13:00:26.688283  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1694) took 1965 ms. Trace:
I20260709 13:00:26.473374  3696 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:26.688757  1539 rpcz_store.cc:276] 0709 13:00:24.722779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.687363 (+1964584us) service_pool.cc:224] Handling call
0709 13:00:26.687756 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.687965 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.687992 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:26.688033 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:26.688068 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.688119 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.688228 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.633183  2237 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:26.486455  3615 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:26.473824  2232 rpcz_store.cc:276] 0709 13:00:24.844236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.448449 (+1604213us) service_pool.cc:224] Handling call
0709 13:00:26.448791 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.448985 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.449026 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:26.449069 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:26.449113 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.449170 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.449301 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.493731  3625 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:00:26.537585  3349 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:26.452993  1611 rpcz_store.cc:276] 0709 13:00:25.364888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.451353 (+1086465us) service_pool.cc:224] Handling call
0709 13:00:26.452363 (+  1010us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.646718  1617 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:26.638530  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.631878  1438 catalog_manager.cc:5697] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 3, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 3 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:26.650905  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:26.493160  2201 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:26.476334  1535 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 1. Current term is 2. Ops: []
W20260709 13:00:26.652269  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 194486us
I20260709 13:00:26.641510  3421 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
W20260709 13:00:26.653465  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 174959us
W20260709 13:00:26.680569  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.724340  1657 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:26.493917  3676 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:26.686087  1536 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:26.686911  1612 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:26.687677  3555 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:26.693889  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 778) took 1759 ms. Trace:
I20260709 13:00:26.669188  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 762) took 1865 ms. Trace:
W20260709 13:00:26.696646  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 233770us
W20260709 13:00:26.725739  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 232702us
I20260709 13:00:26.743223  1508 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:26.727360  1617 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1220) took 2045 ms (client timeout 2163 ms). Trace:
W20260709 13:00:26.745699  1617 rpcz_store.cc:269] 0709 13:00:24.682120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.398670 (+1716550us) service_pool.cc:224] Handling call
0709 13:00:26.727285 (+328615us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.729323  1688 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:26.736044  2237 rpcz_store.cc:276] 0709 13:00:24.934720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.502697 (+1567977us) service_pool.cc:224] Handling call
0709 13:00:26.633013 (+130316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.693629 (+ 60616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.693831 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.695353  3620 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:26.750571  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 72580.1us
I20260709 13:00:26.697247  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.725025  3189 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.725364  3349 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:26.753907  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.764098  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1216) took 2083 ms. Trace:
I20260709 13:00:26.764469  1617 rpcz_store.cc:276] 0709 13:00:24.680818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.753665 (+2072847us) service_pool.cc:224] Handling call
0709 13:00:26.764020 (+ 10355us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.765407  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.732195  3676 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:26.763064  1657 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:26.762578  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:26.731045  3706 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:26.773770  3702 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:26.734155  3708 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:26.740921  3615 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:26.779062  1688 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:00:26.777626  1615 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:26.780750  1614 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:26.781734  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 318) took 1444 ms. Trace:
I20260709 13:00:26.750059  3701 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:26.742982  1609 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:26.768748  3349 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.764247  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2158) took 3545 ms. Trace:
I20260709 13:00:26.787041  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:26.757218  3300 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:00:26.785642  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1714) took 3278 ms. Trace:
I20260709 13:00:26.777335  3676 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.787339  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:26.795333  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:26.809970  2237 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:26.787335  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.785918  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1717) took 3278 ms. Trace:
I20260709 13:00:26.786069  1617 rpcz_store.cc:276] 0709 13:00:25.337765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.765181 (+1427416us) service_pool.cc:224] Handling call
0709 13:00:26.781671 (+ 16490us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.784619  3706 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:26.811429  1615 rpcz_store.cc:276] 0709 13:00:23.506975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.493963 (+2986988us) service_pool.cc:224] Handling call
0709 13:00:26.785587 (+291624us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:26.766549  3189 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:26.822201  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:26.812111  1584 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1740) took 1018 ms. Trace:
I20260709 13:00:26.844506  1584 rpcz_store.cc:276] 0709 13:00:25.793676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.812054 (+1018378us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:00:26.835996  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.836604  1692 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:26.836951  1614 rpcz_store.cc:276] 0709 13:00:23.508068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.511569 (+3003501us) service_pool.cc:224] Handling call
0709 13:00:26.785866 (+274297us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:26.845323  1584 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43412 dropped due to backpressure: service queue is full with 50 items
I20260709 13:00:26.852874  1690 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:26.785317  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:26.774899  3698 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:26.857445  1688 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:00:26.863940  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 81319us
I20260709 13:00:26.792007  3555 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.791898  3551 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:26.869457  3421 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:26.784461  3710 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:26.792667  1609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1219) took 2110 ms (client timeout 2202 ms). Trace:
W20260709 13:00:26.902667  1609 rpcz_store.cc:269] 0709 13:00:24.681796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.632139 (+1950343us) service_pool.cc:224] Handling call
0709 13:00:26.790274 (+158135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44416}
I20260709 13:00:26.904742  3260 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:26.906217  1616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1217) took 2225 ms (client timeout 1577 ms). Trace:
I20260709 13:00:26.779312  3689 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260709 13:00:26.906500  1540 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:26.816438  3554 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:26.819213  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:26.886770  3702 consensus_queue.cc:1243] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:00:26.888000  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 121243us
I20260709 13:00:26.890965  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:26.892922  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1695) took 2163 ms. Trace:
W20260709 13:00:26.898389  1440 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55368 (request call id 325) took 12477 ms (client timeout 15000 ms). Trace:
I20260709 13:00:26.901160  1438 catalog_manager.cc:5697] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 4, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 4 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:26.906307  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 366) took 1212 ms. Trace:
I20260709 13:00:26.814834  3230 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:26.742353  2230 rpcz_store.cc:276] 0709 13:00:24.803761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.353862 (+1550101us) service_pool.cc:224] Handling call
0709 13:00:26.354155 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.667724 (+313569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.668933 (+  1209us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54002f49d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:26.669125 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":719691520}
I20260709 13:00:26.844933  3619 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
W20260709 13:00:26.906802  1616 rpcz_store.cc:269] 0709 13:00:24.681125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.288637 (+607512us) service_pool.cc:224] Handling call
0709 13:00:26.906044 (+1617407us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b5400105754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:26.906150 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1116933376}
W20260709 13:00:26.976598  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.068727  3554 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:26.792333  1535 rpcz_store.cc:276] 0709 13:00:23.215305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.370158 (+3154853us) service_pool.cc:224] Handling call
0709 13:00:26.423013 (+ 52855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.733902 (+310889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.760188 (+ 26286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.071686  3400 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:27.072341  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:27.070744  3600 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:26.922557  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:27.081499  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:26.990005  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:27.082211  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:27.082928  1692 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
W20260709 13:00:27.081085  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 158637us
I20260709 13:00:27.078800  1438 catalog_manager.cc:5697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:27.084739  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:26.910151  2237 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:27.084524  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:26.881956  3709 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:27.083954  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 320992us
I20260709 13:00:27.076057  1541 rpcz_store.cc:276] 0709 13:00:25.693592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:25.747532 (+ 53940us) service_pool.cc:224] Handling call
0709 13:00:26.906123 (+1158591us) spinlock_profiling.cc:243] Waited 35.3 ms on lock 0x7b5400343ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:26.906254 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81212288}
I20260709 13:00:27.068990  3667 raft_consensus.cc:3055] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:26.847354  1584 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:43412 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1705)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1706)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1708)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 312)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1709)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1711)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1713)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1715)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1199)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1208)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1733)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1734)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1737)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1738)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 315)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 316)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 325)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 328)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 334)
I20260709 13:00:26.870097  3698 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:27.091374  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:27.097663  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1681) took 4449 ms. Trace:
I20260709 13:00:27.120347  1540 rpcz_store.cc:276] 0709 13:00:22.647842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.270041 (+3622199us) service_pool.cc:224] Handling call
0709 13:00:26.906268 (+636227us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.907041 (+   773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.097386 (+190345us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b5400343ad0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:27.097578 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1432796928}
I20260709 13:00:27.109119  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:27.109651  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:27.102048  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:27.103412  1438 catalog_manager.cc:5697] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 3, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 3 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:27.118980  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:27.087003  1539 rpcz_store.cc:276] 0709 13:00:24.729736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.690690 (+1960954us) service_pool.cc:224] Handling call
0709 13:00:26.691011 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.691198 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:26.705896 (+ 14698us) log.cc:844] Serialized 14 byte log entry
0709 13:00:26.892582 (+186686us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:26.892638 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:26.892678 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.892740 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:26.892864 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4979,"prepare.queue_time_us":33030,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":5805,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":7042,"thread_start_us":181332,"threads_started":1}
I20260709 13:00:27.121930  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:26.909946  3710 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.126925  1540 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:27.122861  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:27.127548  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 384368us
I20260709 13:00:27.124418  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:27.131676  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:27.132164  1539 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.132331  1508 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:27.090389  2203 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:27.113021  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:27.125857  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:27.120035  3698 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.128691  1438 catalog_manager.cc:5697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 3, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 3 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:00:27.076545  1440 rpcz_store.cc:269] 0709 13:00:14.420573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:14.424079 (+  3506us) service_pool.cc:224] Handling call
0709 13:00:14.908607 (+484528us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:00:26.898308 (+11989701us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:19.195892 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:19.445503 (+249611us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:19.445559 (+    56us) write_op.cc:435] Acquired schema lock
0709 13:00:19.445576 (+    17us) tablet.cc:662] Decoding operations
0709 13:00:19.454131 (+  8555us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:19.454227 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:19.454258 (+    31us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:00:19.454759 (+   501us) tablet.cc:701] Row locks acquired
0709 13:00:19.454806 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:00:19.454994 (+   188us) write_op.cc:270] Start()
0709 13:00:19.455148 (+   154us) write_op.cc:276] Timestamp: P: 1783602019454932 usec, L: 0
0709 13:00:19.455184 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:00:19.459292 (+  4108us) log.cc:844] Serialized 1686 byte log entry
0709 13:00:20.059568 (+600276us) op_driver.cc:464] REPLICATION: finished
0709 13:00:20.484594 (+425026us) write_op.cc:301] APPLY: starting
0709 13:00:20.496401 (+ 11807us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:20.496608 (+   207us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:20.545616 (+ 49008us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:20.585996 (+ 40380us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:20.586316 (+   320us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:00:20.586372 (+    56us) write_op.cc:312] APPLY: finished
0709 13:00:20.586699 (+   327us) log.cc:844] Serialized 41 byte log entry
0709 13:00:20.586892 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:20.586986 (+    94us) write_op.cc:454] Released schema lock
0709 13:00:20.587039 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:20.587454 (+   415us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:20.587481 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":698948,"reported_tablets":11,"spinlock_wait_cycles":436714240,"child_traces":[["op",{"apply.queue_time_us":361144,"apply.run_cpu_time_us":3223,"apply.run_wall_time_us":140723,"num_ops":4,"prepare.queue_time_us":142048,"prepare.run_cpu_time_us":19067,"prepare.run_wall_time_us":396171,"raft.queue_time_us":202385,"raft.run_cpu_time_us":23395,"raft.run_wall_time_us":166465,"replication_time_us":604206,"thread_start_us":462071,"threads_started":4,"wal-append.queue_time_us":220434,"wal-append.run_cpu_time_us":59130,"wal-append.run_wall_time_us":2030600}]]}
I20260709 13:00:27.123658  1541 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:26.907104  2232 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:27.062721  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.990299  3303 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.143s
I20260709 13:00:27.096091  3680 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:00:27.081058  3706 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.183199  3680 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:27.182498  1438 catalog_manager.cc:5697] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 4, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 4 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:27.184123  3680 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.184386  3676 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.096046  3667 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:27.185657  3726 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:27.186105  3667 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:27.186715  1610 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:27.187121  3667 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.187224  2228 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
W20260709 13:00:27.187821  2228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 456) took 3543 ms (client timeout 3076 ms). Trace:
W20260709 13:00:27.188265  2228 rpcz_store.cc:269] 0709 13:00:23.644889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:23.647184 (+  2295us) service_pool.cc:224] Handling call
0709 13:00:25.848531 (+2201347us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:27.046556 (+1198025us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7e9c 00007fe08c9c7a5b 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:27.187745 (+141189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40792320}
I20260709 13:00:27.137238  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:27.189941  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:27.190498  2237 raft_consensus.cc:2376] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6. Re-sending same reply.
I20260709 13:00:27.158702  3600 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:27.191263  3600 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:27.191320  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:27.192454  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:27.192227  3600 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:27.151954  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 61636.1us
W20260709 13:00:27.193905  3516 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.708s	user 0.010s	sys 0.000s
I20260709 13:00:27.193842  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:27.119832  3696 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:27.194356  3516 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.194904  3696 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.195288  3696 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.195485  2233 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:27.196480  3516 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:27.091022  3599 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:27.096199  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:27.155458  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:27.214911  1543 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:27.099078  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:27.138015  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:27.219388  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:27.159624  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:27.160494  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1680) took 4477 ms. Trace:
I20260709 13:00:27.095347  3349 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.135200  3710 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:27.179041  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:26.910164  3421 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.121403  3230 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:26.990947  3308 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:27.184634  1438 catalog_manager.cc:5697] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 3, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 3 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:27.051398  3615 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.095661  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 780) took 2140 ms. Trace:
I20260709 13:00:27.205369  3594 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:27.206373  3704 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:27.207969  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:27.211637  1539 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:27.204586  3599 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.153868  3400 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:27.129935  3551 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:27.220924  1508 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:27.205288  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 96171.2us
I20260709 13:00:27.221314  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:27.221926  1616 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:27.227093  1540 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:27.220466  1537 rpcz_store.cc:276] 0709 13:00:22.647559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.255669 (+3608110us) service_pool.cc:224] Handling call
0709 13:00:26.433480 (+177811us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b5800003998. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:00:26.433728 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.433950 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:26.434274 (+   324us) log.cc:844] Serialized 14 byte log entry
0709 13:00:27.124759 (+690485us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:27.124848 (+    89us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:27.124918 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.125003 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.125107 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2602,"prepare.queue_time_us":606,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":113924,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":9702,"spinlock_wait_cycles":2824960,"thread_start_us":689922,"threads_started":2,"wal-append.queue_time_us":243563}
I20260709 13:00:27.230629  1438 catalog_manager.cc:5697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 3, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 3 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:27.265394  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1709) took 6359 ms. Trace:
I20260709 13:00:27.265832  1616 rpcz_store.cc:276] 0709 13:00:20.905876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.173745 (+6267869us) service_pool.cc:224] Handling call
0709 13:00:27.221733 (+ 47988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.265112 (+ 43379us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.265332 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.266250  1438 catalog_manager.cc:5697] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 3, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 3 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:00:27.201926  1617 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:27.268589  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1705) took 6467 ms. Trace:
I20260709 13:00:27.234831  3230 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:27.235060  3594 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:27.222514  2203 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:27.273168  3726 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:27.274201  3726 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:27.274358  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 51943.6us
I20260709 13:00:27.274521  1692 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:27.274619  3726 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.275166  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:27.245114  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 113683us
I20260709 13:00:27.275660  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:27.276257  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:27.276906  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:27.222878  3580 consensus_meta.cc:287] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.551s	user 0.004s	sys 0.012s
I20260709 13:00:27.277606  3308 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:27.241989  1583 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:27.228711  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:27.241389  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:27.240270  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 127299us
I20260709 13:00:27.138703  3670 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:27.268889  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:27.268589  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 363) took 1604 ms. Trace:
I20260709 13:00:27.282770  1688 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.282755  1537 rpcz_store.cc:276] 0709 13:00:25.664288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.267791 (+1603503us) service_pool.cc:224] Handling call
0709 13:00:27.268120 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.268304 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:27.268325 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:27.268363 (+    38us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:27.268393 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.268434 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.268517 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.250128  1539 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:00:27.268924  1617 rpcz_store.cc:276] 0709 13:00:20.801008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.175506 (+6374498us) service_pool.cc:224] Handling call
0709 13:00:27.201711 (+ 26205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.268364 (+ 66653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.268529 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.284157  1539 pending_rounds.cc:85] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 4
W20260709 13:00:27.273138  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 52366.2us
I20260709 13:00:27.242383  3631 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:27.273228  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:27.278034  3230 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:27.243014  3599 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.287784  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1704) took 6487 ms. Trace:
I20260709 13:00:27.284603  3704 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:27.227578  3554 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:27.291095  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:27.279877  1692 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:27.267239  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2177) took 2333 ms. Trace:
I20260709 13:00:27.273550  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:27.234616  3676 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:27.280634  2237 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.226449  3724 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.340658  1617 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:27.283788  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:27.393821  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 102820us
I20260709 13:00:27.270459  1543 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.394011  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1713) took 5905 ms. Trace:
I20260709 13:00:27.392441  1688 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:27.394586  1617 rpcz_store.cc:276] 0709 13:00:21.488610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.339786 (+5851176us) service_pool.cc:224] Handling call
0709 13:00:27.340219 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.393604 (+ 53385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.393910 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.248317  1612 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:00:27.394033  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 110302us
I20260709 13:00:27.307113  1615 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:27.311132  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1711) took 6219 ms. Trace:
W20260709 13:00:27.329150  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 46045.9us
I20260709 13:00:27.248489  3700 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:27.290637  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 48781.4us
I20260709 13:00:27.396205  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1706) took 6595 ms. Trace:
I20260709 13:00:27.428010  1615 rpcz_store.cc:276] 0709 13:00:20.801252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.306591 (+6505339us) service_pool.cc:224] Handling call
0709 13:00:27.306898 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.395980 (+ 89082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.396153 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.228034  3189 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:27.240535  3571 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:27.392947  1540 rpcz_store.cc:276] 0709 13:00:24.933951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.222077 (+2288126us) service_pool.cc:224] Handling call
0709 13:00:27.222346 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.266996 (+ 44650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.267186 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.396175  2203 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:27.398487  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:27.428670  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:27.437451  1584 leader_election.cc:400] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:27.437863  1584 leader_election.cc:403] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:00:27.279217  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 312) took 6417 ms. Trace:
I20260709 13:00:27.290894  3715 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:27.395077  1539 pending_rounds.cc:107] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:00:27.454623  1539 raft_consensus.cc:2889] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:27.398654  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 117546us
I20260709 13:00:27.245870  3546 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:27.432572  1615 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:27.432927  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 128329us
I20260709 13:00:27.432168  3554 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:27.299599  3537 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:27.438037  2228 raft_consensus.cc:2368] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4.
I20260709 13:00:27.286409  3400 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:00:27.298802  3162 consensus_meta.cc:287] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.637s	user 0.009s	sys 0.001s
I20260709 13:00:27.438918  1584 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:27.449470  3676 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:27.444885  2237 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:27.298373  1535 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:27.455762  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 361) took 1808 ms. Trace:
I20260709 13:00:27.281807  3325 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:27.486599  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:27.301860  3704 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.486126  1617 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:27.398216  3300 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:27.437963  3580 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:00:27.430104  3717 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:27.488687  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1199) took 3602 ms. Trace:
I20260709 13:00:27.396675  1616 rpcz_store.cc:276] 0709 13:00:21.091801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.266658 (+6174857us) service_pool.cc:224] Handling call
0709 13:00:27.310443 (+ 43785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.310685 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:27.310722 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:27.310780 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:27.310831 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.310902 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.311058 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.575205  1615 rpcz_store.cc:276] 0709 13:00:23.885985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.432046 (+3546061us) service_pool.cc:224] Handling call
0709 13:00:27.432402 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.485055 (+ 52653us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.488628 (+  3573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.504475  3676 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.433176  3700 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:27.495428  1657 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:27.498764  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 59940.8us
I20260709 13:00:27.301079  1614 rpcz_store.cc:276] 0709 13:00:20.800769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.861603 (+6060834us) service_pool.cc:224] Handling call
0709 13:00:26.861913 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.862232 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.862257 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:26.882036 (+ 19779us) log.cc:844] Serialized 26 byte log entry
0709 13:00:27.129152 (+247116us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:27.129192 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.129271 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:27.287507 (+158236us) raft_consensus.cc:1685] finished
0709 13:00:27.287526 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.287691 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3852,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":101,"thread_start_us":82996,"threads_started":1,"wal-append.queue_time_us":286881}
I20260709 13:00:27.438647  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 331) took 1143 ms. Trace:
I20260709 13:00:27.450196  3619 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4.
I20260709 13:00:27.457242  3588 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:27.504860  1539 rpcz_store.cc:276] 0709 13:00:25.647794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.247696 (+1599902us) service_pool.cc:224] Handling call
0709 13:00:27.248843 (+  1147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.455418 (+206575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.455681 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.605512  1539 pending_rounds.cc:85] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 1
I20260709 13:00:27.605758  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1734) took 2680 ms. Trace:
I20260709 13:00:27.605993  1539 pending_rounds.cc:107] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:00:27.606261  1614 rpcz_store.cc:276] 0709 13:00:24.924842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.588700 (+2663858us) service_pool.cc:224] Handling call
0709 13:00:27.604937 (+ 16237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.605254 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:27.605296 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:27.605355 (+    59us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:27.605399 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.605518 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.605685 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.591233  3700 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.606403  1539 raft_consensus.cc:2889] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:27.584772  3651 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:27.591578  3715 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:27.624994  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1737) took 2698 ms. Trace:
I20260709 13:00:27.286207  3594 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.400230  3670 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:27.517917  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1715) took 4010 ms. Trace:
I20260709 13:00:27.485862  1508 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:27.542908  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.449038  3727 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:00:27.584344  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1693) took 2858 ms. Trace:
I20260709 13:00:27.429765  3724 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.439013  3709 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
W20260709 13:00:27.590345  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 95014.4us
I20260709 13:00:27.592639  1616 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:27.647025  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1208) took 3636 ms. Trace:
I20260709 13:00:27.506572  3346 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:27.595323  3710 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.503769  3595 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.081s
I20260709 13:00:27.520166  3704 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.601949  1615 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:27.655645  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1733) took 2731 ms. Trace:
I20260709 13:00:27.656041  1615 rpcz_store.cc:276] 0709 13:00:24.924596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.577712 (+2653116us) service_pool.cc:224] Handling call
0709 13:00:27.601497 (+ 23785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.655351 (+ 53854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.655573 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.589161  3676 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.504088  3162 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.603257  3725 consensus_queue.cc:1059] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:00:27.491009  1543 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:27.591327  3580 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:27.485033  3546 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:27.285569  3516 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:27.490903  3719 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
I20260709 13:00:27.453163  1611 rpcz_store.cc:276] 0709 13:00:20.847389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.125454 (+6278065us) service_pool.cc:224] Handling call
0709 13:00:27.190267 (+ 64813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.219554 (+ 29287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:27.223005 (+  3451us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:27.230641 (+  7636us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:27.235621 (+  4980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.240719 (+  5098us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.264784 (+ 24065us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.511075  3325 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:27.625735  1614 rpcz_store.cc:276] 0709 13:00:24.926175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.607571 (+2681396us) service_pool.cc:224] Handling call
0709 13:00:27.624274 (+ 16703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.624527 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:27.624570 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:27.624641 (+    71us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:27.624687 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.624762 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.624919 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.593502  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 364) took 1897 ms. Trace:
I20260709 13:00:27.591337  1612 rpcz_store.cc:276] 0709 13:00:26.291809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.764754 (+472945us) service_pool.cc:224] Handling call
0709 13:00:27.095966 (+331212us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:27.435573 (+339607us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6950144}
I20260709 13:00:27.600850  3421 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:27.604712  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:27.612268  3260 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:00:27.637825  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 152051us
I20260709 13:00:27.638773  1536 rpcz_store.cc:276] 0709 13:00:24.722589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.445764 (+1723175us) service_pool.cc:224] Handling call
0709 13:00:26.446042 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.734642 (+288600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.519632 (+784990us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54001e4454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:27.580743 (+ 61111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551548800}
I20260709 13:00:27.636694  1617 rpcz_store.cc:276] 0709 13:00:23.507382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.455870 (+3948488us) service_pool.cc:224] Handling call
0709 13:00:27.456884 (+  1014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.517518 (+ 60634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.517837 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.514863  3720 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.114s
I20260709 13:00:27.611644  3588 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:27.647548  1657 leader_election.cc:400] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5.
I20260709 13:00:27.687624  1657 leader_election.cc:403] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:00:27.687973  3588 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.688838  3719 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5.
I20260709 13:00:27.647403  1616 rpcz_store.cc:276] 0709 13:00:24.011163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.575789 (+3564626us) service_pool.cc:224] Handling call
0709 13:00:27.592383 (+ 16594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.646747 (+ 54364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.646966 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.689481  1657 leader_election.cc:400] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5.
W20260709 13:00:27.690830  3732 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
W20260709 13:00:27.653249  3555 consensus_meta.cc:287] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.763s	user 0.006s	sys 0.006s
I20260709 13:00:27.621450  3400 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.621986  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:27.632305  3715 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:27.659351  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 70234.3us
I20260709 13:00:27.674311  1540 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:27.646763  3721 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:27.657867  3717 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:27.669261  3667 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:27.680572  1686 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:27.231662  2231 rpcz_store.cc:276] 0709 13:00:24.955405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.670590 (+1715185us) service_pool.cc:224] Handling call
0709 13:00:26.671013 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:26.671433 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:26.671469 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:26.690355 (+ 18886us) log.cc:844] Serialized 26 byte log entry
0709 13:00:26.754283 (+ 63928us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:26.761345 (+  7062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:26.769479 (+  8134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:27.095406 (+325927us) raft_consensus.cc:1685] finished
0709 13:00:27.095433 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:27.095593 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2390,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":178,"thread_start_us":59693,"threads_started":1,"wal-append.queue_time_us":178133}
W20260709 13:00:27.691845  3736 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:00:27.702634  3667 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:27.706825  3667 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:27.708072  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:27.708442  3667 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.708984  2230 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:27.710000  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:27.711489  2228 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:27.712556  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:27.713392  2228 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:27.714738  2228 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:27.684973  3260 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:27.683777  1536 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:27.674638  3594 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.684274  3421 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:27.721048  3421 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:27.722050  3421 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.723462  3421 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.727696  1536 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:27.729097  1536 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:27.729592  3546 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.679432  1535 rpcz_store.cc:276] 0709 13:00:25.693006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.297786 (+1604780us) service_pool.cc:224] Handling call
0709 13:00:27.298175 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.514947 (+216772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.589853 (+ 74906us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.730737  3546 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.733309  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:27.735390  3651 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:27.736161  3600 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:27.701404  3715 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:27.718802  3729 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.222s	user 0.001s	sys 0.000s
I20260709 13:00:27.739090  3568 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:27.740095  3568 consensus_queue.cc:1243] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:00:27.676928  3516 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.741645  3516 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.741945  3325 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:27.742499  3325 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:27.742861  3325 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:27.743625  3325 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.744719  3260 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:27.675361  3580 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:27.746191  3580 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.747198  3580 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:27.749791  1609 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:27.750739  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1708) took 6908 ms. Trace:
I20260709 13:00:27.751186  1612 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:27.751871  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 328) took 1905 ms. Trace:
I20260709 13:00:27.752211  1612 rpcz_store.cc:276] 0709 13:00:25.846383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.692105 (+1845722us) service_pool.cc:224] Handling call
0709 13:00:27.718983 (+ 26878us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.751608 (+ 32625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.751814 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.753002  3600 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:27.754684  3600 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.756321  3710 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:27.701838  3709 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:00:27.758750  3709 raft_consensus.cc:3055] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:27.759140  3709 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:27.759903  3709 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.730119  1611 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:27.738145  3300 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:27.698069  3555 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:27.762403  3162 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:27.695078  1614 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:27.785928  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 315) took 2462 ms. Trace:
I20260709 13:00:27.786325  1614 rpcz_store.cc:276] 0709 13:00:25.324042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.680789 (+2356747us) service_pool.cc:224] Handling call
0709 13:00:27.694816 (+ 14027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.785589 (+ 90773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.785855 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.683264  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:27.788482  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 105299us
I20260709 13:00:27.790153  1508 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:27.791675  1508 leader_election.cc:400] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4.
I20260709 13:00:27.791963  1508 leader_election.cc:403] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:27.736903  3651 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:27.699425  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:27.777560  1583 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:27.795876  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 96551.4us
I20260709 13:00:27.796276  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:27.779783  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:27.784653  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:27.783130  3726 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.507s	user 0.005s	sys 0.005s
I20260709 13:00:27.703596  3720 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:00:27.812259  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:27.820878  1584 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.830s (SENT) [suppressed 61 similar messages]
I20260709 13:00:27.798440  3537 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:27.866331  1536 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:27.704663  3717 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:27.802445  3555 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:27.870297  1536 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 2. Current term is 3. Ops: []
W20260709 13:00:27.872107  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 71332.6us
W20260709 13:00:27.872284  2234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 561) took 4172 ms (client timeout 2395 ms). Trace:
I20260709 13:00:27.802064  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 316) took 2477 ms. Trace:
I20260709 13:00:27.873147  1536 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:27.809477  3710 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.806805  3727 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:27.872751  2234 rpcz_store.cc:269] 0709 13:00:23.699448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:24.936592 (+1237144us) service_pool.cc:224] Handling call
0709 13:00:27.872039 (+2935447us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b5400530054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:27.872181 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2204199168}
I20260709 13:00:27.874254  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:27.751201  1609 rpcz_store.cc:276] 0709 13:00:20.842054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.749064 (+6907010us) service_pool.cc:224] Handling call
0709 13:00:27.749523 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.750401 (+   878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.750667 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.806301  3739 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:27.812999  3619 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:27.932161  1536 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:27.933240  1614 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:27.934429  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:27.873046  3709 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:27.852134  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:27.948258  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:27.950954  1536 raft_consensus.cc:2410] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:00:27.932969  3308 consensus_meta.cc:287] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.655s	user 0.009s	sys 0.003s
I20260709 13:00:27.953128  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:27.954435  1541 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:27.958633  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:27.811236  3726 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.964346  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:27.964535  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 457275us
I20260709 13:00:27.934237  3739 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.971060  3739 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:27.968647  3740 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:00:27.969832  1612 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:27.974319  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:27.975157  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 375) took 1204 ms. Trace:
I20260709 13:00:27.975734  1612 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:27.976409  2230 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:27.975487  1684 rpcz_store.cc:276] 0709 13:00:26.770698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.775356 (+  4658us) service_pool.cc:224] Handling call
0709 13:00:27.973747 (+1198391us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b5800010e98. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:27.975102 (+  1355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71335168}
I20260709 13:00:27.983484  1690 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:27.970163  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:27.871843  3676 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:27.918784  3721 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:00:27.949956  3300 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:27.954553  3670 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:27.990604  1536 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:27.883112  1611 rpcz_store.cc:276] 0709 13:00:25.324382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.685452 (+2361070us) service_pool.cc:224] Handling call
0709 13:00:27.729791 (+ 44339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.780989 (+ 51198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.798761 (+ 17772us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:27.961779  3308 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:00:28.010053  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:28.010620  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.010741  1508 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:28.007853  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 759) took 3204 ms. Trace:
I20260709 13:00:28.014926  2236 rpcz_store.cc:276] 0709 13:00:24.803034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.256362 (+1453328us) service_pool.cc:224] Handling call
0709 13:00:26.275137 (+ 18775us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.281436 (+  6299us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.007631 (+1726195us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b5400630e50. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:28.007788 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24533376}
I20260709 13:00:28.008987  1616 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:28.017565  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 322992us
I20260709 13:00:27.991204  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:27.993815  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:27.971364  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:27.986524  3698 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.757s	user 0.012s	sys 0.000s
I20260709 13:00:28.023655  1536 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:28.024928  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:28.025976  1536 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:00:28.029515  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 58273.6us
I20260709 13:00:28.030856  1658 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:28.001336  3414 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.305s	user 0.008s	sys 0.006s
I20260709 13:00:27.961328  3704 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.041640  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:28.044121  1658 leader_election.cc:336] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.016s (SENT)
I20260709 13:00:28.045337  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:28.048951  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.035495  3594 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:28.055701  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:28.058259  1658 leader_election.cc:336] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.929s (SENT)
I20260709 13:00:28.046268  3720 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:28.084837  3720 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:27.996367  1612 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:00:28.045606  3230 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:27.994879  3300 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.048049  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2180) took 2143 ms. Trace:
I20260709 13:00:28.053241  3739 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.061046  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:28.062028  3748 log.cc:927] Time spent T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:00:28.089840  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:28.090049  3749 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.090413  3739 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:28.087965  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:28.090605  3749 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:28.091100  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:28.091665  1610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1731) took 4397 ms (client timeout 3241 ms). Trace:
I20260709 13:00:28.092311  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:28.063410  3635 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:28.070178  3676 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:28.070479  3349 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:28.071698  3599 consensus_meta.cc:287] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.771s	user 0.009s	sys 0.000s
I20260709 13:00:28.049039  3719 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:00:28.072386  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 781) took 1754 ms. Trace:
I20260709 13:00:28.072702  3555 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:28.073369  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:28.075600  3516 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.078471  3667 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:28.035240  3698 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.081782  3715 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.082768  3726 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:28.083369  3704 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:28.007534  3260 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.050319  3580 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:00:28.092465  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:28.093590  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1703) took 1004 ms. Trace:
I20260709 13:00:28.094939  1508 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:28.068073  3696 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.872s	user 0.007s	sys 0.002s
I20260709 13:00:28.127604  1537 rpcz_store.cc:276] 0709 13:00:27.088983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.743177 (+654194us) service_pool.cc:224] Handling call
0709 13:00:27.743596 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:27.744068 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:27.744108 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:27.744681 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 13:00:27.794432 (+ 49751us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:27.794487 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:27.794584 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:28.093325 (+298741us) raft_consensus.cc:1685] finished
0709 13:00:28.093345 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.093531 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12980,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"thread_start_us":49280,"threads_started":1,"wal-append.queue_time_us":190353}
I20260709 13:00:28.092801  3749 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:28.085276  3720 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.088609  1539 rpcz_store.cc:276] 0709 13:00:25.905098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.603203 (+1698105us) service_pool.cc:224] Handling call
0709 13:00:27.605278 (+  2075us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:27.611605 (+  6327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:27.611638 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:27.612216 (+   578us) log.cc:844] Serialized 52 byte log entry
0709 13:00:27.673783 (+ 61567us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:28.035147 (+361364us) log.cc:844] Serialized 14 byte log entry
0709 13:00:28.035813 (+   666us) log.cc:844] Serialized 14 byte log entry
0709 13:00:28.047598 (+ 11785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.047702 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:28.047762 (+    60us) raft_consensus.cc:1685] finished
0709 13:00:28.047780 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.047976 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17704,"raft.queue_time_us":10663,"raft.run_cpu_time_us":436,"raft.run_wall_time_us":19111,"spinlock_wait_cycles":1908992,"thread_start_us":407282,"threads_started":2,"wal-append.queue_time_us":166707}
W20260709 13:00:28.097680  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 276822us
I20260709 13:00:28.098887  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:28.045568  3162 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:00:28.053752  3414 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:28.146885  1438 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.007s	sys 0.014s
W20260709 13:00:28.147229  1438 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.007s	sys 0.014s
I20260709 13:00:28.131975  3696 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:28.148380  3414 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:28.096052  3635 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:28.149602  3635 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.150559  3635 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.100611  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:28.004012  3700 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.154335  3716 raft_consensus.cc:3055] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
W20260709 13:00:28.106467  1686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1208) took 1925 ms (client timeout 1530 ms). Trace:
I20260709 13:00:28.154742  3716 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:00:28.155018  1686 rpcz_store.cc:269] 0709 13:00:26.180801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.180941 (+   140us) service_pool.cc:224] Handling call
0709 13:00:26.434879 (+253938us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b5800010e98. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:28.106192 (+1671313us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540003f054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:28.106379 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2812098176}
I20260709 13:00:28.107666  1691 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:28.155879  3700 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:28.155618  3716 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.156569  1612 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.156905  3716 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:28.092105  1610 rpcz_store.cc:269] 0709 13:00:23.693953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.690610 (+2996657us) service_pool.cc:224] Handling call
0709 13:00:28.091429 (+1400819us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b540008cad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:28.091591 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":983379328}
I20260709 13:00:28.158188  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:28.109889  3710 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4.
I20260709 13:00:28.159476  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.160744  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.110878  3519 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:28.163136  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:28.164288  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:28.164677  1612 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:28.165015  1618 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:28.165685  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.111537  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:28.120352  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2168) took 4595 ms. Trace:
I20260709 13:00:28.166316  1612 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
W20260709 13:00:28.121299  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 62015.7us
I20260709 13:00:28.167217  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:28.167290  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.168174  2236 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:28.166564  1538 rpcz_store.cc:276] 0709 13:00:23.525437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.439919 (+2914482us) service_pool.cc:224] Handling call
0709 13:00:26.440321 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.440586 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:26.441319 (+   733us) log.cc:844] Serialized 14 byte log entry
0709 13:00:28.005454 (+1564135us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b1800384be0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a 00007fe08faf6a2a
0709 13:00:28.120005 (+114551us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:28.120053 (+    48us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:28.120110 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.120180 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.120297 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":402,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":108,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":2880,"spinlock_wait_cycles":3682688,"thread_start_us":114289,"threads_started":1,"wal-append.queue_time_us":1712407}
I20260709 13:00:28.111537  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:28.124331  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 82813.1us
W20260709 13:00:28.131069  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.132959  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:28.136803  1584 leader_election.cc:336] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.773s (SENT)
I20260709 13:00:28.138022  2231 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:28.140653  3698 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:28.141141  1535 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:28.168709  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.141072  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:28.142343  3516 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:28.118659  2232 rpcz_store.cc:276] 0709 13:00:26.324506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.734054 (+409548us) service_pool.cc:224] Handling call
0709 13:00:26.771262 (+ 37208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.168209 (+396947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.072122 (+903913us) spinlock_profiling.cc:243] Waited 91.5 ms on lock 0x7b54002a3fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:28.072322 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":210490624}
I20260709 13:00:28.143446  3260 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:28.119611  3599 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:28.239578  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:28.240403  2232 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:28.240566  3599 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:28.146287  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:28.241536  1543 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:28.238066  3519 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:28.242437  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 568) took 3402 ms. Trace:
I20260709 13:00:28.242560  3519 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.243149  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 564) took 3468 ms. Trace:
I20260709 13:00:28.243605  2229 rpcz_store.cc:276] 0709 13:00:24.774378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.180468 (+1406090us) service_pool.cc:224] Handling call
0709 13:00:26.181726 (+  1258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:26.386740 (+205014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.242906 (+1856166us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b5400730fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:28.243093 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248703744}
I20260709 13:00:28.245208  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 586) took 1479 ms. Trace:
I20260709 13:00:28.245491  3700 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:28.245872  2235 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:00:28.245996  3700 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:28.246475  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.247152  1610 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:28.246929  3700 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:28.248055  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 468) took 3350 ms. Trace:
I20260709 13:00:28.248153  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.248771  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2174) took 4542 ms. Trace:
I20260709 13:00:28.249172  1534 rpcz_store.cc:276] 0709 13:00:23.706674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.481143 (+2774469us) service_pool.cc:224] Handling call
0709 13:00:26.648782 (+167639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.426516 (+777734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:27.426585 (+    69us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:27.426659 (+    74us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:27.426703 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.248546 (+821843us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54001e5854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:28.248594 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.248715 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27213,"raft.queue_time_us":135167,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":53224,"spinlock_wait_cycles":1447945984}
I20260709 13:00:28.250661  1535 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:53670
W20260709 13:00:28.252758  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 102090us
I20260709 13:00:28.242811  2233 rpcz_store.cc:276] 0709 13:00:24.839998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.407623 (+1567625us) service_pool.cc:224] Handling call
0709 13:00:26.407952 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.196057 (+788105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.242201 (+1046144us) spinlock_profiling.cc:243] Waited 787 ms on lock 0x7b5400560c54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:28.242378 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1810537344}
I20260709 13:00:28.173789  1617 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:28.194994  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:28.254303  2233 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:28.254837  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 589) took 1179 ms. Trace:
I20260709 13:00:28.255151  2233 rpcz_store.cc:276] 0709 13:00:27.075872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.253728 (+1177856us) service_pool.cc:224] Handling call
0709 13:00:28.254008 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:28.254634 (+   626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.254788 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:28.248471  2235 rpcz_store.cc:276] 0709 13:00:24.897851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.484962 (+1587111us) service_pool.cc:224] Handling call
0709 13:00:28.245639 (+1760677us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:28.246315 (+   676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.247849 (+  1534us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b5400730fd0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:28.247994 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4048323328}
I20260709 13:00:28.170176  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.256544  1618 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:28.151458  3726 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:28.151044  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:28.153687  3595 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:28.153977  1683 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:28.258200  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:28.260494  1612 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:28.261503  3595 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:28.261324  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:28.262439  3595 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.262953  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.263118  3162 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:28.263464  3595 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:28.234860  1539 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:28.234984  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:28.169075  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:28.251353  1534 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:28.251333  1535 raft_consensus.cc:606] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
W20260709 13:00:28.202975  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.346202  1535 raft_consensus.cc:614] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:28.340482  1539 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:28.242660  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1700) took 1469 ms. Trace:
I20260709 13:00:28.348975  1543 rpcz_store.cc:276] 0709 13:00:26.773078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.678251 (+905173us) service_pool.cc:224] Handling call
0709 13:00:27.678491 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:28.241927 (+563436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.242474 (+   547us) spinlock_profiling.cc:243] Waited 563 ms on lock 0x7b5400411c54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:28.242610 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1294430592}
W20260709 13:00:28.337014  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 100310us
I20260709 13:00:28.352155  1507 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:28.241344  1609 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.357173  1584 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:00:28.257797  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 109668us
I20260709 13:00:28.364835  1582 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:28.249377  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.280535  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 334) took 1983 ms. Trace:
I20260709 13:00:28.264729  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:28.392659  1618 raft_consensus.cc:2376] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5. Re-sending same reply.
I20260709 13:00:28.394402  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.395440  1618 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
W20260709 13:00:28.272203  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:28.400348  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.402717  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:28.396978  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.403697  1539 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.283067  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:28.404461  1618 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:28.405933  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:28.406381  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 123289us
I20260709 13:00:28.408242  1618 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:00:28.408608  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:28.367511  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.411160  1618 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:28.412050  1611 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:28.412176  3414 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:28.412391  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 325) took 2723 ms. Trace:
I20260709 13:00:28.413537  1617 rpcz_store.cc:276] 0709 13:00:25.688590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.683175 (+1994585us) service_pool.cc:224] Handling call
0709 13:00:27.683449 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:28.253789 (+570340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.411889 (+158100us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b5400104354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:28.412227 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1127043328}
I20260709 13:00:28.237833  1613 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:28.170903  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 382) took 1322 ms. Trace:
I20260709 13:00:28.272326  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:28.417117  1691 rpcz_store.cc:276] 0709 13:00:26.848105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.849081 (+   976us) service_pool.cc:224] Handling call
0709 13:00:28.107452 (+1258371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:28.156165 (+ 48713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.170668 (+ 14503us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540003f050. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:28.170834 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2893237120}
W20260709 13:00:28.299875  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.419440  1539 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:28.420969  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1696) took 3027 ms. Trace:
I20260709 13:00:28.421530  1541 rpcz_store.cc:276] 0709 13:00:25.393875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.226595 (+1832720us) service_pool.cc:224] Handling call
0709 13:00:27.226890 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.955100 (+728210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.420330 (+465230us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b54003e2954. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:28.420725 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1672732032}
I20260709 13:00:28.370194  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1738) took 3443 ms. Trace:
I20260709 13:00:28.423182  1615 rpcz_store.cc:276] 0709 13:00:24.926423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.656776 (+2730353us) service_pool.cc:224] Handling call
0709 13:00:27.657135 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:27.657528 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:27.657557 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:28.276056 (+618499us) spinlock_profiling.cc:243] Waited 7.32 ms on lock 0x7b18002b4ec0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9cd2a6 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a
0709 13:00:28.276471 (+   415us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b2c0005b3b8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:00:28.276920 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:00:28.369171 (+ 92251us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:28.369289 (+   118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.369519 (+   230us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:28.369585 (+    66us) raft_consensus.cc:1685] finished
0709 13:00:28.369609 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.369934 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":564,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":10581,"spinlock_wait_cycles":19509504,"thread_start_us":91509,"threads_started":1,"wal-append.queue_time_us":30444}
W20260709 13:00:28.301535  2228 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.023s	sys 0.023s
I20260709 13:00:28.271275  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 588) took 1380 ms. Trace:
W20260709 13:00:28.426092  2228 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.023s	sys 0.023s
I20260709 13:00:28.241326  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:28.440311  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:28.371815  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 590) took 1278 ms. Trace:
I20260709 13:00:28.284314  2234 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:00:28.441641  2233 rpcz_store.cc:276] 0709 13:00:27.106759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.256058 (+1149299us) service_pool.cc:224] Handling call
0709 13:00:28.256374 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:28.256742 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:28.256771 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:28.260357 (+  3586us) log.cc:844] Serialized 26 byte log entry
0709 13:00:28.270425 (+ 10068us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:28.270476 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.270572 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:28.371003 (+100431us) raft_consensus.cc:1685] finished
0709 13:00:28.371056 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.371480 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1142,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":34304,"thread_start_us":9254,"threads_started":1,"wal-append.queue_time_us":44573}
I20260709 13:00:28.444258  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:28.444658  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:28.256675  2235 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:28.168955  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:28.446027  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:28.446959  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:28.448510  2236 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:28.277510  3421 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:28.302393  3516 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.274739  3619 consensus_queue.cc:1059] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:28.275377  3625 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:28.452428  3516 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.266409  3635 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:28.268750  3676 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:00:28.456300  3516 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:28.252614  3519 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:28.266904  3698 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:00:28.235113  3696 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:28.248172  3700 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.459754  3698 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:28.460484  3698 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:28.285179  3716 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:28.461671  1538 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:28.306890  3720 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.446960  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:28.263897  3725 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:00:28.466097  3720 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:28.275971  3739 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:28.467722  3568 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:28.468519  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 60101.1us
I20260709 13:00:28.380223  3726 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:28.384896  3670 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:28.391993  1616 rpcz_store.cc:276] 0709 13:00:26.297535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.689776 (+1392241us) service_pool.cc:224] Handling call
0709 13:00:27.690101 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:28.016416 (+326315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.280175 (+263759us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54002dc3d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:28.280437 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":732863744}
I20260709 13:00:28.469642  3670 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:28.141352  3745 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:28.470816  3745 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:28.387172  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:28.264151  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:28.476438  1612 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:28.415217  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2010) took 1994 ms. Trace:
I20260709 13:00:28.477381  1683 rpcz_store.cc:276] 0709 13:00:26.420278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:26.631195 (+210917us) service_pool.cc:224] Handling call
0709 13:00:26.631529 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:28.259569 (+1628040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.414562 (+154993us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b5400375854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:28.414915 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3494343040}
I20260709 13:00:28.412938  1618 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:28.477974  3716 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:00:28.418251  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 145881us
I20260709 13:00:28.480156  2201 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:28.479923  3260 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:28.422641  3303 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.482002  3303 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.483919  2236 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:28.485854  1609 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
W20260709 13:00:28.487587  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.426157  2229 rpcz_store.cc:276] 0709 13:00:26.888100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.244609 (+1356509us) service_pool.cc:224] Handling call
0709 13:00:28.245030 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:28.245251 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:28.245290 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:28.245338 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:28.245383 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.264926 (+ 19543us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.268323 (+  3397us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:28.472380  3745 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:28.491923  3300 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:28.493482  3745 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:28.497615  3749 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:28.498240  3749 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:28.442378  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 78009.9us
I20260709 13:00:28.500766  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:28.468712  3739 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:28.498728  3749 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.892340  3739 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:28.894433  3751 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:28.893890  3739 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:28.521634  1509 leader_election.cc:336] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.599s (SENT)
W20260709 13:00:28.469172  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:28.943898  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.526278  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:28.524842  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:28.947809  1689 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:28.948406  1690 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:28.949460  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:28.523717  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:28.951534  2203 leader_election.cc:336] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.122s (SENT)
I20260709 13:00:28.951759  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:28.951921  1691 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:28.952135  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:28.570770  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:28.522228  1683 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:28.959367  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:28.526351  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:28.962595  1684 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:28.527747  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:28.964792  1685 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:28.574893  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:28.588709  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.603305  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:28.616614  3738 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.000s	sys 0.017s
W20260709 13:00:28.969877  3738 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.200s	user 0.001s	sys 0.017s
I20260709 13:00:28.970645  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:28.971994  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:28.972699  1538 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.245682  2230 rpcz_store.cc:276] 0709 13:00:26.765629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.709696 (+944067us) service_pool.cc:224] Handling call
0709 13:00:28.245030 (+535334us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54005b0254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:28.245149 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":611143424}
I20260709 13:00:28.503015  2232 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:28.979393  2230 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:28.599938  2234 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:28.478077  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 587) took 1696 ms. Trace:
I20260709 13:00:28.623749  1539 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:00:28.578689  2228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 578) took 1831 ms. Trace:
I20260709 13:00:28.504367  3303 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:28.528829  3260 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:28.541519  3325 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.548346  3588 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:28.380789  3599 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:00:28.470248  3670 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:28.578735  3680 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:28.501681  3698 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:28.516266  3709 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:28.508780  3717 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:28.234387  3715 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:28.995523  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:28.571982  3720 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:00:28.657099  1541 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:29.027567  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:29.028410  1541 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:28.996176  3519 consensus_queue.cc:1059] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 13:00:28.674842  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 370) took 2980 ms. Trace:
I20260709 13:00:29.029291  3519 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:28.730324  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:28.737540  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:28.739915  3600 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:28.771530  2236 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:29.029599  1540 rpcz_store.cc:276] 0709 13:00:25.694512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.454750 (+1760238us) service_pool.cc:224] Handling call
0709 13:00:27.455060 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.698647 (+243587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.674260 (+975613us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b540030bbd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:28.674663 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503775488}
I20260709 13:00:28.796861  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:28.801182  3300 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:28.814143  3727 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:28.819084  3725 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:00:28.846779  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:28.859305  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:28.502048  1535 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:00:28.895437  3726 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:28.895627  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:28.892838  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 392196us
I20260709 13:00:28.896826  3739 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.898664  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:28.903910  1688 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:00:28.912459  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:28.912958  3749 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:28.922282  3595 consensus_meta.cc:287] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.658s	user 0.005s	sys 0.017s
I20260709 13:00:29.037653  1542 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:29.043120  3727 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.043677  1686 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.044104  3749 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:28.925392  3764 raft_consensus.cc:3055] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:28.934355  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1745) took 1136 ms. Trace:
I20260709 13:00:29.044852  3764 raft_consensus.cc:740] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:28.936101  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.045513  3727 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:29.046047  3600 raft_consensus.cc:3055] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:00:29.046473  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 110540us
I20260709 13:00:28.953248  3546 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:28.957553  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:29.047060  3764 consensus_queue.cc:260] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:29.048558  3764 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:29.048919  1543 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:28.459621  2233 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:29.053642  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 494) took 1164 ms. Trace:
I20260709 13:00:29.053989  2233 rpcz_store.cc:276] 0709 13:00:27.889231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.458089 (+568858us) service_pool.cc:224] Handling call
0709 13:00:28.459060 (+   971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:29.053343 (+594283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:29.053577 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:28.987056  3349 consensus_queue.cc:1059] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:29.039438  1537 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:29.055725  3349 raft_consensus.cc:3055] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:29.056108  3349 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:28.988711  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:28.990027  1657 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.041051  1534 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.057991  1685 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
W20260709 13:00:29.058460  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 68599.3us
W20260709 13:00:29.059126  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.059492  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:29.060379  1692 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:29.060843  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:29.017184  3162 consensus_queue.cc:1175] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cecad4f86a114a7696e398b3e8cc511e" responder_term: 4 status { last_received { term: 3 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:28.994580  2231 rpcz_store.cc:276] 0709 13:00:26.781500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.783097 (+1001597us) service_pool.cc:224] Handling call
0709 13:00:27.815178 (+ 32081us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:28.257526 (+442348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.477553 (+220027us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b540065bb54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:28.477884 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606987648}
I20260709 13:00:28.842556  1584 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.025686  1539 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:28.582675  3724 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:29.030833  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:29.103539  1541 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:29.031200  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:29.104578  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 73460.7us
I20260709 13:00:29.001822  1691 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:28.995344  2234 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:28.778074  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:29.034077  3346 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:29.035395  3725 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:29.014904  3709 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.035717  3519 raft_consensus.cc:3055] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:28.956256  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:29.049917  2237 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:29.049971  3720 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:29.000061  3260 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:29.051718  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:29.012305  3680 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:29.045064  1614 rpcz_store.cc:276] 0709 13:00:27.797928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.840684 (+ 42756us) service_pool.cc:224] Handling call
0709 13:00:28.929882 (+1089198us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b540008c5d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:28.934051 (+  4169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32251264}
I20260709 13:00:29.057643  3599 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:28.987810  2201 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:29.057032  3349 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.112396  3300 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:29.063913  1685 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:28.996249  2228 rpcz_store.cc:276] 0709 13:00:26.747208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:27.715744 (+968536us) service_pool.cc:224] Handling call
0709 13:00:27.716078 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:27.716301 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:28.233763 (+517462us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08c9e20ac 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fc455 00007fe0934e43c7
0709 13:00:28.429009 (+195246us) log.cc:844] Serialized 14 byte log entry
0709 13:00:28.575832 (+146823us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:28.578039 (+  2207us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:28.578169 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:28.578313 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:28.578621 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8827,"prepare.queue_time_us":590162,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":9112,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":273,"spinlock_wait_cycles":3248512,"thread_start_us":826582,"threads_started":2,"wal-append.queue_time_us":204342}
I20260709 13:00:29.057530  3516 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.146039  3516 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.146222  3725 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:29.039892  3726 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.147562  3725 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.046517  3600 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:29.040652  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 519257us
I20260709 13:00:29.067472  3670 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:29.067973  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.036394  1538 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
W20260709 13:00:29.069557  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:29.075017  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.155697  1686 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:29.039186  1535 pending_rounds.cc:85] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 5
I20260709 13:00:29.083022  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:29.157830  1692 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:00:29.088666  1540 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 1. Current term is 3. Ops: []
I20260709 13:00:29.093900  1687 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:29.158666  1690 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:00:29.097451  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 255015us
I20260709 13:00:29.098163  3715 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:29.098385  3698 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:29.094775  3727 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.007s
I20260709 13:00:29.118624  3519 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:29.109870  3346 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:29.134706  3546 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:29.137810  3702 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:29.141804  2203 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:29.142356  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:29.142678  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 154966us
I20260709 13:00:29.120457  1684 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:29.120760  3720 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:29.157004  1543 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:29.132779  3680 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:29.143671  3300 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:29.152760  3724 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:29.173873  3421 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.174808  2201 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:29.174929  3516 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:29.170944  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:29.175475  3349 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.159356  1535 pending_rounds.cc:107] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:00:29.179543  1689 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:29.170639  3739 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:29.171000  3720 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:29.214624  1584 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:29.214812  3260 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:29.216246  3670 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:29.211630  1535 raft_consensus.cc:2889] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:29.211324  3162 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 13:00:29.219791  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:29.176353  3300 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.224175  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:29.180541  3726 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:29.235702  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 167897us
I20260709 13:00:29.221032  1542 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:00:29.228951  1692 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
W20260709 13:00:29.241524  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 66875.4us
I20260709 13:00:29.243320  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:29.244644  2201 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:29.245608  3516 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:29.224856  3546 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:29.231356  3303 raft_consensus.cc:3055] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:29.224562  3689 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:29.252336  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 92100us
I20260709 13:00:29.233858  3739 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:29.222301  3421 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:29.249047  3546 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:29.257862  3727 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.268746  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.251010  3303 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:29.266208  3346 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:29.332235  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:29.286128  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 71630.3us
W20260709 13:00:29.273324  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 48898.5us
W20260709 13:00:29.294504  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:29.302286  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55815.6us
I20260709 13:00:29.247036  3600 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:29.285566  1583 leader_election.cc:336] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 3.011s (SENT)
W20260709 13:00:29.278249  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 136614us
I20260709 13:00:29.349846  1584 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:29.357520  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:29.360460  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.355068  1582 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:29.346125  1686 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:29.269222  3546 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.375895  1583 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.388042  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:00:29.352807  3793 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.091s	user 0.000s	sys 0.004s
I20260709 13:00:29.346316  3346 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.306380  1683 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:29.354694  3325 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:29.401676  1508 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.285491  3709 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:29.288901  3724 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:29.285756  3670 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:29.295262  3689 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:29.272717  3588 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:29.285869  3791 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.170s
I20260709 13:00:29.286505  3421 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:29.293025  1534 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:29.412389  3725 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.407377  3715 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:29.404942  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 134861us
W20260709 13:00:29.406121  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 51164.1us
I20260709 13:00:29.288849  3680 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.300652  3717 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:29.394845  3727 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.280129  3786 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.169s
I20260709 13:00:29.275285  3519 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.415745  3349 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:29.415851  1685 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:29.287253  3303 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.431137  3303 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:29.414553  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 65212.3us
I20260709 13:00:29.434029  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:29.404855  3764 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:29.436156  3421 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.437870  3749 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.438560  3749 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:29.441493  1509 leader_election.cc:336] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.794s (SENT)
I20260709 13:00:29.442211  1509 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:29.443748  1509 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.817s (SENT)
I20260709 13:00:29.444239  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:29.420238  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 70483.8us
I20260709 13:00:29.409821  3599 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.418529  3720 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.446235  3676 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:29.446658  3720 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:29.447587  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.422844  3670 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:29.401345  3325 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:29.411316  3689 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:29.460754  3689 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.461218  3802 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:29.412530  3588 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:29.465739  3588 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.468317  3764 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:29.469301  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 43967.6us
I20260709 13:00:29.414455  3797 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.470155  3797 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.470863  2201 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:29.471736  3800 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:29.472189  2201 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:29.472555  3797 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:29.472723  3749 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:29.424654  3717 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:29.459714  3720 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:29.289894  3739 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.448657  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.433784  3724 consensus_queue.cc:1059] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:29.473914  3717 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:29.489087  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 102137us
I20260709 13:00:29.485354  3303 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:29.485788  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:29.491362  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45480.5us
I20260709 13:00:29.447968  3349 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:29.501991  3349 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:29.491575  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:29.493961  1683 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:29.495190  3764 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:29.496312  3725 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:29.496631  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.458328  3726 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.494603  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.498348  3789 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.499153  3303 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:29.506858  1688 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
W20260709 13:00:29.527032  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 54936.2us
W20260709 13:00:29.529708  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.530750  1584 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:29.531520  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:29.532186  1683 raft_consensus.cc:2376] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4. Re-sending same reply.
I20260709 13:00:29.532689  3670 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.533553  1658 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.533847  3745 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:29.534778  3745 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.535416  3745 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:00:29.511376  3702 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:29.539695  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 96040.8us
I20260709 13:00:29.489552  3516 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:00:29.555302  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:29.547283  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 126057us
I20260709 13:00:29.556751  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:29.532186  3725 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:29.536217  3764 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:29.565119  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:29.539041  3727 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:29.571472  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:29.579373  1658 leader_election.cc:336] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.074s (SENT)
I20260709 13:00:29.579970  1658 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.557291  3162 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.584826  1541 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:29.592319  3805 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:29.592746  1684 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:29.592957  3805 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:29.594429  3715 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.594110  3805 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:29.589650  3162 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.595436  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:29.567996  3702 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.597098  3700 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.568617  3725 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:29.597726  3700 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:29.598318  3725 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:29.598850  3702 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:29.599627  3725 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.599768  3700 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:29.594164  3546 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.577157  3791 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:29.551478  3303 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:00:29.539211  3588 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.600675  1536 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:29.584896  3795 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:29.612730  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.594000  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:29.555529  3789 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:00:29.627460  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:29.574234  3346 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:00:29.564561  3676 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:00:29.590492  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:29.630522  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 91165.7us
I20260709 13:00:29.631310  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:29.589036  3800 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:29.582517  3599 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:00:29.602173  3162 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:29.634352  1691 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:29.623098  1683 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:29.611538  3698 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:29.636745  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:29.656502  1509 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.846s (SENT)
I20260709 13:00:29.626557  3795 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:29.663661  1509 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:29.650135  1472 debug-util.cc:398] Leaking SignalData structure 0x7b080003bf00 after lost signal to thread 434
I20260709 13:00:29.661630  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:29.578769  3325 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.666452  3325 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.666503  3800 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.668665  3809 consensus_queue.cc:1059] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:00:29.670063  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 139406us
I20260709 13:00:29.661653  2201 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:29.670548  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:29.672287  1690 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:00:29.675896  1472 debug-util.cc:398] Leaking SignalData structure 0x7b08000351a0 after lost signal to thread 1475
I20260709 13:00:29.654072  3680 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:29.632690  3346 consensus_queue.cc:1243] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:00:29.680769  1472 debug-util.cc:398] Leaking SignalData structure 0x7b08003ec2c0 after lost signal to thread 1567
W20260709 13:00:29.687341  1472 debug-util.cc:398] Leaking SignalData structure 0x7b08003ec800 after lost signal to thread 1568
W20260709 13:00:29.688597  1472 debug-util.cc:398] Leaking SignalData structure 0x7b0800381f20 after lost signal to thread 1642
W20260709 13:00:29.689924  1472 debug-util.cc:398] Leaking SignalData structure 0x7b080037e140 after lost signal to thread 1716
W20260709 13:00:29.690341  1472 debug-util.cc:398] Leaking SignalData structure 0x7b0800269e00 after lost signal to thread 1717
I20260709 13:00:29.690547  1688 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:00:29.691591  1472 debug-util.cc:398] Leaking SignalData structure 0x7b080019d3c0 after lost signal to thread 2261
W20260709 13:00:29.691927  1472 debug-util.cc:398] Leaking SignalData structure 0x7b0800361d80 after lost signal to thread 2262
W20260709 13:00:29.692183  1472 debug-util.cc:398] Leaking SignalData structure 0x7b080025d2e0 after lost signal to thread 3189
W20260709 13:00:29.692443  1472 debug-util.cc:398] Leaking SignalData structure 0x7b08000c8960 after lost signal to thread 3308
W20260709 13:00:29.692761  1472 debug-util.cc:398] Leaking SignalData structure 0x7b08001c5460 after lost signal to thread 3537
I20260709 13:00:29.635092  3726 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:00:29.634529  3300 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.678894  1685 raft_consensus.cc:2410] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:29.663250  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:29.694034  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 114698us
I20260709 13:00:29.699297  3698 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.670357  3745 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.707459  1584 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.710983  3588 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:29.711849  3807 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:29.662300  3764 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.693594  2228 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:00:29.719094  3812 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:29.719626  3300 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:29.708628  3689 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.720489  3546 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:29.721180  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 79675.9us
I20260709 13:00:29.721618  3349 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:29.699153  1472 debug-util.cc:398] Leaking SignalData structure 0x7b08003bd340 after lost signal to thread 3555
I20260709 13:00:29.729058  1438 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40172 (request call id 229) took 10251 ms. Trace:
I20260709 13:00:29.659534  3162 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:29.729736  3804 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:29.771809  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:29.730891  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.730998  1437 catalog_manager.cc:5697] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:29.771117  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:29.783362  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:29.757494  3676 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:00:29.807518  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 100183us
I20260709 13:00:29.767973  3812 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:29.809077  2235 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:29.775369  1688 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:29.808346  3303 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:29.808485  3349 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:29.805258  1536 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
W20260709 13:00:29.804533  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 151068us
I20260709 13:00:29.773547  3807 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:29.823625  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 247870us
W20260709 13:00:29.820019  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 192325us
I20260709 13:00:29.823249  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:29.767005  3599 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:29.759701  3698 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:29.769338  3162 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.819978  3600 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:29.774638  3745 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.811628  3804 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:29.847468  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:29.822750  3805 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:29.809510  3812 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.826834  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:29.875758  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.769038  1438 rpcz_store.cc:276] 0709 13:00:19.477972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:19.478144 (+   172us) service_pool.cc:224] Handling call
0709 13:00:19.614739 (+136595us) spinlock_profiling.cc:243] Waited 68.7 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:00:29.728818 (+10114079us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:27.684423 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:27.687144 (+  2721us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:27.687182 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:00:27.687202 (+    20us) tablet.cc:662] Decoding operations
0709 13:00:27.688203 (+  1001us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:27.688289 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:27.688314 (+    25us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:00:27.767751 (+ 79437us) tablet.cc:701] Row locks acquired
0709 13:00:27.767768 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:00:27.767953 (+   185us) write_op.cc:270] Start()
0709 13:00:27.768489 (+   536us) write_op.cc:276] Timestamp: P: 1783602027767898 usec, L: 0
0709 13:00:27.768506 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:00:27.769185 (+   679us) log.cc:844] Serialized 3601 byte log entry
0709 13:00:28.432574 (+663389us) spinlock_profiling.cc:243] Waited 9 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08c9e20ac 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b
0709 13:00:28.982311 (+549737us) op_driver.cc:464] REPLICATION: finished
0709 13:00:29.085793 (+103482us) write_op.cc:301] APPLY: starting
0709 13:00:29.085985 (+   192us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:29.086376 (+   391us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:29.086432 (+    56us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:29.088493 (+  2061us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:29.089039 (+   546us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:00:29.089112 (+    73us) write_op.cc:312] APPLY: finished
0709 13:00:29.089750 (+   638us) log.cc:844] Serialized 71 byte log entry
0709 13:00:29.107572 (+ 17822us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:29.118114 (+ 10542us) write_op.cc:454] Released schema lock
0709 13:00:29.118193 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:29.118768 (+   575us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:29.118795 (+    27us) write_op.cc:454] Released schema lock
0709 13:00:29.166014 (+ 47219us) spinlock_profiling.cc:243] Waited 321 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fae932a 00007fe08fa6ef14 00007fe08fa6eec3 00007fe08fa6ee38 00007fe08fa6edf3 00007fe08fa6e0dc 00007fe08fa97e16 00007fe08fa7fcbb 00007fe08fa6a39d 00007fe08fa6dd75
Metrics: {"mutex_wait_us":2058209,"reported_tablets":14,"spinlock_wait_cycles":157990144,"child_traces":[["op",{"apply.queue_time_us":103088,"apply.run_cpu_time_us":5086,"apply.run_wall_time_us":32991,"num_ops":9,"prepare.queue_time_us":171632,"prepare.run_cpu_time_us":23882,"prepare.run_wall_time_us":1294362,"raft.queue_time_us":8553,"raft.run_cpu_time_us":16567,"raft.run_wall_time_us":506973,"replication_time_us":1213573,"spinlock_wait_cycles":758303872,"thread_start_us":1947086,"threads_started":4,"wal-append.queue_time_us":860058}]]}
W20260709 13:00:29.876263  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.880182  1688 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:29.881172  3745 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:29.881412  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.883508  3820 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.884156  3820 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.886412  3820 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:29.886735  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:29.887043  3820 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:29.824888  3676 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:29.892002  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 52772.8us
W20260709 13:00:29.891088  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 114815us
I20260709 13:00:29.903110  1538 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:29.874274  3325 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:29.893265  1437 catalog_manager.cc:5697] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 2 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
W20260709 13:00:29.913865  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 90678us
I20260709 13:00:29.914268  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:29.914417  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:29.915053  1584 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:29.889842  3812 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:29.887502  3820 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:29.923013  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:29.927701  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.921044  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:29.930635  1688 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:29.931573  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 346972us
I20260709 13:00:29.875572  3805 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:00:29.936830  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:29.935943  3798 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.128s
I20260709 13:00:29.941018  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:29.943397  3804 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.943754  1437 catalog_manager.cc:5697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:29.945398  1437 catalog_manager.cc:5697] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 4, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 4 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:29.948774  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:29.952116  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.929996  3820 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:29.956112  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:29.921882  1535 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
W20260709 13:00:29.956452  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.968053  3635 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:29.969127  1537 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 13:00:29.988489  1691 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:29.972476  3795 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.990386  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:29.991853  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:29.992470  1690 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:30.040457  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 120707us
W20260709 13:00:30.047570  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.052729  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 60228.6us
W20260709 13:00:30.053552  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.056813  1614 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88cbab659ea041148748114fae549ae0"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:00:29.982205  3689 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:30.057667  1614 raft_consensus.cc:606] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:00:30.057904  3826 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.058104  1614 raft_consensus.cc:614] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:30.058463  3826 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:29.992954  3725 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.060111  3725 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:30.063709  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.063845  1690 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:30.069546  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:29.932660  3325 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:30.082486  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:29.982060  3635 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:30.067209  1437 catalog_manager.cc:5697] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 4, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 4 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.076098  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:30.120515  3725 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:30.122434  3819 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:30.123103  3819 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.123500  3819 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:30.120342  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.053812  3670 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:30.144914  1437 catalog_manager.cc:5697] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 4, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 4 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
W20260709 13:00:30.147562  3818 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:00:30.174337  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:30.175356  1539 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:30.186187  3831 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:30.189920  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:30.190723  1611 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:30.191048  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.191663  1613 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:30.192559  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:30.193807  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:30.195480  3725 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.196223  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:30.196072  3725 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:30.197093  2231 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:30.157924  3162 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:29.925235  3807 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.158828  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:30.160584  3325 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:30.163594  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.164206  1610 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:30.164367  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:30.164896  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.222863  1618 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:00:30.168653  3827 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 13:00:30.148810  3698 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:30.169590  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:30.169874  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:30.170540  3670 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:30.171182  1614 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:30.172367  1437 catalog_manager.cc:5697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
W20260709 13:00:30.200572  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.207978  1688 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
W20260709 13:00:30.209992  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 380371us
I20260709 13:00:30.165402  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.233196  1539 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:30.235134  1616 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 1.
I20260709 13:00:30.170563  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:30.243716  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 258800us
I20260709 13:00:30.249352  1618 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:30.181027  3819 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.254467  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:30.249114  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 332216us
I20260709 13:00:30.287551  1688 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:30.296099  1437 catalog_manager.cc:5697] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 3, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 3 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:30.303730  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:30.307623  1543 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 5: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:30.309074  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:30.294651  3725 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:30.321300  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:30.319205  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 334528us
I20260709 13:00:30.324590  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:30.325703  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:30.326388  1618 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:30.329576  3816 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.330209  3816 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.332698  3816 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:30.297111  3676 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:30.334699  3819 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:30.335284  3843 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:30.335677  3843 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:30.336464  3843 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.337678  3843 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:30.315747  3828 log.cc:927] Time spent T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 13:00:30.341797  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:30.343158  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:30.344362  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:30.345577  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.346364  1612 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:30.347198  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:30.347270  3519 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:30.351349  1542 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:00:30.351511  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.368775  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:30.313176  3804 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.371373  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:30.372267  1437 catalog_manager.cc:5697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 3 to 5, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 5 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:30.382490  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.397716  1614 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:30.398739  1618 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
W20260709 13:00:30.400597  1509 leader_election.cc:336] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.969s (SENT)
I20260709 13:00:30.404325  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:30.410530  2237 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:30.411026  1437 catalog_manager.cc:5697] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.415305  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:30.416769  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.370358  3676 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.425724  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.426595  1614 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
W20260709 13:00:30.421871  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 104133us
I20260709 13:00:30.433064  1613 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:30.439105  1616 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:30.436425  1437 catalog_manager.cc:5697] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 1 to 2, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 2 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:30.411099  3842 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.413834  3516 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:30.447180  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.419682  3804 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.422700  3789 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:30.479276  3804 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:30.473815  3833 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:30.482196  3833 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:30.482613  3826 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:30.484808  3833 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:30.488971  3812 consensus_meta.cc:287] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.567s	user 0.007s	sys 0.002s
I20260709 13:00:30.489145  3804 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:30.489889  1615 raft_consensus.cc:2410] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:30.493013  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:30.493862  2237 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
W20260709 13:00:30.497237  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.497774  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.489885  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:30.499868  1618 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:00:30.498549  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 262713us
I20260709 13:00:30.501683  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:30.502257  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.505934  1613 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:30.472263  1437 catalog_manager.cc:5697] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:00:30.502853  1582 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:30.512639  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:30.513229  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.513696  1539 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:00:30.515169  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 145563us
I20260709 13:00:30.516878  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:30.519407  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:30.520236  2234 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:30.471146  1611 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:30.547737  1618 raft_consensus.cc:2376] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4. Re-sending same reply.
I20260709 13:00:30.509723  3843 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:30.533179  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:30.556522  2233 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:30.537652  3833 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:30.538697  3635 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:30.539827  3826 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:30.442058  3676 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.532287  3588 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:30.490314  3804 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.557869  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:30.511267  3812 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:30.563807  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:30.556841  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:30.560448  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 57661.3us
I20260709 13:00:30.559852  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:30.492291  3842 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:30.564754  1613 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:30.566941  1509 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.997s (SENT)
I20260709 13:00:30.568571  1542 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:30.572865  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:30.559516  3843 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.569155  1582 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:30.574714  1509 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:30.575083  1658 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:30.555517  3789 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:30.572937  3727 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:30.583042  1541 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:30.575899  3698 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:30.576277  3843 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:30.572482  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.587664  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:30.584359  3804 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:30.564930  3516 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:00:30.584422  3802 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:30.598312  3739 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.010s	sys 0.002s
I20260709 13:00:30.588268  3819 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:30.586807  3764 raft_consensus.cc:2764] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:00:30.590519  3325 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:30.572727  3812 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.620113  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.576340  3826 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:30.611500  3739 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.010s	sys 0.002s
I20260709 13:00:30.608186  3162 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:30.634449  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46858.8us
W20260709 13:00:30.640688  3715 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.006s	sys 0.012s
I20260709 13:00:30.611166  3817 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:00:30.643352  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.599597  3789 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:30.654241  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 61482.3us
I20260709 13:00:30.655879  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:30.636744  3635 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:30.659797  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 96734.9us
I20260709 13:00:30.669518  2235 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:00:30.658842  3715 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.007s	sys 0.012s
I20260709 13:00:30.657527  3802 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:30.683602  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:30.685916  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:30.686998  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:30.688680  1582 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:30.689620  1509 leader_election.cc:336] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.028s (SENT)
I20260709 13:00:30.690243  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:30.690646  1582 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:30.659876  3749 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:00:30.674778  3805 consensus_queue.cc:1059] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.018s
W20260709 13:00:30.696396  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.697876  3850 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
W20260709 13:00:30.698262  1509 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.095s (SENT)
I20260709 13:00:30.698262  3841 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:30.699397  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:30.682040  3739 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:30.681308  1609 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:30.683090  3802 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.706611  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.707150  3833 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:30.707618  3802 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:30.709347  3300 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:30.709844  3300 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:30.710841  3300 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.712001  3300 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:30.712378  3805 raft_consensus.cc:3055] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:30.712956  3805 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:30.689656  3325 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:30.687842  3727 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.717172  3325 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:30.717643  3325 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:30.718557  3812 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:30.721066  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:30.721313  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:30.722106  1690 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:30.721750  1688 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:30.720387  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:30.716627  3805 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.682009  3807 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:00:30.693681  3764 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:30.727356  1687 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:30.695622  3846 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:30.730316  1509 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:30.689152  3670 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:00:30.701218  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:30.717649  3162 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:30.734620  2203 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:30.750020  3516 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:00:30.750555  3516 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:30.750973  3516 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:30.752228  3853 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:00:30.717345  3727 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.737735  3726 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:30.738396  3162 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:30.775709  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 49401.4us
W20260709 13:00:30.792315  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 57791.7us
I20260709 13:00:30.795783  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:30.799644  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:30.800134  2235 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:30.800288  2237 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:30.801105  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:30.801496  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:30.801576  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:30.802692  3670 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.802861  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:30.803103  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:30.803527  2230 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:30.803262  3670 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.804723  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:30.805507  2235 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:30.805750  3670 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:30.807286  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:30.729278  3698 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:00:30.801821  2232 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:30.812017  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:30.810791  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1730) took 1044 ms. Trace:
I20260709 13:00:30.813684  1538 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:00:30.818526  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 127930us
I20260709 13:00:30.814481  2237 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:30.808046  3804 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.814451  1540 rpcz_store.cc:276] 0709 13:00:29.766255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.766386 (+   131us) service_pool.cc:224] Handling call
0709 13:00:29.766733 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:29.804825 (+ 38092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:29.804861 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:29.814050 (+  9189us) log.cc:844] Serialized 26 byte log entry
0709 13:00:30.794251 (+980201us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b1800401c80. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08f6944bc 00007fe08c9e248b 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1
0709 13:00:30.794415 (+   164us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:30.794453 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:30.794524 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:30.794571 (+    47us) raft_consensus.cc:1685] finished
0709 13:00:30.794582 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:30.810712 (+ 16130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93829,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":4407,"spinlock_wait_cycles":2353536,"thread_start_us":21385,"threads_started":1,"wal-append.queue_time_us":88232}
I20260709 13:00:30.818310  3789 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:30.836737  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:30.837788  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 139590us
I20260709 13:00:30.808069  3162 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:30.809849  3805 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:30.836282  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:30.837121  3866 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:30.846271  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:30.835410  3826 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:30.817917  3802 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:30.857931  1538 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:30.858955  1582 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:30.847710  2236 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:30.859411  3635 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:30.861490  1582 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:30.865350  3635 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:30.862883  3866 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:30.865758  1614 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:30.865891  3635 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:30.866472  3635 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:30.867282  1614 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:00:30.867730  3815 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.943s	user 0.000s	sys 0.006s
I20260709 13:00:30.865559  3852 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
W20260709 13:00:30.874140  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:30.874094  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 79152.9us
I20260709 13:00:30.875476  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:30.875618  1509 leader_election.cc:336] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.100s (SENT)
I20260709 13:00:30.878383  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:30.878865  1690 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:30.867697  3866 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:30.849653  3804 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:30.894639  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:30.897747  1582 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:30.902200  3869 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:30.896275  3826 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:30.900213  3698 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:30.945781  3869 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:30.965564  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 95775.8us
I20260709 13:00:30.967962  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:30.971225  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:30.947465  3670 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 13:00:30.958182  3698 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:31.008015  1685 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 13:00:30.982105  1509 leader_election.cc:336] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.102s (SENT)
I20260709 13:00:31.009821  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:31.014068  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:31.016394  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:31.017452  1614 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:00:31.017926  1614 raft_consensus.cc:606] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:31.018260  1614 raft_consensus.cc:614] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:31.019477  3826 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:31.025529  3727 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:31.026320  3698 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:30.978793  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:31.041266  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 59233.6us
I20260709 13:00:31.035084  3698 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:31.029654  3727 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.044912  3727 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:31.046573  3843 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.047024  3727 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:31.047919  1541 raft_consensus.cc:2393] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:31.048801  3843 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:31.052431  1509 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.119s (SENT)
I20260709 13:00:31.053107  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:31.054123  3843 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:31.054798  1509 leader_election.cc:336] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.163s (SENT)
I20260709 13:00:31.055400  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:31.056389  3843 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:31.067035  3820 consensus_meta.cc:287] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.075s	user 0.012s	sys 0.000s
I20260709 13:00:31.067690  3820 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:31.083763  1509 leader_election.cc:336] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.202s (SENT)
I20260709 13:00:31.084448  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:31.093183  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 217824us
I20260709 13:00:31.098721  1614 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:00:31.099370  1614 raft_consensus.cc:606] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:00:31.099704  1614 raft_consensus.cc:614] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:31.095842  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:31.102751  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:31.095381  3843 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:31.108937  3516 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:31.113513  3854 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.429s	user 0.000s	sys 0.003s
I20260709 13:00:31.114040  3325 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:31.116248  2233 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:31.119303  1610 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:31.119891  3875 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:31.120287  3875 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.120572  3875 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.125710  3421 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:31.126209  3421 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.126595  3421 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:31.131803  3820 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:31.133042  3325 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:31.132162  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:31.131997  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:31.134925  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.131831  3804 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:31.136827  3842 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:31.136713  3162 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:31.101832  3869 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:31.137629  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2029) took 1075 ms. Trace:
I20260709 13:00:31.138438  3873 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:31.138527  3702 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:31.155411  1616 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:31.155563  3873 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.155333  1692 rpcz_store.cc:276] 0709 13:00:30.062132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.064537 (+  2405us) service_pool.cc:224] Handling call
0709 13:00:30.064908 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:31.137118 (+1072210us) spinlock_profiling.cc:243] Waited 4.1 ms on lock 0x7b2c0016e788. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08dd9c291 00007fe08ddb70ca 00007fe08fae54ad 00007fe08faebe1b 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:00:31.137245 (+   127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:31.137269 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:31.137311 (+    42us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:31.137339 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:31.137438 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:31.137575 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9434368}
I20260709 13:00:31.156837  1616 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:31.140151  3325 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:31.158059  3873 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:31.157886  3325 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.160079  3325 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:31.139520  3835 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.726s	user 0.000s	sys 0.001s
I20260709 13:00:31.160641  3717 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:31.142567  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:31.117906  3791 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.466s	user 0.010s	sys 0.028s
I20260709 13:00:31.144294  1685 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: []
W20260709 13:00:31.169166  3791 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.521s	user 0.010s	sys 0.028s
I20260709 13:00:31.145203  3866 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:31.169559  3791 raft_consensus.cc:697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 3 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:31.169957  3866 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.170394  3866 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:31.146883  3764 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:31.118201  3516 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:31.152837  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:31.155952  3162 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:31.144788  3882 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.137683  3635 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.173527  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:31.182448  1537 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.180030  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.141533  2231 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:31.182490  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.177291  3764 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:31.185930  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:31.178659  3797 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.185855  3882 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:31.178531  3702 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:31.178774  3873 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:31.194875  3764 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.196238  3764 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:31.196519  1541 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:31.188452  3635 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:31.198040  1535 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.188233  1617 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:31.170588  3791 consensus_queue.cc:237] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:31.203929  3797 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.194912  3873 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.215574  1613 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:31.212711  1537 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:31.220185  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:31.220248  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 45882.9us
I20260709 13:00:31.209759  3873 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:31.223526  3874 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.322s	user 0.000s	sys 0.003s
I20260709 13:00:31.225755  1692 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:31.210757  3635 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:00:31.211704  3834 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:31.227684  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:31.209935  3797 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:31.209367  3882 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:31.233412  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:31.205643  3875 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.245874  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.246589  1541 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:31.248011  1509 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:00:31.249243  3868 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.347s	user 0.002s	sys 0.000s
I20260709 13:00:31.249938  1509 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:31.250896  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.231362  3834 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:31.253986  3843 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:00:31.254146  3786 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:00:31.251327  3797 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:31.261701  3847 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.261924  3869 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:31.262365  3847 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.258304  3834 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.264539  3847 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:31.264835  3834 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:31.268217  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.268291  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:31.268754  3873 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:31.269654  1509 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:31.269729  3599 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:31.270704  3797 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:31.273589  3700 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:31.258240  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 412) took 1089 ms. Trace:
I20260709 13:00:31.294271  1543 rpcz_store.cc:276] 0709 13:00:30.169183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.169307 (+   124us) service_pool.cc:224] Handling call
0709 13:00:31.258008 (+1088701us) spinlock_profiling.cc:243] Waited 77.5 ms on lock 0x7b54003e1f54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:31.258155 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178205056}
I20260709 13:00:31.269161  1657 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:31.274459  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.275180  1535 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:31.275838  3421 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:31.275868  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:31.279316  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.306653  3749 raft_consensus.cc:3055] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:31.307132  3749 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:31.282024  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:31.308030  3303 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.282471  2229 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:31.308120  3749 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.308604  3303 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:31.283118  2203 leader_election.cc:336] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.123s (SENT)
I20260709 13:00:31.283806  3300 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.283819  1686 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:00:31.289304  1690 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: [3.6-5.7]
W20260709 13:00:31.295993  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.297572  3882 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:31.297103  3700 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:31.304032  2201 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:31.286279  3866 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:31.313046  1691 raft_consensus.cc:2393] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:31.311339  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:31.322820  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:31.314085  2203 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:31.301187  3676 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:31.309391  3749 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:31.309558  3843 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:31.323021  3802 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.322711  3882 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.328058  3724 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:31.329205  3804 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:31.340673  1583 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:31.370710  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:31.371652  2233 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:31.372953  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:31.376492  2233 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:31.377406  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:31.379942  2234 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:00:31.387781  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47206.5us
I20260709 13:00:31.386941  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:31.327842  3300 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.333156  3797 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:31.339730  3882 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:31.417608  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:31.451517  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 107283us
I20260709 13:00:31.420222  3303 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:31.415915  3764 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:31.453256  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:31.434019  3325 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:31.441746  3853 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:00:31.435443  3519 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:31.433714  3727 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:00:31.385757  3875 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:31.339676  3802 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.371570  3884 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:31.475766  1583 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:31.440587  3162 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:31.488422  3795 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:31.447625  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:31.499771  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:31.523746  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:31.567682  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 284580us
I20260709 13:00:31.456089  3421 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:31.419222  3800 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.102s
I20260709 13:00:31.455837  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:31.575488  2234 raft_consensus.cc:2410] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:31.507527  3727 consensus_queue.cc:1243] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:00:31.526213  3739 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:31.572117  3876 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.587345  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:31.597849  2230 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:31.599196  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:31.599989  2230 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:31.602283  3894 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:00:31.582048  3724 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:31.608261  3724 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.609732  2229 raft_consensus.cc:2410] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:31.612869  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:31.620685  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.623564  3300 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:31.578300  3873 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:31.582683  3700 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:31.590101  3739 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.568588  3162 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.525969  3717 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 13:00:31.521454  3325 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.576012  3804 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:31.641547  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 72785us
I20260709 13:00:31.594508  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:31.626210  2230 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:31.660353  3325 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:31.631883  3876 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.637065  3764 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.643326  3739 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:31.679359  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.639747  3866 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:31.685644  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 350) took 1521 ms. Trace:
I20260709 13:00:31.686164  1615 rpcz_store.cc:276] 0709 13:00:30.163966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.164110 (+   144us) service_pool.cc:224] Handling call
0709 13:00:31.684906 (+1520796us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54002bd754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:31.685574 (+   668us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":613042560}
I20260709 13:00:31.688511  2203 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:31.684109  1683 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:31.684016  3764 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.694878  1612 raft_consensus.cc:2368] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6.
I20260709 13:00:31.707733  3900 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:00:31.714107  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.721956  2203 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:31.723210  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:31.724078  2203 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:31.736454  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.737026  1692 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:31.740061  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:31.726626  3516 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:31.741593  1692 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:31.741616  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:31.727571  3802 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:31.730063  3902 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:31.738143  3900 consensus_queue.cc:1243] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:00:31.742267  3901 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:31.719792  3804 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.744529  3346 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:31.750223  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:31.757128  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.754513   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:00:31.761358  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.764489  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.766134  2203 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:31.768512  1540 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:31.769011   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:31.770205  1537 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:31.771354  3902 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:31.771893  3902 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.772549  3902 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:31.777153  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.778038  3599 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:31.778388  3588 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:31.778728  3599 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.779381  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:31.780973  3800 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:31.781586  3834 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:31.782087  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:31.782320  3834 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:31.784808  3834 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:31.785238  1535 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:31.781481  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.752523  3749 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:00:31.758118  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:31.760152  3876 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:31.780992  1509 leader_election.cc:400] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6.
I20260709 13:00:31.794512  1509 leader_election.cc:403] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:31.794337  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:31.785837  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:31.781137  3599 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:31.806643  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:31.790297  1684 raft_consensus.cc:2410] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:31.778277  3162 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:31.826668  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.785554  3819 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:31.793969  3901 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:31.828692  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.835362  1543 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:31.781544  3800 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:31.833350  1686 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:31.796304  3791 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:31.789062  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:31.754925  3896 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.060s
W20260709 13:00:31.792954  1610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1754) took 1476 ms (client timeout 1680 ms). Trace:
I20260709 13:00:31.790618  3702 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:31.785377  3834 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:31.832710  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:31.849274  2203 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.659s (SENT)
I20260709 13:00:31.837383  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.837622  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:31.852070  1616 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:31.840548  1582 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:31.852650  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:31.853629  1618 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:31.841728  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:31.819384  3635 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.809646  3837 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6.
I20260709 13:00:31.810840  3720 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:31.817909  3843 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:31.846227  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.815078  3749 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.851009  1537 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:31.777240  3797 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:31.815142  3599 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:31.850812  3764 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:31.859930  3833 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:31.861545  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 80741.4us
I20260709 13:00:31.859807  1539 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:31.870708  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:31.837249  3162 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:00:31.820340  3876 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:00:31.877482  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:31.884622  1540 raft_consensus.cc:2393] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:31.877658  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:31.885808  1616 raft_consensus.cc:2410] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:31.878320  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:31.850338  1610 rpcz_store.cc:269] 0709 13:00:30.316650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:30.342843 (+ 26193us) service_pool.cc:224] Handling call
0709 13:00:31.792752 (+1449909us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b5400001c54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:31.792870 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1163477376}
I20260709 13:00:31.885535  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:31.864080  3635 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.888657  3800 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.873159  3698 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:31.863654  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.891137  1614 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:31.865636  3843 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:31.890362  3635 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:31.892666  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:31.887686  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.866083  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:31.900556  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:31.916643  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.876989  3834 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.916673  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:31.886500  3599 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:31.918612  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:31.941042  1537 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:00:31.929337  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:31.949980  1537 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:31.951953  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:31.953336  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:31.953979  2228 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:31.861279  3819 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:31.956207  3819 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:31.930825  1540 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:31.957300  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:31.958876  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:31.930876  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 45453.5us
I20260709 13:00:31.899113  3698 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:31.960364  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:31.961030  2229 raft_consensus.cc:2376] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4. Re-sending same reply.
I20260709 13:00:31.931952  3325 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:31.937815  3421 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:31.962039  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:31.962141  3698 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:31.879544  1534 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1720) took 2332 ms (client timeout 2005 ms). Trace:
I20260709 13:00:31.892256  3833 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:31.936589  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:31.914899  3843 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:31.932015  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:31.994287  2234 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:31.962818  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:31.995769  2236 raft_consensus.cc:2393] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:31.876718  3749 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.869625  3882 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:31.987341  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50847.4us
I20260709 13:00:31.917023  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:31.988440  1688 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:31.960353  3837 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:32.005825  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:32.007512  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:31.935693  1535 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:32.017136  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:31.984542  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:32.043193  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 210769us
I20260709 13:00:31.998951  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:32.008567  3880 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 13:00:32.006027  3886 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:00:32.050160  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:32.063434  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:32.000821  3846 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.001729  3807 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:31.964262  3346 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 13:00:31.963073  3516 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260709 13:00:31.985301  3599 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.044723  3300 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:31.997238  3421 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
I20260709 13:00:31.939697  3834 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.024276  3819 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:32.059768  3806 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:31.962968  1534 rpcz_store.cc:269] 0709 13:00:29.537619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:29.553103 (+ 15484us) service_pool.cc:224] Handling call
0709 13:00:31.858136 (+2305033us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b540030b6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:31.868154 (+ 10018us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3907997440}
I20260709 13:00:32.022533  3635 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
W20260709 13:00:32.083642  3795 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.000s
I20260709 13:00:32.089391  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:32.091161  3872 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.189s	user 0.007s	sys 0.004s
I20260709 13:00:32.092454  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:32.015761  3900 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:00:32.098428  2230 raft_consensus.cc:2393] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:32.097297  1657 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:32.096376  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:32.134311  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:32.087965  3599 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:32.137449  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:32.104884  3162 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:32.039330  3797 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:32.105365  3834 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:32.105887  3833 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.106532  1537 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:32.106720  3882 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:32.100656  3764 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:32.110538  3749 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:32.139091  3896 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:32.082962  1685 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:32.078889  3717 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:32.103286  2236 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:32.095619  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:32.093084  3300 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.022516  3837 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:31.988121  3901 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.105082  3918 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:32.141538  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:32.100438  3806 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:32.148489  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 51253.1us
I20260709 13:00:32.150269  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:32.168491  1657 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:32.094453  3739 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.118089  3822 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.082680  3846 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.171648  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:32.111569  3791 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:32.161340  1615 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:32.159399  3896 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.102632  3819 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:32.111399  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 151199us
I20260709 13:00:32.172649  1657 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:32.151873  3921 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:32.151906  3922 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:32.140934  3162 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:00:32.186589  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 90272.9us
I20260709 13:00:32.166574  3842 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:32.196866  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:32.197616  1616 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:32.197811  1613 raft_consensus.cc:2368] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:32.163398  3300 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:32.169931  3916 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:32.160463  3717 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.170360  3806 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:32.185873  3882 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:32.200949  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:32.206390  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:32.170149  3516 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:32.208096  1543 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:00:32.210052  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:32.193090  3896 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:32.205463  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:32.212954  1618 raft_consensus.cc:2376] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6. Re-sending same reply.
I20260709 13:00:32.210732  1536 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:32.209208  1616 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:32.190254  3834 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:32.212317  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:32.209203  1615 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:32.195042  3921 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:32.205410  3717 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:32.228003  2201 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:32.229908  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:32.233064  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:32.233783  2235 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:32.238714  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:32.209391  3516 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.210274  3916 raft_consensus.cc:697] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 3 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:32.220850  3834 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.246239  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:32.245779  1686 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:32.188040  3822 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.221257  3882 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:32.261610  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:32.225951  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:32.262280  1615 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:32.263144  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:32.263687  1615 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:32.227025  3921 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:32.265873  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.234432  3795 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:00:32.256593  3915 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 13:00:32.251627  3516 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:32.258874  3834 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:32.276867  3884 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:32.282516  3819 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:32.222227  3922 consensus_queue.cc:1059] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:00:32.225693  3162 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:32.253234  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:32.302196  1542 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:32.267344  3698 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:32.274783  3902 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.318255  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:32.312487  2231 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
W20260709 13:00:32.325258  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 122899us
I20260709 13:00:32.266975  3804 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.051s
W20260709 13:00:32.329304  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:32.312420  3884 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:32.316959  3698 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.343083  3918 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.322985  1612 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:32.256270  3791 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:32.392920  3739 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:32.393673  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:32.384230  3720 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:32.395273  3720 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:32.344580  3698 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:32.348407  3162 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:32.384097  3876 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:32.396039  3929 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:00:32.334964  3795 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:00:32.396669  3599 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:32.388051  3837 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:32.435153  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:32.435819  1542 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:32.429477  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 623) took 1052 ms. Trace:
I20260709 13:00:32.432494  3519 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:32.437312  2235 rpcz_store.cc:276] 0709 13:00:31.377366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.383877 (+  6511us) service_pool.cc:224] Handling call
0709 13:00:31.434319 (+ 50442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:32.242586 (+808267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.429218 (+186632us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b540065a9d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:32.429394 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1835049856}
I20260709 13:00:32.380688  3918 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:32.452718  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:32.455693  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:32.457998  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:32.380002  3882 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.423131  3939 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:32.439558  3700 raft_consensus.cc:3055] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:32.439924  3935 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:32.407202  3876 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.421959  3300 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.442487  3599 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.385361  3916 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:32.480552  3932 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:00:32.490609  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 108081us
I20260709 13:00:32.496251  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:32.496973  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:32.491976  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:32.556368  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 60495.6us
I20260709 13:00:32.561853  1658 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:32.467756  3930 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:32.565783  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:32.480093  3700 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:32.567137  2236 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:00:32.575007  1583 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:32.600919  1616 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:32.604766  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:32.606019  2229 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:32.563972  3902 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:32.627274  3873 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:00:32.628007  3873 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:32.491051  3939 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.467319  3931 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.150s
I20260709 13:00:32.571197  3930 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.619534  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2039) took 1307 ms. Trace:
I20260709 13:00:32.648973  3300 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:32.647254  3942 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.608093  3922 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.665807  1542 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:32.644467  2229 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:32.731505  1615 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:32.504365  3935 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.616263  3717 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.641388  3822 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:32.676759  3930 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:32.735128  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 160145us
I20260709 13:00:32.648185  3698 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.801383  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:32.763573  3806 consensus_meta.cc:287] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.551s	user 0.003s	sys 0.011s
I20260709 13:00:32.832357  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:32.833060  1542 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 4
W20260709 13:00:32.735138  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 173338us
I20260709 13:00:32.629122  3873 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.834383  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:32.732960  3764 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:32.836886  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 368) took 1006 ms. Trace:
I20260709 13:00:32.722136  3936 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.841687  1614 rpcz_store.cc:276] 0709 13:00:31.830298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.830418 (+   120us) service_pool.cc:224] Handling call
0709 13:00:32.836678 (+1006260us) spinlock_profiling.cc:243] Waited 36.4 ms on lock 0x7b54003c91d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:32.836818 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83659648}
I20260709 13:00:32.849176  2236 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 1. Current term is 2. Ops: []
I20260709 13:00:32.850968  2233 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:32.800427  3925 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:32.797088  3842 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:32.827379  3800 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:32.830961  3927 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:32.867691  3937 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:32.868456  3937 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:32.869649  3949 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:32.870348  3949 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:32.829804  3922 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.870822  3949 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:32.799962  3942 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.844135  3822 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.873805  3822 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:32.875788  3943 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:00:32.876256  3822 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:32.877107  3325 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:32.841399  3806 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:32.878170  3945 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:00:32.879793  3806 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:32.880923  3599 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:32.675807  1685 rpcz_store.cc:276] 0709 13:00:31.307677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.313495 (+  5818us) service_pool.cc:224] Handling call
0709 13:00:31.315359 (+  1864us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:32.165450 (+850091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:32.555065 (+389615us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b540028f0d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:32.615480 (+ 60415us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1668426624}
I20260709 13:00:32.883102  3764 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.883633  3764 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.800168  3700 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:32.893674  3873 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:32.894184  3944 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:00:32.865290  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:32.893379  3842 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:32.869472  1612 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:32.902894  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:32.906328  3300 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:32.906560  3927 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.907464  3300 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.909498  3925 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:32.906440  3924 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:00:32.912185  3925 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:32.914049  3717 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:32.915814  3162 consensus_meta.cc:287] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.508s	user 0.006s	sys 0.002s
I20260709 13:00:32.901988  3325 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:00:32.917929  3912 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.188s	user 0.010s	sys 0.018s
I20260709 13:00:32.928668  3676 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:32.920920  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:32.929446  3676 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:32.922058  3931 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:32.930787  3931 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:32.931788  3676 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:32.932292  3676 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:32.932948  3931 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:32.932762  3676 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.933254  3933 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:32.922616  3325 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:32.928668  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1265) took 1399 ms. Trace:
I20260709 13:00:32.919210  3876 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:32.935019  3931 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.935230  3937 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:32.986413  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:32.909261  3927 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:32.938975  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:32.993923  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 55010.6us
W20260709 13:00:32.939924  3516 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.000s
I20260709 13:00:32.940466  3934 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:32.995294  3950 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:32.995764  3934 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:32.996210  3934 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:32.996065  3950 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:32.995309  3516 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.940932  2203 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:32.941553  3795 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:32.941680  3918 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:32.857405  3936 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.994098  3931 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.944666  3837 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:32.920401  3939 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:33.009351  3936 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:33.010046  3931 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:33.010694  3842 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:33.010844  3837 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:32.995420  3676 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:33.011713  3834 consensus_meta.cc:287] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.694s	user 0.010s	sys 0.000s
I20260709 13:00:33.012274  3834 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:33.013010  3837 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:32.947475  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.014118  3934 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:32.869556  3941 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.067s	user 0.000s	sys 0.002s
I20260709 13:00:33.007298  3918 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:00:33.016789  3676 raft_consensus.cc:3055] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:32.944937  3720 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.000s
I20260709 13:00:32.954097  1509 leader_election.cc:400] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:32.984664  1691 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:33.017683  1509 leader_election.cc:403] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 13:00:33.018503  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64624us
I20260709 13:00:33.018595  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:33.019408  1691 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:33.020079  3720 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:33.021620  3837 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:33.017198  3300 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:33.023553  3934 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:33.027410  3930 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:33.037719  3884 consensus_meta.cc:287] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.689s	user 0.005s	sys 0.005s
I20260709 13:00:33.038343  3884 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:33.043552  3896 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.816s	user 0.007s	sys 0.003s
W20260709 13:00:33.051538  1657 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.654s (SENT) [suppressed 20 similar messages]
I20260709 13:00:32.984935  1542 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:32.987290  1689 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:32.935288  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:32.971918  1617 rpcz_store.cc:276] 0709 13:00:31.528944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.529061 (+   117us) service_pool.cc:224] Handling call
0709 13:00:31.580624 (+ 51563us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:31.586066 (+  5442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:31.586114 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:31.633218 (+ 47104us) log.cc:844] Serialized 26 byte log entry
0709 13:00:31.694470 (+ 61252us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:31.705672 (+ 11202us) log.cc:844] Serialized 14 byte log entry
0709 13:00:31.706066 (+   394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:31.706170 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:31.774013 (+ 67843us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b180035e540. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:00:32.917462 (+1143449us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b2400104708. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c8859c4 00007fe08c883acb 00007fe08ca3edc9 00007fe08ca3ea55 00007fe08ca3e09d 00007fe08f67010e 00007fe08f68788c 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221
0709 13:00:32.928431 (+ 10969us) raft_consensus.cc:1685] finished
0709 13:00:32.928451 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:32.928576 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11249,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":1978,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":37679,"spinlock_wait_cycles":31630976,"thread_start_us":36944,"threads_started":1,"wal-append.queue_time_us":77242}
I20260709 13:00:32.990419  3935 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:33.058369  1611 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:32.990922  1687 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:00:32.991601  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.059710  1685 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:32.946732  3717 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:33.060737  3717 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:33.061197  2201 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:32.954643  3866 raft_consensus.cc:3055] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:32.994829  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:33.004406  3795 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:33.006322  3843 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.997s	user 0.015s	sys 0.006s
W20260709 13:00:33.007333  3921 consensus_meta.cc:287] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.001s
I20260709 13:00:32.985631  3764 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:33.014528  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:32.947603  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.017156  3676 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:32.949836  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:32.971746  3933 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:33.063410  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 122555us
I20260709 13:00:33.041105  3884 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:33.078359  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 83641.4us
I20260709 13:00:33.079286  2201 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:33.044178  3896 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:32.995483  3876 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:33.087265  2234 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:33.089083  3949 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:33.091135  3930 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:33.113822  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.081200  3724 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:33.077872  3866 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:33.117862  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.080535  3822 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:00:33.119849  1688 raft_consensus.cc:2368] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6.
I20260709 13:00:33.082039  3935 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:33.121886  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:33.133831  1691 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:33.098368  3933 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:33.124953  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:33.134294  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.142683  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.146334  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.147660  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.084369  3806 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:00:33.152374  2233 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:33.118976  3519 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:33.150604  2235 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:33.153963  1688 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:33.123376  3724 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.085992  3843 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:33.095314  3921 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:33.124841  3876 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:33.133337  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:33.159977  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 330717us
W20260709 13:00:33.253358  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 201842us
I20260709 13:00:33.200995  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:33.511335  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.156621  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:33.512836  1542 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:33.510537  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:33.513877  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.514623  1688 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:33.190567  1692 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:33.515666  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.516215  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.511837  1537 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
W20260709 13:00:33.248123  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 308118us
I20260709 13:00:33.512063  1687 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:33.523114  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:33.527693  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:33.528612  1615 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:33.560786  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:33.563899  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:33.528502  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:33.581216  1616 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 2: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:33.566639  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:33.581308  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:33.582376  1615 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:33.566483  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.571671  1538 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:33.579097  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:33.585119  1542 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:33.584065  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:33.585829  1617 raft_consensus.cc:2410] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:33.584195  1687 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:33.582232  1614 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:33.590898  1613 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:00:33.591799  1613 pending_rounds.cc:85] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 5
I20260709 13:00:33.592240  1613 pending_rounds.cc:107] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:00:33.592666  1613 raft_consensus.cc:2889] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:33.216336  1684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.006s	sys 0.040s
I20260709 13:00:33.157591  3519 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:33.210305  3717 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:33.583329  3791 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.017s	sys 0.022s
I20260709 13:00:33.607723  3717 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:33.608054  3791 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.017s	sys 0.022s
I20260709 13:00:33.587738  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.611016  1691 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:33.190104  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:33.215634  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:33.153494  3935 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:33.568862  2232 raft_consensus.cc:2410] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:33.661633  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:33.506747  3797 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.527458  3724 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:33.608966  3717 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
W20260709 13:00:33.614979  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 51210.4us
I20260709 13:00:33.623937  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:33.612766  1684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.944s	user 0.014s	sys 0.058s
I20260709 13:00:33.628602  3950 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:33.609252  3791 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.610025  3519 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:33.661168  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:33.665791  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.617895  2229 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:33.506309  3837 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:33.669224  1583 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:33.761899  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 92774.7us
I20260709 13:00:33.754254  3958 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:33.660857  3931 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.050s	sys 0.102s
I20260709 13:00:33.231712  3834 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:33.523413  3930 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:33.152004  3676 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:33.774189  3930 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:33.775341  3959 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
W20260709 13:00:33.776639  3739 consensus_queue.cc:1175] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cecad4f86a114a7696e398b3e8cc511e" responder_term: 4 status { last_received { term: 4 index: 6 } last_committed_idx: 5 last_received_current_leader { term: 4 index: 6 } }
I20260709 13:00:33.776980  3930 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:33.782004  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 433) took 1117 ms. Trace:
I20260709 13:00:33.782568  1536 rpcz_store.cc:276] 0709 13:00:32.665094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.665332 (+   238us) service_pool.cc:224] Handling call
0709 13:00:32.665839 (+   507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:32.674072 (+  8233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:32.674144 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:32.818776 (+144632us) log.cc:844] Serialized 26 byte log entry
0709 13:00:33.119478 (+300702us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:33.119552 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.119651 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:33.781697 (+662046us) raft_consensus.cc:1685] finished
0709 13:00:33.781728 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.781924 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3624,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":27345,"thread_start_us":255249,"threads_started":1,"wal-append.queue_time_us":614291}
I20260709 13:00:33.520730  3866 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:33.677402  1685 raft_consensus.cc:2393] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:33.728999  3876 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:33.734236  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:33.750384  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 241930us
I20260709 13:00:33.664458  2230 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:33.671782  3516 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:33.672052  3937 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:33.762627  1583 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:33.601969  3802 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.760757  1687 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 2.
I20260709 13:00:33.728304  3717 consensus_queue.cc:1243] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:00:33.768453  3931 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.051s	sys 0.102s
W20260709 13:00:33.819198  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 241018us
I20260709 13:00:33.676031  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:33.820313  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 144390us
I20260709 13:00:33.678032  3797 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:33.808738  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:33.754820  1612 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:00:33.821401  1540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.020s	sys 0.026s
W20260709 13:00:33.832958  1540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.020s	sys 0.026s
I20260709 13:00:33.822382  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:33.753048  3791 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:33.856987  3963 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:33.834740  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:33.858268  3931 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:33.858994  3931 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:33.859607  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:33.860105  3931 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:33.861166  3924 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:33.857223  3948 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:33.831983  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 69431.4us
I20260709 13:00:33.820451  3957 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:33.870105  3957 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:33.874342  3966 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:00:33.876845  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:33.877532  1687 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:33.822456  3935 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:33.786693  3837 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:33.865104  3961 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:33.803551  3958 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:33.833689  3950 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:33.887684  3795 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:33.826444  3930 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
W20260709 13:00:33.922359  2232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 809) took 1994 ms (client timeout 1654 ms). Trace:
I20260709 13:00:33.827406  3802 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:33.833261  3717 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260709 13:00:33.927731  3957 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:33.952765  3949 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:33.971808  1687 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:33.958369  3806 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:33.993847  3806 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:33.994997  3802 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:33.978261  3717 consensus_queue.cc:1243] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:00:33.995777  3791 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:33.996047  3717 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:33.996438  3717 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:33.997272  3717 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:33.960580  3921 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:33.936374  3961 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:33.960891  3516 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:33.992920  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 420) took 2169 ms. Trace:
I20260709 13:00:33.971392  3843 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:34.008009  1538 rpcz_store.cc:276] 0709 13:00:31.824012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.824219 (+   207us) service_pool.cc:224] Handling call
0709 13:00:31.842250 (+ 18031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.584518 (+1742268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.992606 (+408088us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b54000ed8d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:33.992814 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3976300288}
I20260709 13:00:33.996989  3806 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:34.020169  3967 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
W20260709 13:00:34.020666  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 394769us
I20260709 13:00:34.021478  3516 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:34.022469  2203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.006s	sys 0.008s
W20260709 13:00:34.022832  2203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.006s	sys 0.008s
I20260709 13:00:34.023767  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:34.021880  3958 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:33.983052  3950 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:33.997056  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 363) took 2195 ms. Trace:
I20260709 13:00:34.001665  3949 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.031873  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:33.958582  2232 rpcz_store.cc:269] 0709 13:00:31.892626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.894703 (+  2077us) service_pool.cc:224] Handling call
0709 13:00:33.830053 (+1935350us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54003d3bd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:33.886987 (+ 56934us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2647034624}
I20260709 13:00:34.038404  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:34.006259  3896 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.015612  3973 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.017769  3717 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.023219  3921 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.026588  3806 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.082573  3346 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:34.083510  3346 raft_consensus.cc:3055] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:34.083587  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:34.083938  3346 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:34.084733  2229 raft_consensus.cc:2410] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:00:34.085768  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 225260us
W20260709 13:00:34.087001  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 223989us
I20260709 13:00:34.083804  2230 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:34.032994  3958 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:34.031404  3843 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:34.063982  3933 consensus_meta.cc:287] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.855s	user 0.011s	sys 0.001s
I20260709 13:00:34.086782  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:34.053164  3802 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:34.068342  3937 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:34.092629  3846 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:34.121171  3846 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.121903  3846 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:34.085045  3346 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.123268  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 435) took 1350 ms. Trace:
I20260709 13:00:34.082629  3300 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:34.123956  3300 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.119664  3802 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:34.125948  3802 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.123667  1540 rpcz_store.cc:276] 0709 13:00:32.772611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.879734 (+107123us) service_pool.cc:224] Handling call
0709 13:00:32.880083 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:33.116681 (+236598us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:33.116708 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:33.133151 (+ 16443us) log.cc:844] Serialized 26 byte log entry
0709 13:00:33.833337 (+700186us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:33.833375 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.833462 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:34.123049 (+289587us) raft_consensus.cc:1685] finished
0709 13:00:34.123068 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.123208 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":64086,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":836123,"threads_started":2,"wal-append.queue_time_us":868848}
I20260709 13:00:34.065145  3795 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:34.132479  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:34.141734  1618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1792) took 1941 ms (client timeout 1858 ms). Trace:
W20260709 13:00:34.095404  3965 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 13:00:34.145714  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:34.146518  1536 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.131587  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:34.153218  1538 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:34.155234  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.045863  3950 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.130443  3720 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:34.142210  1618 rpcz_store.cc:269] 0709 13:00:32.200325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.224792 (+ 24467us) service_pool.cc:224] Handling call
0709 13:00:34.132331 (+1907539us) spinlock_profiling.cc:243] Waited 5.05 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:34.141659 (+  9328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11622272}
I20260709 13:00:34.081015  3979 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.065402  3980 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:00:34.126933  3960 log.cc:927] Time spent T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:00:34.144971  3882 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:34.222069  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:34.222805  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:34.216869  1657 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:34.211804  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 125157us
I20260709 13:00:34.229190  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:34.230455  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:34.239925  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:34.241300  1618 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:34.142962  3846 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.128403  3797 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.242744  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:34.126400  3300 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.127652  3749 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:34.099341  3921 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:34.233220  1538 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:34.208796  3984 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.243800  1618 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:34.270768  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:34.271385  1618 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:34.239149  1686 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:34.067833  3949 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.234055  3980 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:34.119756  3958 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:34.208837  3948 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:34.207945  3981 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:34.093379  3806 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.120345  3924 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:34.219990  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:34.145608  3974 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.131942  3843 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.281728  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:34.269636  3720 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:34.161048  3933 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:34.289598  1618 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:34.086769  3973 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:34.247762  3837 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.248338  3749 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:34.268926  3846 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.269368  3300 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:34.284962  1611 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.226s	user 0.009s	sys 0.005s
I20260709 13:00:34.164319  3937 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.279574  3797 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.299250  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:34.283542  3980 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:00:34.315402  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 98625.6us
I20260709 13:00:34.297647  3974 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.311398  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 432) took 1359 ms. Trace:
W20260709 13:00:34.326766  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45103.3us
I20260709 13:00:34.302974  3987 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:00:34.308156  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 94028.3us
I20260709 13:00:34.313222  1617 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:34.321992  3994 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:34.244602  3979 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:34.298214  3981 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.310714  3961 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:34.332953  3969 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.333s	user 0.000s	sys 0.001s
W20260709 13:00:34.344602  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.298180  3795 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:34.346517  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:34.337538  3974 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.356102  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:34.337914  1690 rpcz_store.cc:276] 0709 13:00:32.951783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.952911 (+  1128us) service_pool.cc:224] Handling call
0709 13:00:32.988823 (+ 35912us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:32.989228 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:32.989254 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:32.989873 (+   619us) log.cc:844] Serialized 52 byte log entry
0709 13:00:33.248640 (+258767us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:33.248708 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.248818 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:34.311060 (+1062242us) raft_consensus.cc:1685] finished
0709 13:00:34.311091 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.311308 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5304,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":35207,"spinlock_wait_cycles":28160,"thread_start_us":258294,"threads_started":1,"wal-append.queue_time_us":772625}
W20260709 13:00:34.357890  1541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2222) took 2595 ms (client timeout 2672 ms). Trace:
I20260709 13:00:34.299276  1612 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:34.341144  1536 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:34.032586  1609 rpcz_store.cc:276] 0709 13:00:31.801781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.815915 (+ 14134us) service_pool.cc:224] Handling call
0709 13:00:31.860996 (+ 45081us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:32.192601 (+331605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:32.194292 (+  1691us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:32.249623 (+ 55331us) log.cc:844] Serialized 26 byte log entry
0709 13:00:32.485137 (+235514us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:32.845058 (+359921us) log.cc:844] Serialized 14 byte log entry
0709 13:00:32.865686 (+ 20628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.996568 (+1130882us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54003219d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:33.996624 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:33.996678 (+    54us) raft_consensus.cc:1685] finished
0709 13:00:33.996797 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:33.996965 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31395,"prepare.queue_time_us":438229,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":64,"raft.queue_time_us":98888,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":7951,"spinlock_wait_cycles":430726912,"thread_start_us":383133,"threads_started":3,"wal-append.queue_time_us":257826,"wal-append.run_cpu_time_us":4851,"wal-append.run_wall_time_us":1426779}
I20260709 13:00:34.312529  1611 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
W20260709 13:00:34.398154  1611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1783) took 2203 ms (client timeout 1833 ms). Trace:
I20260709 13:00:34.398474  3806 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:34.398604  1611 rpcz_store.cc:269] 0709 13:00:32.195180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.205215 (+ 10035us) service_pool.cc:224] Handling call
0709 13:00:32.935110 (+729895us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:34.398048 (+1462938us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4056960}
W20260709 13:00:34.399199  3519 consensus_meta.cc:287] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.001s
I20260709 13:00:34.312367  3300 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.399680  3519 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.401314  3300 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.312266  3846 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.402830  3702 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:34.404306  3846 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:34.337383  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2048) took 2542 ms. Trace:
I20260709 13:00:34.345166  3994 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.405716  3994 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.368304  3958 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.329376  3843 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:34.344475  3797 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:34.278818  3984 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:34.358649  3933 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:34.312585  3989 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:34.326850  3988 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:34.365170  3948 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:34.363044  3924 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:34.351436  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:34.385763  1541 rpcz_store.cc:269] 0709 13:00:31.762725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.762852 (+   127us) service_pool.cc:224] Handling call
0709 13:00:34.357657 (+2594805us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400411c54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:34.357817 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2804898688}
I20260709 13:00:34.418412  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:34.397248  3739 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:34.436053  1690 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:34.420650  3797 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.338575  3749 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.439338  1687 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:34.442315  3795 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.387548  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 427) took 2153 ms. Trace:
I20260709 13:00:34.444572  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:34.440914  3739 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:34.447196  3739 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:34.447345  3999 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:00:34.448004  3802 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.449561  3949 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:34.355993  3987 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:34.451795  3987 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.453898  3749 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.411893  3516 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:34.433861  2231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 511) took 2650 ms (client timeout 1970 ms). Trace:
W20260709 13:00:34.458043  2231 rpcz_store.cc:269] 0709 13:00:31.782917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.788755 (+  5838us) service_pool.cc:224] Handling call
0709 13:00:34.433597 (+2644842us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b54004ba854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:34.433736 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1043632000}
I20260709 13:00:34.464411  3519 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.470422  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 426) took 2380 ms. Trace:
I20260709 13:00:34.405037  1683 rpcz_store.cc:276] 0709 13:00:31.795510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.808400 (+ 12890us) service_pool.cc:224] Handling call
0709 13:00:31.830668 (+ 22268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:31.894045 (+ 63377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:31.897176 (+  3131us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:32.051665 (+154489us) log.cc:844] Serialized 26 byte log entry
0709 13:00:32.342598 (+290933us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:32.367004 (+ 24406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.047860 (+680856us) spinlock_profiling.cc:243] Waited 35 ms on lock 0x7b54000749d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:33.047980 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:34.334149 (+1286169us) spinlock_profiling.cc:243] Waited 7.52 ms on lock 0x7b18001bd0c0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:00:34.337078 (+  2929us) raft_consensus.cc:1685] finished
0709 13:00:34.337106 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.337286 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37764,"raft.queue_time_us":46354,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":28459,"spinlock_wait_cycles":97898240,"thread_start_us":216349,"threads_started":1,"wal-append.queue_time_us":311458}
I20260709 13:00:34.469228  3980 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.474046  3980 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:34.474160  3802 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:34.471364  3994 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.414709  3974 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.562605  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 373) took 1018 ms. Trace:
I20260709 13:00:34.484444  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:34.542747  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 61156.3us
I20260709 13:00:34.576973  1614 rpcz_store.cc:276] 0709 13:00:33.544149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.669249 (+125100us) service_pool.cc:224] Handling call
0709 13:00:33.726415 (+ 57166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:34.029635 (+303220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:34.029663 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:34.034084 (+  4421us) log.cc:844] Serialized 26 byte log entry
0709 13:00:34.283136 (+249052us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:34.560684 (+277548us) log.cc:844] Serialized 14 byte log entry
0709 13:00:34.561691 (+  1007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.561818 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:34.561883 (+    65us) raft_consensus.cc:1685] finished
0709 13:00:34.561906 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.562505 (+   599us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9703,"prepare.queue_time_us":41577,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":119318,"spinlock_wait_cycles":15081216,"thread_start_us":745339,"threads_started":3,"wal-append.queue_time_us":356385}
I20260709 13:00:34.584652  3994 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:34.585541  4001 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:34.585991  1691 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:34.527359  3717 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:34.542933  3843 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.609875  3843 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.541723  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:34.586265  4001 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.527932  3950 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:34.550833  3973 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.555857  3921 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.587747  3700 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:34.588022  3984 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.637545  1618 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:34.661604  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:34.663522  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:34.664379  2230 raft_consensus.cc:2410] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:34.664573  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:34.602782  3727 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:00:34.670997  2233 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:34.665266  2235 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 2: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:34.665292  2231 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:34.665661  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:34.666812  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:34.668313  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:34.613379  3843 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:34.609295  3717 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:34.583006  3974 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.674960  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1249) took 1472 ms. Trace:
W20260709 13:00:34.687326  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.637300  3979 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.629169  3981 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:34.693544  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:34.690021  3724 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.969s	user 0.007s	sys 0.006s
I20260709 13:00:34.687433  1689 rpcz_store.cc:276] 0709 13:00:33.202375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.202558 (+   183us) service_pool.cc:224] Handling call
0709 13:00:33.513419 (+310861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.513664 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:33.890751 (+377087us) log.cc:844] Serialized 14 byte log entry
0709 13:00:34.674487 (+783736us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:34.674561 (+    74us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:34.674642 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.674732 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.674883 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2695,"prepare.queue_time_us":465336,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":49392,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":20224,"thread_start_us":1113913,"threads_started":3,"wal-append.queue_time_us":719198}
I20260709 13:00:34.680807  1690 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:34.470995  1535 rpcz_store.cc:276] 0709 13:00:32.090476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.099874 (+  9398us) service_pool.cc:224] Handling call
0709 13:00:32.152660 (+ 52786us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:32.231003 (+ 78343us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:32.258742 (+ 27739us) log.cc:844] Serialized 14 byte log entry
0709 13:00:32.496684 (+237942us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:32.496751 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:32.496812 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:34.470065 (+1973253us) spinlock_profiling.cc:243] Waited 56.8 ms on lock 0x7b54000ae454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:34.470135 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.470321 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5520,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":186,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":72,"spinlock_wait_cycles":130536704,"thread_start_us":213158,"threads_started":1,"wal-append.queue_time_us":438564,"wal-append.run_cpu_time_us":2390,"wal-append.run_wall_time_us":1212268}
W20260709 13:00:34.715525  4006 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:00:34.716606  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:34.717962  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:34.722875  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.689730  3717 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:34.724709  4001 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:34.462215  1684 rpcz_store.cc:276] 0709 13:00:32.228449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.230368 (+  1919us) service_pool.cc:224] Handling call
0709 13:00:32.308196 (+ 77828us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:32.425951 (+117755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:32.430544 (+  4593us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:32.556075 (+125531us) log.cc:844] Serialized 26 byte log entry
0709 13:00:33.725617 (+1169542us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:33.736073 (+ 10456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:33.753878 (+ 17805us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:34.373961 (+620083us) raft_consensus.cc:1685] finished
0709 13:00:34.373984 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:34.381705 (+  7721us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6460,"raft.queue_time_us":125186,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":31933,"thread_start_us":1085543,"threads_started":2,"wal-append.queue_time_us":1216405}
I20260709 13:00:34.726940  3635 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:00:34.727522  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:34.738986  3635 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:34.739346  4009 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.704155  3980 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.692444  3749 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.753347  1690 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:34.727977  3918 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:34.712581  3724 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.739986  3635 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:34.755100  3980 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:34.755869  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:34.757706  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:34.913393  1691 raft_consensus.cc:2368] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4.
I20260709 13:00:34.914693  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:34.915293  1691 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:34.762576  3698 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:34.740067  4009 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:34.763698  3749 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:34.771039  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:34.894851  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 548396us
I20260709 13:00:34.750378  3700 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:34.898633  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 156304us
W20260709 13:00:34.901134  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 113864us
W20260709 13:00:34.870390  3876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.013s	sys 0.023s
I20260709 13:00:34.905093  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:35.002770  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:35.100929  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:35.079195  1583 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.041s	sys 0.046s
W20260709 13:00:35.242498  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 82851.8us
I20260709 13:00:35.242697  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:35.243595  1690 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
W20260709 13:00:35.167270  1584 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.006s	sys 0.005s
W20260709 13:00:35.246251  1584 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.010s	sys 0.007s
W20260709 13:00:35.191514  1583 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.049s	sys 0.053s
I20260709 13:00:34.898548  3930 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:35.246645  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 686426us
I20260709 13:00:35.287046  3930 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.289469  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:35.290118  1614 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
W20260709 13:00:35.291044  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.292168  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:35.278966  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.282018  4013 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.282101  1610 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.283977  3749 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:35.285907  4003 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:35.112102  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:35.241719  1717 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:36375 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:36375 timed out after 15.000s (SENT)
I20260709 13:00:35.295235  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:35.302549  1690 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 4: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:00:35.154951  3739 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.007s
I20260709 13:00:35.248508  2201 leader_election.cc:400] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4.
I20260709 13:00:35.304672  2201 leader_election.cc:403] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:00:34.916114  3916 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.985s	user 0.041s	sys 0.021s
W20260709 13:00:35.229388  3876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.013s	sys 0.024s
I20260709 13:00:35.314502  3727 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.316355  3727 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.317217  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:35.318559  3727 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:35.318899  1685 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:35.320494  3964 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.243474  3984 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:35.310595  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:35.200234  3843 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.132257  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2229) took 2928 ms. Trace:
I20260709 13:00:34.900386  4007 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:35.136869  3980 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:35.147311  3724 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:34.909588  3884 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:35.272374  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:35.117228  3866 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:35.314954  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 69107.5us
I20260709 13:00:35.111733  3717 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.323817  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:34.959985  4001 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.355s
W20260709 13:00:35.311757  3916 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.381s	user 0.046s	sys 0.056s
I20260709 13:00:34.908380  3519 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:35.349116  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:35.350358  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.321668  3981 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.351786  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:35.295416  4013 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.363216  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:35.131322  4022 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:34.944067  3918 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:35.364702  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:35.345662  3800 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.003s	sys 0.009s
I20260709 13:00:35.400218  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:35.400835  3800 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.004s	sys 0.009s
I20260709 13:00:35.401080  1685 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.314808  3749 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:35.358841  1540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.069s	user 0.093s	sys 0.126s
I20260709 13:00:35.367398  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:35.403506  1540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.115s	user 0.093s	sys 0.127s
I20260709 13:00:35.392856  1615 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:35.244532  3700 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:35.346454  1658 connection.cc:256] RPC call timeout handler was delayed by -2.3475060769999914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18330000000000002s for a response
I20260709 13:00:35.393589  1690 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:35.351809  3901 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.315296  3739 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:35.398540  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:35.400092  3918 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:35.321043  3964 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:35.393277  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:35.406868  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:35.408821  1658 connection.cc:256] RPC call timeout handler was delayed by -2.2355018200000245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
I20260709 13:00:35.409693  1691 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:35.351980  3963 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:00:35.449924  1688 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:35.352944  3698 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:00:35.343096  1684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.030s	sys 0.068s
I20260709 13:00:35.347569  4007 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:35.402971  4011 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.347834  1610 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:35.448540  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:35.409533  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:35.486132  1690 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:35.411592  3749 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:35.450980  1658 connection.cc:256] RPC call timeout handler was delayed by -2.1315060769999832s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2073s for a response
W20260709 13:00:35.489295  1658 connection.cc:256] RPC call timeout handler was delayed by -2.0411018199999944s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2074s for a response
I20260709 13:00:35.490760  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:35.490988  1657 leader_election.cc:336] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.654s (SENT)
I20260709 13:00:35.491673  1657 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:35.455513  1687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.900s	user 0.008s	sys 0.020s
I20260709 13:00:35.419375  3930 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:35.528738  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:35.530058  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:35.461623  2228 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:35.408200  3916 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:35.418761  3974 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:35.473572  1615 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:35.553822  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 377) took 2008 ms. Trace:
I20260709 13:00:35.554183  1615 rpcz_store.cc:276] 0709 13:00:33.545381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.600421 (+ 55040us) service_pool.cc:224] Handling call
0709 13:00:35.363059 (+1762638us) spinlock_profiling.cc:243] Waited 85.2 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:35.553762 (+190703us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195872000}
I20260709 13:00:35.555424  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:35.457172  2229 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:35.373600  3884 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:35.447674  3918 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:35.395915  3519 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:35.459517  1684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.031s	sys 0.069s
I20260709 13:00:35.361523  3843 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:35.510936  1687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.955s	user 0.008s	sys 0.020s
I20260709 13:00:35.407514  4022 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.357722  3984 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.419050  3901 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:35.538272  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47345.9us
I20260709 13:00:35.478107  3802 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:35.557250  4019 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:35.481498  3963 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:35.470409  3698 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.544739  3876 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:35.584118  4020 log.cc:927] Time spent T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.190s	user 0.000s	sys 0.000s
I20260709 13:00:35.586452  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:35.591370  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 380) took 2045 ms. Trace:
W20260709 13:00:35.592741  3962 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.996s	user 0.014s	sys 0.000s
W20260709 13:00:35.543730  1610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1275) took 2014 ms (client timeout 1528 ms). Trace:
I20260709 13:00:35.558352  1612 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:35.557302  3948 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:35.574601  3930 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:35.601439  3873 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.648366  1618 rpcz_store.cc:276] 0709 13:00:33.546284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.290500 (+744216us) service_pool.cc:224] Handling call
0709 13:00:34.290781 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.660065 (+369284us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.590999 (+930934us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b540008c0d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:35.591286 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796418816}
W20260709 13:00:35.645058  4014 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.168s	user 0.000s	sys 0.003s
W20260709 13:00:35.657881  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.659034  2237 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:35.645279  4017 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.605361  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:35.672290  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:35.690243  1618 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.535953  4011 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.675482  3882 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:35.580472  3916 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.665079  3948 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:35.561959  4007 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.701798  2235 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.671501  4009 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:35.706295  1618 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
W20260709 13:00:35.710704  1506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.026s	sys 0.071s
I20260709 13:00:35.682003  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:35.707015  1507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.009s	sys 0.019s
I20260709 13:00:35.683334  3842 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:35.711786  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:35.721979  1618 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:35.723264  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:35.586506  3843 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:35.717141  1507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.010s	sys 0.019s
I20260709 13:00:35.697932  3882 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:35.652562  1610 rpcz_store.cc:269] 0709 13:00:33.529212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.530430 (+  1218us) service_pool.cc:224] Handling call
0709 13:00:34.770752 (+1240322us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:35.533518 (+762766us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37723520}
W20260709 13:00:35.713086  1506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.026s	sys 0.071s
I20260709 13:00:35.687096  3717 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:35.735395  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:35.708488  3948 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:35.742198  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:35.743044  1538 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:35.723927  3635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.021s	sys 0.068s
I20260709 13:00:35.740262  1536 raft_consensus.cc:2368] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6.
I20260709 13:00:35.672849  3930 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.745669  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:35.746374  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:35.752140  1542 raft_consensus.cc:2410] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:35.706122  2229 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:35.679154  3873 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:35.749851  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:35.757886  1543 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:35.754086  1618 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:35.551486  1688 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:35.724177  3964 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:35.754259  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:35.694278  4017 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.768234  1541 raft_consensus.cc:2410] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:35.345070  1534 rpcz_store.cc:276] 0709 13:00:32.203738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.205537 (+  1799us) service_pool.cc:224] Handling call
0709 13:00:32.250682 (+ 45145us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:32.270392 (+ 19710us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:32.324358 (+ 53966us) log.cc:844] Serialized 14 byte log entry
0709 13:00:34.338395 (+2014037us) spinlock_profiling.cc:243] Waited 4.04 ms on lock 0x7b180044c8c0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a 00007fe08faf6a2a
0709 13:00:35.066234 (+727839us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:35.079092 (+ 12858us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:35.131882 (+ 52790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.132000 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:35.132171 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15880,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":6110,"raft.queue_time_us":51985,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":12559,"spinlock_wait_cycles":9288448,"thread_start_us":446501,"threads_started":1,"wal-append.queue_time_us":2791695}
W20260709 13:00:35.768409  1508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.012s	sys 0.024s
W20260709 13:00:35.854485  1508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.012s	sys 0.025s
I20260709 13:00:35.768705  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:35.854940  1508 connection.cc:256] RPC call timeout handler was delayed by -1.1964738149999903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
I20260709 13:00:35.856499  3873 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:35.859335  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:35.860271  1683 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:35.860721  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 449) took 1104 ms. Trace:
I20260709 13:00:35.797139  1538 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:35.800308  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:35.866328  1543 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:00:35.799438  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.804037  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:35.868116  1542 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:35.797660  3749 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:35.801179  4028 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
W20260709 13:00:35.807228  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 439850us
W20260709 13:00:35.748943  3635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.021s	sys 0.068s
I20260709 13:00:35.748617  4007 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:35.810583  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.881771  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:35.882678  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:35.812518  1536 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.814198  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:35.883929  1616 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:35.814559  1537 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:35.767930  4013 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:35.861037  1683 rpcz_store.cc:276] 0709 13:00:34.756327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.859016 (+1102689us) service_pool.cc:224] Handling call
0709 13:00:35.860656 (+  1640us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:35.729437  3842 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.854187  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:35.801254  3930 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:35.805377  2234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 630) took 3309 ms (client timeout 3364 ms). Trace:
I20260709 13:00:35.753858  3843 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
W20260709 13:00:35.813365  4027 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 13:00:35.813345  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:35.906217  1543 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:35.892376  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:35.796149  1618 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:35.874194  3519 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:35.879776  3717 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:35.871506  4001 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:35.885147  3884 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:35.856379  1508 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:35.898833  1536 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:35.892877  3749 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:00:35.900763  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:35.913951  1616 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:35.897838  4022 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:35.878422  4009 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.904474  1542 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:35.899250  3980 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
W20260709 13:00:35.876688  4016 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.270s	user 0.000s	sys 0.002s
I20260709 13:00:35.925348  3980 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:35.926216  3980 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:35.904934  3942 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:35.927270  3980 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.928526  3698 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:35.928766  3980 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:35.934002  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:35.900133  2234 rpcz_store.cc:269] 0709 13:00:32.495698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:32.496038 (+   340us) service_pool.cc:224] Handling call
0709 13:00:35.805133 (+3309095us) spinlock_profiling.cc:243] Waited 526 ms on lock 0x7b54005b0254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:35.805305 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1208811264}
I20260709 13:00:35.934736  1683 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:35.914749  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:35.935977  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:35.911309  4001 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:35.936695  4001 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:35.922151  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 65851.4us
I20260709 13:00:35.913003  3930 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.933665  4009 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.936105  1616 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:35.944782  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:35.968885  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:35.947935  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2066) took 1598 ms. Trace:
I20260709 13:00:35.972234  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:35.953975  1536 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:35.906701  3984 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:35.934643  3884 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:36.002362  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:36.002527  1691 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.016s	sys 0.033s
W20260709 13:00:36.003365  1691 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.532s	user 0.016s	sys 0.033s
I20260709 13:00:36.003283  1616 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
W20260709 13:00:35.986131  1582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.008s	sys 0.040s
I20260709 13:00:35.976157  1508 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:35.970538  1686 rpcz_store.cc:276] 0709 13:00:34.349336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.349561 (+   225us) service_pool.cc:224] Handling call
0709 13:00:34.349972 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.422575 (+ 72603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:34.422619 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:34.422676 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:34.422730 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.947627 (+1524897us) spinlock_profiling.cc:243] Waited 72.3 ms on lock 0x7b54000aafd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:35.947690 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:35.947849 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166317824}
I20260709 13:00:35.979902  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:35.934629  3937 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:35.914438  1610 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:35.944316  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:36.104445  1655 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.007s	sys 0.007s
W20260709 13:00:36.096417  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 84035.9us
W20260709 13:00:35.938277  4024 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 13:00:36.095950  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:36.101883  2203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.083s	sys 0.144s
W20260709 13:00:36.004809  1685 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.002s
I20260709 13:00:35.941401  3922 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:36.096653  1582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.013s	sys 0.046s
W20260709 13:00:36.120082  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 717180us
W20260709 13:00:36.102310  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 126255us
I20260709 13:00:36.104030  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:35.945035  3957 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.946557  4001 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.976975  4031 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:36.004998  3980 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:35.917686  3717 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:35.940308  3842 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:36.002265  4032 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.001863  3698 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:36.003876  4011 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:36.003542  3884 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:36.121587  4030 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:36.008044  3916 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:35.937330  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:36.008677  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 439) took 1264 ms. Trace:
I20260709 13:00:36.124899  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:36.152706  1536 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:00:36.110579  1655 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.007s	sys 0.007s
I20260709 13:00:36.112888  1616 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:36.109019  2235 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
W20260709 13:00:36.113078  2203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.083s	sys 0.145s
W20260709 13:00:36.154407  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:36.168454  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 819365us
W20260709 13:00:36.127554  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.131325  1686 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:36.108484  3984 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:35.977677  3930 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:36.145661  1540 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.010s	sys 0.014s
W20260709 13:00:36.178797  1540 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.010s	sys 0.014s
W20260709 13:00:36.109045  3833 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.182s	user 0.036s	sys 0.061s
W20260709 13:00:36.145469  1690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.003s	sys 0.019s
I20260709 13:00:36.139115  2230 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:00:36.136143  3866 consensus_meta.cc:287] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.762s	user 0.003s	sys 0.008s
I20260709 13:00:36.116581  3843 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:36.123459  2200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.014s	sys 0.005s
I20260709 13:00:36.121110  1685 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:36.151022  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2070) took 1669 ms. Trace:
I20260709 13:00:36.133168  4031 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:36.170898  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:00:36.180258  1690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.004s	sys 0.019s
I20260709 13:00:36.142198  3980 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:36.179773  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 442) took 2037 ms. Trace:
I20260709 13:00:36.195560  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:36.179935  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:36.198817  1540 rpcz_store.cc:276] 0709 13:00:34.142175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.142660 (+   485us) service_pool.cc:224] Handling call
0709 13:00:34.143854 (+  1194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:34.144076 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:34.288232 (+144156us) log.cc:844] Serialized 14 byte log entry
0709 13:00:36.179228 (+1890996us) log.cc:844] Serialized 14 byte log entry
0709 13:00:36.179435 (+   207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:36.179488 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:36.179532 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.179596 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:36.179717 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":244126,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":327365,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":9625,"thread_start_us":1953452,"threads_started":3,"wal-append.queue_time_us":1238041}
W20260709 13:00:36.185276  1658 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.014s	sys 0.011s
I20260709 13:00:36.181669  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:36.201650  1658 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.014s	sys 0.011s
I20260709 13:00:36.176210  4019 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:36.147446  3698 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:36.257995  3698 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:36.260535  3980 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:36.261526  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 66042.1us
I20260709 13:00:36.262821  3700 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:36.263273  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:36.263866  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:36.264192  4036 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:36.264763  4036 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:36.266512  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 446) took 1511 ms. Trace:
W20260709 13:00:36.231472  1539 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1754) took 4361 ms (client timeout 2074 ms). Trace:
I20260709 13:00:36.231472  1537 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:36.268962  1688 rpcz_store.cc:276] 0709 13:00:34.755495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.755606 (+   111us) service_pool.cc:224] Handling call
0709 13:00:36.257322 (+1501716us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b54000aa5d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:36.266427 (+  9105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244180736}
W20260709 13:00:36.181015  3833 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.254s	user 0.036s	sys 0.062s
W20260709 13:00:36.190371  2200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.014s	sys 0.006s
I20260709 13:00:36.192145  1692 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:36.276872  1537 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:36.190140  1616 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
W20260709 13:00:36.273412  1539 rpcz_store.cc:269] 0709 13:00:31.869928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:31.886646 (+ 16718us) service_pool.cc:224] Handling call
0709 13:00:36.231192 (+4344546us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54001e4454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:36.231371 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3696838016}
I20260709 13:00:36.149147  3884 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.150177  4030 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:36.181766  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:36.202453  1582 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:36.248755  4019 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:36.251597  3739 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:36.176272  1683 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:36.254030  3724 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:00:36.254557  2228 consensus_meta.cc:287] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.522s	user 0.001s	sys 0.008s
W20260709 13:00:36.255357  3981 consensus_meta.cc:287] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.860s	user 0.007s	sys 0.005s
I20260709 13:00:36.186277  3882 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:36.271086  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 291280us
I20260709 13:00:36.219497  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:36.275489  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:36.265195  4036 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.190591  1687 rpcz_store.cc:276] 0709 13:00:34.482017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.482189 (+   172us) service_pool.cc:224] Handling call
0709 13:00:34.482607 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:34.482824 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:34.482847 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:34.555139 (+ 72292us) log.cc:844] Serialized 26 byte log entry
0709 13:00:35.576401 (+1021262us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:35.576459 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.576551 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:36.150717 (+574166us) raft_consensus.cc:1685] finished
0709 13:00:36.150741 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:36.150939 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1020848,"threads_started":1,"wal-append.queue_time_us":1171172}
I20260709 13:00:36.427805  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:36.428172  1509 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:36.171607  4017 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:36.243278  3717 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:36.138512  3957 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:36.230756  4029 log.cc:927] Time spent T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:00:36.172894  3984 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:36.248471  4016 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 13:00:36.201958  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 642184us
W20260709 13:00:36.451335  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:36.265312  4033 log.cc:927] Time spent T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:00:32.848652   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:00:36.313208  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:36.299116  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 371) took 2769 ms. Trace:
I20260709 13:00:36.267227  3676 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.292479  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:36.463297  1535 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:36.463892  3930 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:36.465814  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:36.466656  1542 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:36.467554  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:36.467931  4007 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.001s
I20260709 13:00:36.431216  3981 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4.
I20260709 13:00:36.225499  4038 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:36.366328  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:36.387321  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 184922us
I20260709 13:00:36.266311  3822 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:36.267685  4001 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:36.225744  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:36.266247  4009 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:36.203254  4044 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:36.395584  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:36.290670  1537 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:36.459528  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:36.282275  4005 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.494191  4005 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:36.194667  4031 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.496564  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1806) took 1094 ms. Trace:
I20260709 13:00:36.496938  1618 rpcz_store.cc:276] 0709 13:00:35.401827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.723001 (+321174us) service_pool.cc:224] Handling call
0709 13:00:36.496380 (+773379us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b5400087ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:36.496505 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69503872}
I20260709 13:00:36.473639  4038 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.497710  4038 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:36.498065  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:36.498687  1618 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:36.499622  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:36.500154  1618 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:36.500878  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:36.506069  1611 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:36.506760  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:36.507860  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:36.508426  1617 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:36.511018  3957 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:36.511660  3957 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.512200  3957 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:36.511746  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:36.518232  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:36.518958  1691 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:36.520007  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:36.428448  1687 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:36.431083  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:36.528020  1686 raft_consensus.cc:2368] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6.
I20260709 13:00:36.509153  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:36.393204  1610 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:36.434113  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:36.298328  4030 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.299420  3820 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:36.284932  4003 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.449455  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:36.396914  3833 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:36.391268  3724 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:36.397595  2228 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:36.297976  4022 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.155s
I20260709 13:00:36.430938  3800 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.457913  1613 rpcz_store.cc:276] 0709 13:00:33.529721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.529875 (+   154us) service_pool.cc:224] Handling call
0709 13:00:33.590291 (+ 60416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:33.594000 (+  3709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.298858 (+2704858us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b54002bd750. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:36.299030 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4416384}
I20260709 13:00:36.471004  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:36.435205  3843 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:36.300192  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:36.450753  3984 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.432875  3916 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:00:36.480569  1615 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:36.484750  1616 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:36.451741  1685 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:36.429929  4046 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:00:36.488773  1537 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:36.489388  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:36.188941  4040 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:36.528210  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:36.917879  2237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 526) took 1496 ms (client timeout 1561 ms). Trace:
W20260709 13:00:37.170930  1688 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1264) took 1631 ms (client timeout 1845 ms). Trace:
I20260709 13:00:36.735553  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 521) took 1372 ms. Trace:
I20260709 13:00:36.701530  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:36.711509  1437 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55374 (request call id 320) took 16457 ms (client timeout 15000 ms). Trace:
I20260709 13:00:36.567239  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.170323  1687 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:00:36.728080  1692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1252) took 3383 ms (client timeout 2078 ms). Trace:
I20260709 13:00:36.153509  1684 rpcz_store.cc:276] 0709 13:00:34.742109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:34.750578 (+  8469us) service_pool.cc:224] Handling call
0709 13:00:34.758504 (+  7926us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:34.758967 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:34.758999 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:34.759546 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:00:35.615330 (+855784us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:35.619352 (+  4022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:35.633865 (+ 14513us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:35.974170 (+340305us) raft_consensus.cc:1685] finished
0709 13:00:35.974206 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:36.006147 (+ 31941us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"thread_start_us":816062,"threads_started":1,"wal-append.queue_time_us":733565}
W20260709 13:00:36.728981  1689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 453) took 1971 ms (client timeout 1554 ms). Trace:
I20260709 13:00:37.172415  1617 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:36.699743  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.197940  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:36.700253  2201 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:36.739729  1440 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:36.622787  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:36.631349  1534 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:37.284399  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.211648  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.285394  1613 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:37.285861  1440 catalog_manager.cc:5697] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:00:37.203189  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:36.998792  1509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.004s	sys 0.023s
W20260709 13:00:37.287097  1509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.004s	sys 0.028s
I20260709 13:00:37.285809  1538 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
W20260709 13:00:37.288869  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 588662us
I20260709 13:00:36.477008  4050 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:36.696569  1614 raft_consensus.cc:2410] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:36.549225  1618 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:37.303901  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:36.559234  4049 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.123s	user 0.003s	sys 0.000s
W20260709 13:00:37.310542  1615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1807) took 1908 ms (client timeout 1656 ms). Trace:
W20260709 13:00:37.310837  1615 rpcz_store.cc:269] 0709 13:00:35.402094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.555164 (+153070us) service_pool.cc:224] Handling call
0709 13:00:37.310400 (+1755236us) spinlock_profiling.cc:243] Waited 913 ms on lock 0x7b5400190854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:37.310487 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2098781568}
W20260709 13:00:36.739908  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:36.648248  3948 consensus_meta.cc:287] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.882s	user 0.012s	sys 0.000s
I20260709 13:00:37.169021  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:37.319720  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 150771us
I20260709 13:00:37.321066  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.321888  1615 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:37.322728  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.323825  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.324518  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.324873  1615 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 2.
I20260709 13:00:37.172624  2235 rpcz_store.cc:276] 0709 13:00:35.363286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.364141 (+   855us) service_pool.cc:224] Handling call
0709 13:00:36.735346 (+1371205us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54002a3fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:36.735475 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583240960}
W20260709 13:00:37.197510  1692 rpcz_store.cc:269] 0709 13:00:33.344173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:33.515912 (+171739us) service_pool.cc:224] Handling call
0709 13:00:36.727772 (+3211860us) spinlock_profiling.cc:243] Waited 2.67 s on lock 0x7b54000f5854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:36.727942 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6152089472}
W20260709 13:00:37.288555  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.325625  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:37.174249  1688 rpcz_store.cc:269] 0709 13:00:35.539767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.277734 (+737967us) service_pool.cc:224] Handling call
0709 13:00:37.170867 (+893133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.326339  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:37.203989  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:37.203275  1689 rpcz_store.cc:269] 0709 13:00:34.757402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.728746 (+1971344us) service_pool.cc:211] Skipping call since client already timed out
0709 13:00:36.728893 (+   147us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 13:00:37.289140  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 84468.2us
I20260709 13:00:37.167444  3346 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.202579  1543 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:36.560829  4019 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:37.288153  1686 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:37.170641  2237 rpcz_store.cc:269] 0709 13:00:35.422048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.586098 (+164050us) service_pool.cc:224] Handling call
0709 13:00:36.917613 (+1331515us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b54002a35d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:36.917792 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23467392}
I20260709 13:00:36.516494  3822 raft_consensus.cc:697] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:37.293602  3795 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:36.631858  3882 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:37.293942  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.171595  3820 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:00:37.205804  2228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 523) took 1816 ms (client timeout 1970 ms). Trace:
I20260709 13:00:36.634508  4036 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:36.741127  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1796) took 1447 ms. Trace:
I20260709 13:00:37.287926  1440 catalog_manager.cc:5697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 3 to 4, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:00:37.077064  3884 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.689s	user 0.012s	sys 0.000s
I20260709 13:00:37.177556  3797 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.695524  3935 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.506776  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1810) took 1103 ms. Trace:
I20260709 13:00:37.296237  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.355566  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.355906  1538 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:37.354575  1509 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:37.356863  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:37.173347  1437 rpcz_store.cc:269] 0709 13:00:20.254043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:20.254282 (+   239us) service_pool.cc:224] Handling call
0709 13:00:36.711391 (+16457109us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:30.868046 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:30.868754 (+   708us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:30.868800 (+    46us) write_op.cc:435] Acquired schema lock
0709 13:00:30.868818 (+    18us) tablet.cc:662] Decoding operations
0709 13:00:31.034730 (+165912us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:31.034822 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:31.034852 (+    30us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:00:31.091535 (+ 56683us) tablet.cc:701] Row locks acquired
0709 13:00:31.091588 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:00:31.091764 (+   176us) write_op.cc:270] Start()
0709 13:00:31.091929 (+   165us) write_op.cc:276] Timestamp: P: 1783602031091698 usec, L: 0
0709 13:00:31.091949 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:00:31.133982 (+ 42033us) log.cc:844] Serialized 4795 byte log entry
0709 13:00:31.639538 (+505556us) op_driver.cc:464] REPLICATION: finished
0709 13:00:31.740828 (+101290us) write_op.cc:301] APPLY: starting
0709 13:00:31.742476 (+  1648us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:31.761803 (+ 19327us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:31.801934 (+ 40131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:31.888518 (+ 86584us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:31.888825 (+   307us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:00:31.888876 (+    51us) write_op.cc:312] APPLY: finished
0709 13:00:31.889204 (+   328us) log.cc:844] Serialized 89 byte log entry
0709 13:00:31.889380 (+   176us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:31.889550 (+   170us) write_op.cc:454] Released schema lock
0709 13:00:31.889604 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:31.890094 (+   490us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:31.890122 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":284842,"reported_tablets":25,"child_traces":[["op",{"apply.queue_time_us":87847,"apply.run_cpu_time_us":5103,"apply.run_wall_time_us":159159,"num_ops":12,"prepare.queue_time_us":172172,"prepare.run_cpu_time_us":11242,"prepare.run_wall_time_us":318686,"raft.queue_time_us":266072,"raft.run_cpu_time_us":16012,"raft.run_wall_time_us":125239,"replication_time_us":502022,"thread_start_us":263676,"threads_started":4,"wal-append.queue_time_us":97453,"wal-append.run_cpu_time_us":47112,"wal-append.run_wall_time_us":1845244}]]}
I20260709 13:00:37.357175  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:37.298527  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2237) took 1110 ms. Trace:
I20260709 13:00:37.300762  4056 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:37.199807  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2089) took 1012 ms. Trace:
I20260709 13:00:36.533818  1691 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:37.284523  4040 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.304713  1618 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:37.367966  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.369014  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:36.489946  4044 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:36.885442  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.204260  1616 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:37.372750  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:37.301999  4038 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.804s	user 0.009s	sys 0.002s
I20260709 13:00:37.171839  1540 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:37.348601  3795 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:37.377317  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.378074  1618 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:37.378202  1540 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:37.379511  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.380256  1540 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 1.
I20260709 13:00:37.293342  2201 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:36.561235  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1266) took 1021 ms. Trace:
I20260709 13:00:37.331542  1537 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:00:37.381336  1537 pending_rounds.cc:85] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 2
I20260709 13:00:37.381635  1537 pending_rounds.cc:107] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:00:37.381862  1537 raft_consensus.cc:2889] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:37.177583  3833 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:36.718111  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 520) took 1355 ms. Trace:
I20260709 13:00:36.647176  4032 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:36.850359  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1798) took 1482 ms. Trace:
I20260709 13:00:36.562304  3866 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.333997  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2086) took 1163 ms. Trace:
I20260709 13:00:37.335266  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:37.208853  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.207090  3843 raft_consensus.cc:697] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:37.303076  3727 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:36.725826  3901 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:37.206537  4005 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:37.341348  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.335817  3974 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:37.342875  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:37.344936  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:37.359622  1534 rpcz_store.cc:276] 0709 13:00:36.189154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.191022 (+  1868us) service_pool.cc:224] Handling call
0709 13:00:37.298449 (+1107427us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.308051  3957 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:37.407538  1536 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:37.361862  3797 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.332728  4021 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:37.351684  1440 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 4 to 6, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:37.200604  4003 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.424837  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:37.360514  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:37.433730  1689 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:37.413552  3727 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:36.905586  4045 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.475s	user 0.005s	sys 0.000s
I20260709 13:00:37.362715  1687 rpcz_store.cc:276] 0709 13:00:36.187494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.527968 (+340474us) service_pool.cc:224] Handling call
0709 13:00:37.199737 (+671769us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.418895  3974 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:37.373380  1686 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:37.365371  3599 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:37.374727  1691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 459) took 1503 ms (client timeout 1938 ms). Trace:
W20260709 13:00:37.440606  1691 rpcz_store.cc:269] 0709 13:00:35.871598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.519819 (+648221us) service_pool.cc:224] Handling call
0709 13:00:37.374638 (+854819us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.441779  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 458) took 1737 ms. Trace:
I20260709 13:00:37.351730  3884 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:37.390196  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 214839us
I20260709 13:00:37.316211  4050 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.351909  3882 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:37.352198  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:37.381973  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 88733.2us
I20260709 13:00:37.442075  1691 rpcz_store.cc:276] 0709 13:00:35.704310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.441119 (+1736809us) service_pool.cc:224] Handling call
0709 13:00:37.441323 (+   204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.441563 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:37.441579 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:37.441606 (+    27us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:37.441630 (+    24us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.441667 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.441740 (+    73us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.393790  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.465853  1540 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:37.404117  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.362679  4054 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:37.348105  2228 rpcz_store.cc:269] 0709 13:00:35.384883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.630684 (+245801us) service_pool.cc:224] Handling call
0709 13:00:37.201314 (+1570630us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.407021  1692 rpcz_store.cc:276] 0709 13:00:36.170933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.326118 (+1155185us) service_pool.cc:224] Handling call
0709 13:00:37.333931 (+  7813us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.472235  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.472697  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.408852  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.343415  3724 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.473533  1613 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:37.391080  3833 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.348306  4019 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:37.377853  4044 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.351214  3948 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:37.407275  1543 raft_consensus.cc:2376] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5. Re-sending same reply.
I20260709 13:00:37.479182  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.479660  1543 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:37.307726  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1804) took 1906 ms. Trace:
I20260709 13:00:37.393193  4022 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:37.414227  1541 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:37.486045  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.486344  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.346086  1615 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:37.418566  1542 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:37.362128  3935 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.421731  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.392006  1683 rpcz_store.cc:276] 0709 13:00:35.540298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.431969 (+891671us) service_pool.cc:224] Handling call
0709 13:00:36.561163 (+129194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.425045  1440 catalog_manager.cc:5697] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 2 to 4, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:37.489073  1536 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:37.487164  1543 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
W20260709 13:00:37.359105  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.350178  1612 rpcz_store.cc:276] 0709 13:00:35.293206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.293344 (+   138us) service_pool.cc:224] Handling call
0709 13:00:35.525828 (+232484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.661593 (+135765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.740792 (+1079199us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b5400102550. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:36.741021 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441840896}
I20260709 13:00:37.436096  3795 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.359393  1611 rpcz_store.cc:276] 0709 13:00:35.403092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.147789 (+744697us) service_pool.cc:224] Handling call
0709 13:00:36.505676 (+357887us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:00:36.505865 (+   189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:36.506534 (+   669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.506721 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28110208}
I20260709 13:00:37.490089  1440 catalog_manager.cc:5697] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 2 to 4, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:37.434330  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.202170  3800 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:36.698805  4031 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:37.408420  3866 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.439749  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2084) took 1269 ms. Trace:
I20260709 13:00:37.464043  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 457) took 1936 ms. Trace:
I20260709 13:00:37.496948  1538 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:37.403833  2231 rpcz_store.cc:276] 0709 13:00:35.362978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.372237 (+  9259us) service_pool.cc:224] Handling call
0709 13:00:35.453042 (+ 80805us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.651910 (+198868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:35.656183 (+  4273us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:35.665614 (+  9431us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:35.669600 (+  3986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.717727 (+1048127us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b540036a6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:36.717799 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:36.717999 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":394182528}
I20260709 13:00:37.437785  4036 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:37.497301  1690 rpcz_store.cc:276] 0709 13:00:35.527843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.531474 (+  3631us) service_pool.cc:224] Handling call
0709 13:00:35.531862 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:35.537898 (+  6036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:35.537942 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:35.538540 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 13:00:36.193804 (+655264us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:36.193843 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.193910 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:36.905163 (+711253us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b240025af78. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c8859c4 00007fe08c883acb 00007fe08ca3edc9 00007fe08ca3ea55 00007fe08ca3e09d 00007fe08f67010e 00007fe08f68788c 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221
0709 13:00:37.443163 (+538000us) raft_consensus.cc:1685] finished
0709 13:00:37.443190 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.463953 (+ 20763us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5555,"raft.queue_time_us":12637,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":18376,"spinlock_wait_cycles":54102784,"thread_start_us":622196,"threads_started":1,"wal-append.queue_time_us":735105}
I20260709 13:00:37.407158  3820 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.499505  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:37.499626  3820 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:37.500391  3800 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:37.500815  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.501704  1539 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:37.502290  3866 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:37.502686  4031 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:37.492465  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.503383  1612 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:37.475914  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.476095  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.505618  1618 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:37.477118  1540 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:37.492494  1683 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:37.424484  4021 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.370538  4056 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.487905  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:37.414191  3822 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.440444  3599 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.490967  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.491199  1582 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:37.393779  4038 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.469506  4054 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:37.492895  1692 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:00:37.472494  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:37.497174  1686 rpcz_store.cc:276] 0709 13:00:36.170403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.528605 (+358202us) service_pool.cc:224] Handling call
0709 13:00:37.439688 (+911083us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.498025  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.498162  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.552376  1536 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:37.506760  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.504662  4022 raft_consensus.cc:697] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:37.507895  1584 leader_election.cc:336] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.784s (SENT)
I20260709 13:00:37.553332  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.468927  3882 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:37.488030  4019 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.510752  3935 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:37.511325  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.512858  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.549670  3599 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:37.514920  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.474905  1684 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 2.
I20260709 13:00:37.486634  3724 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:37.542230  1543 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:37.477411  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.592541  2233 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
W20260709 13:00:37.553591  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 239086us
I20260709 13:00:37.553719  1540 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
W20260709 13:00:37.556202  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 48391.8us
I20260709 13:00:37.558367  4063 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:37.504287  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.598376  1612 raft_consensus.cc:2368] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5.
I20260709 13:00:37.598079  4063 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.599839  1612 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:37.600584  4063 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:37.514592  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1282) took 1028 ms. Trace:
I20260709 13:00:37.601735  1618 rpcz_store.cc:276] 0709 13:00:36.486370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.513512 (+1027142us) service_pool.cc:224] Handling call
0709 13:00:37.513859 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.514100 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:37.514146 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:37.514209 (+    63us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:37.514269 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.514349 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.514495 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.603736  1618 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:37.604007  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.604617  1540 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:37.604969  1618 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:37.605966  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:37.606175  1618 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:37.607007  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:37.607183  1618 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:37.566346  1536 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:37.608115  3800 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.608748  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.608613  3800 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.609827  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.610678  3800 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:37.576972  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:37.612008  3924 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:37.612615  3924 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:37.529081  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.613643  3924 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:37.615337  1611 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:37.481518  1610 rpcz_store.cc:276] 0709 13:00:35.401219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.112306 (+711087us) service_pool.cc:224] Handling call
0709 13:00:36.192383 (+ 80077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.188372 (+995989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.302960 (+114588us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:37.591079  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 270670us
I20260709 13:00:37.617074  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:37.616020  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 63498.5us
I20260709 13:00:37.617887  3935 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:37.618211  1658 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:37.620661  4005 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:37.621091  4005 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.621517  4005 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.623222  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.619278  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:37.625378  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 134219us
I20260709 13:00:37.619776  2201 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:37.551103  4054 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:37.627084  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:37.628418  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.617079  2203 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:00:37.635600  1614 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1795) took 2343 ms (client timeout 2166 ms). Trace:
W20260709 13:00:37.636238  1614 rpcz_store.cc:269] 0709 13:00:35.291778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.291894 (+   116us) service_pool.cc:224] Handling call
0709 13:00:37.635348 (+2343454us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54003c96d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:37.635517 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3227320576}
I20260709 13:00:37.636963  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:37.476660  3843 consensus_queue.cc:237] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:37.631165  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 273051us
I20260709 13:00:37.633625  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:37.635035  1688 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.638248  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.561717  4040 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:37.640337  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:37.641682  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.642069  4003 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:37.642827  4050 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:37.643404  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.644908  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.648576  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.647598  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:37.649610  2203 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:37.649245  1543 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 1.
I20260709 13:00:37.629900  4063 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:37.656464  2203 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:37.660267  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:37.660987  1534 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:37.654315  4060 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:37.662245  4060 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.662583  4060 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.657239  3896 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:37.657661  2230 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:37.665880  3896 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:37.666307  3896 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:37.667162  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.405104  1609 rpcz_store.cc:276] 0709 13:00:35.348454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.351839 (+  3385us) service_pool.cc:224] Handling call
0709 13:00:35.409449 (+ 57610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:35.447246 (+ 37797us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:35.535261 (+ 88015us) log.cc:844] Serialized 14 byte log entry
0709 13:00:36.274607 (+739346us) spinlock_profiling.cc:243] Waited 9.31 ms on lock 0x7b18003c5da0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a 00007fe08faf6a2a
0709 13:00:36.588030 (+313423us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:36.620988 (+ 32958us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:36.689446 (+ 68458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:36.780720 (+ 91274us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:36.830526 (+ 49806us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11219,"prepare.queue_time_us":24994,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":10406,"raft.queue_time_us":38738,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":91,"spinlock_wait_cycles":21412224,"thread_start_us":203265,"threads_started":1,"wal-append.queue_time_us":1019690}
I20260709 13:00:37.668334  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.635023  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2088) took 1446 ms. Trace:
I20260709 13:00:37.669673  1684 rpcz_store.cc:276] 0709 13:00:36.183055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.307953 (+1124898us) service_pool.cc:224] Handling call
0709 13:00:37.629108 (+321155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.670281  3802 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:37.670789  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:37.670810  3802 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.671579  1684 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:37.671883  4067 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:37.672389  2236 raft_consensus.cc:2410] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
W20260709 13:00:37.675283  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.640340  3800 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:37.673170  1613 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:37.677044  2233 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:00:37.678953  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:37.680794  1536 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:37.680471  1509 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:37.684652  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:37.691068  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:37.691056  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:37.692364  1688 raft_consensus.cc:2410] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:00:37.692576  1683 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:37.693002  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:37.561334  3974 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.694833  3884 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:37.695693  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:37.697269  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:37.697969  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.698786  1536 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:37.699340  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2256) took 1065 ms. Trace:
I20260709 13:00:37.699741  1536 rpcz_store.cc:276] 0709 13:00:36.634331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.697659 (+1063328us) service_pool.cc:224] Handling call
0709 13:00:37.699284 (+  1625us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.700899  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.701891  1536 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:37.657395  4063 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.673849  4035 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:37.703593  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.655059  4055 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:37.673324  4050 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:37.670812  4058 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:37.715191  4058 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:37.716199  4058 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.705869  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:37.697660  3957 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:37.672665  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:37.709301  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.711230  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:37.678308  3800 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:37.714184  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.676935  4017 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:37.697854  4031 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:37.747730  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.747773  1536 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:37.709388  4035 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.750025  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:37.713367  4055 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.687202  4062 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:37.749629  2235 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:37.756821  1689 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:37.750939  3896 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.719997  4040 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:37.758564  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 52793us
I20260709 13:00:37.752600  3346 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:37.759617  4019 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:37.719682  4050 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.778846  4050 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:37.760843  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 133822us
I20260709 13:00:37.708994  4063 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:37.786000  4021 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:37.762156  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.787024  1536 raft_consensus.cc:2410] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:37.765694  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2090) took 1276 ms. Trace:
I20260709 13:00:37.758313  4038 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:37.766753  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.747051  4056 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:37.751611  3800 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:37.770430  4067 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:37.751245  3948 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:37.752857  1684 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:37.802981  2237 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
W20260709 13:00:37.805541  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:37.807245  4063 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:37.760547  4031 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:37.761485  4035 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:37.778331  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1257) took 2241 ms. Trace:
I20260709 13:00:37.767077  4062 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:37.781188  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.781910  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:37.783955  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:37.770991  3984 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.000s
I20260709 13:00:37.809096  3727 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:37.788007  1689 rpcz_store.cc:276] 0709 13:00:36.489130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.413788 (+924658us) service_pool.cc:224] Handling call
0709 13:00:37.765617 (+351829us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.788398  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:37.790208  1615 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:37.790782  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.791538  1614 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:37.766177  4055 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:37.747663  3957 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.779518  1683 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:37.834759  1536 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:37.837092  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2253) took 1203 ms. Trace:
I20260709 13:00:37.779418  4054 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.780566  4040 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.838163  1538 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:37.839742  1683 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:37.840272  4005 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:37.841993  1685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1265) took 2302 ms (client timeout 1766 ms). Trace:
I20260709 13:00:37.811436  3797 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:37.842466  1685 rpcz_store.cc:269] 0709 13:00:35.540030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.262963 (+722933us) service_pool.cc:224] Handling call
0709 13:00:37.841813 (+1578850us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54000aafd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:37.841926 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396470528}
I20260709 13:00:37.845202  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:37.706812  4022 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.835457  1616 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:37.815313  1687 rpcz_store.cc:276] 0709 13:00:35.536638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.440353 (+1903715us) service_pool.cc:224] Handling call
0709 13:00:37.440647 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.503910 (+ 63263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:37.503957 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:37.504018 (+    61us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:37.504094 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.778078 (+273984us) spinlock_profiling.cc:243] Waited 62.8 ms on lock 0x7b540014a354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:37.778125 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.778265 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5811,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":13814,"spinlock_wait_cycles":144493952}
I20260709 13:00:37.810186  4031 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:37.813248  1584 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.542s (SENT)
I20260709 13:00:37.835482  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1291) took 1132 ms. Trace:
I20260709 13:00:37.848877  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:37.843945  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:37.837549  1536 rpcz_store.cc:276] 0709 13:00:36.633284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.790547 (+1157263us) service_pool.cc:224] Handling call
0709 13:00:37.837025 (+ 46478us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.859081  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.849639  3346 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:00:37.811394  3896 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:37.834553  4062 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.863332  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:37.855424  1615 rpcz_store.cc:276] 0709 13:00:36.703339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.527363 (+824024us) service_pool.cc:224] Handling call
0709 13:00:37.789693 (+262330us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:00:37.790002 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.835207 (+ 45205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.835417 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24094080}
I20260709 13:00:37.851209  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:37.864694  1687 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:37.865029  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:37.844357  3957 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.866088  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.866500  4040 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:37.850066  1610 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:37.860209  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:37.853329  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:37.870854  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 57662.8us
I20260709 13:00:37.868021  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:37.868615  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:37.860445  1536 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:37.901631  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:37.902628  1536 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:37.904043  4021 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:37.904999  4021 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:37.905478  4021 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:37.905884  4021 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:37.910513  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2252) took 1277 ms. Trace:
I20260709 13:00:37.860038  4075 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:37.911809  4075 raft_consensus.cc:697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:37.912784  4075 consensus_queue.cc:237] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:37.914645  3599 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:37.911055  1536 rpcz_store.cc:276] 0709 13:00:36.633001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.903237 (+1270236us) service_pool.cc:224] Handling call
0709 13:00:37.903590 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.909997 (+  6407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:37.910044 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:37.910114 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:37.910173 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.910262 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.910435 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5748,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109}
I20260709 13:00:37.916846  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2265) took 1279 ms. Trace:
W20260709 13:00:37.911316  4077 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:00:37.929831  1687 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:37.917222  1536 rpcz_store.cc:276] 0709 13:00:36.637024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.916111 (+1279087us) service_pool.cc:224] Handling call
0709 13:00:37.916405 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.916553 (+   148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:37.916579 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:37.916619 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:37.916647 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:37.916697 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:37.916792 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:37.845520  3984 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:37.852723  4076 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:37.850688  3727 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:00:38.033725  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:37.986589  3797 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:38.036553  3797 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.037774  4067 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.038719  3800 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.038720  3797 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:38.034363  1687 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.040910  3800 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:38.043912  4050 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:38.047128  1687 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:37.855289  3833 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:38.051112  2235 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:38.026641  3599 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:37.924224  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:38.058669  1685 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:37.923238  3896 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:38.059759  3896 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:38.060762  3896 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:38.061790  1657 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.121s (SENT) [suppressed 57 similar messages]
I20260709 13:00:38.062135  3896 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:38.050014  1582 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:38.045008  1684 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:38.031329  4076 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.073925  4076 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:38.073935  1617 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:38.074872  3833 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:38.028683  1534 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:38.076298  3957 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.078474  1534 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:38.078787  1614 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:38.082115  4050 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:38.034453  4063 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:38.069998  4054 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:38.072495  3984 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:38.073151  4005 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:38.073500  3727 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:00:38.083894  4030 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.873s	user 0.002s	sys 0.008s
I20260709 13:00:38.084017  3727 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:38.084340  4030 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.085454  1540 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.085012  3727 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:38.081797  4060 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:38.067523  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 433857us
W20260709 13:00:38.093915  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.054409  4040 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:38.092156  1658 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:38.097450  4060 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:38.101095  1540 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:38.106003  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:38.109403  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:38.109736  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:38.113328  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:38.114898  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:38.117914  1539 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2240) took 1718 ms (client timeout 2013 ms). Trace:
I20260709 13:00:38.118047  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:38.118432  1539 rpcz_store.cc:269] 0709 13:00:36.400039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.405593 (+  5554us) service_pool.cc:224] Handling call
0709 13:00:38.117727 (+1712134us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54000eddd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:38.117856 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756751104}
I20260709 13:00:38.125484  2203 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:38.120078  1584 leader_election.cc:336] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.812s (SENT)
I20260709 13:00:38.128054  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:38.130369  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.103478  4040 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:38.087393  4076 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:38.153412  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:38.128388  4005 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:38.119280  4017 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:38.157693  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.125382  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.133903  4036 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.126398  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.157811  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:38.173585  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2250) took 1541 ms. Trace:
I20260709 13:00:38.132825  3795 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:38.102644  4055 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.125964  3346 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:00:38.118781  4081 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:38.134493  4063 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:38.157295  4056 consensus_queue.cc:1059] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 13:00:38.171530  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:38.199195  1543 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:38.174069  1541 rpcz_store.cc:276] 0709 13:00:36.632355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.632566 (+   211us) service_pool.cc:224] Handling call
0709 13:00:36.632896 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.486172 (+853276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.173295 (+687123us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b540030a7d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:38.173505 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1796404608}
I20260709 13:00:38.142705  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:38.143666  3924 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:38.178088  2233 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:38.178922  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:38.189136  2234 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:38.190483  4055 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:38.204386  1584 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:38.180694  4044 consensus_meta.cc:287] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.692s	user 0.010s	sys 0.000s
I20260709 13:00:38.178525  3820 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.175709  4005 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:38.219283  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:38.221460  1541 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:38.222648  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:38.223239  1541 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.223278  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:38.227030  1584 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:00:38.229200  1508 connection.cc:256] RPC call timeout handler was delayed by -2.0138619389999803s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15610000000000002s for a response
I20260709 13:00:38.214345  4060 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
W20260709 13:00:38.229763  1508 connection.cc:256] RPC call timeout handler was delayed by -1.6457619389999536s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.197s for a response
I20260709 13:00:38.230208  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.208729  3942 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:38.150902  4035 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.196616  1610 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
W20260709 13:00:38.231746  1508 connection.cc:256] RPC call timeout handler was delayed by -1.6295619389999842s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1988s for a response
I20260709 13:00:38.215406  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:38.214727  3924 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.230822  2231 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:38.261499  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:38.228686  4084 log.cc:927] Time spent T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.113s	user 0.002s	sys 0.000s
I20260709 13:00:38.211001  4058 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 13:00:38.264148  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:38.265066  1616 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:38.238294  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:38.239980  1508 connection.cc:256] RPC call timeout handler was delayed by -1.548561938999967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
I20260709 13:00:38.212177  3795 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:38.235893  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:38.216389  4017 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.243127  1658 leader_election.cc:400] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6.
I20260709 13:00:38.273792  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:38.274538  1616 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.269026  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:38.245304  3727 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:38.245882  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.232786  3942 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:38.246853  4035 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:38.205492  4046 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 13:00:38.220265  4044 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.295425  4044 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:38.244328  4056 raft_consensus.cc:3055] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:38.296403  4056 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:38.248198  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:38.298307  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:38.298997  1534 raft_consensus.cc:2368] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5.
I20260709 13:00:38.230002  4062 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:38.211161  4073 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:00:38.219259  4040 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:38.231688  4063 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:38.304729  4062 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:38.305061  4063 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.305611  4063 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:38.306160  1688 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1260) took 2768 ms (client timeout 2882 ms). Trace:
I20260709 13:00:38.306231  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:38.306702  1688 rpcz_store.cc:269] 0709 13:00:35.537583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.332593 (+1795010us) service_pool.cc:224] Handling call
0709 13:00:38.305970 (+973377us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54000ab9d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:38.306107 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":513151360}
I20260709 13:00:38.307559  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:38.297277  4056 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:38.230989  3820 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.249168  4005 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.251473  3957 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:38.255050  3935 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:38.257678  4030 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:38.257994  1509 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:38.259722  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:38.260707  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:38.262835  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:38.271416  2203 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.274030  1658 leader_election.cc:403] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:38.245257  4093 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:38.267707  1612 raft_consensus.cc:2410] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:38.276414  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:38.261344  4076 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:38.284972  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.285882  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1774) took 2399 ms. Trace:
I20260709 13:00:38.288645  3802 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:38.299716  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:38.262871  3924 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:38.321383  2236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 524) took 2924 ms (client timeout 2882 ms). Trace:
W20260709 13:00:38.334178  2236 rpcz_store.cc:269] 0709 13:00:35.397321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.800033 (+402712us) service_pool.cc:224] Handling call
0709 13:00:38.320191 (+2520158us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54002f49d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:38.321306 (+  1115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4159232896}
I20260709 13:00:38.323508  1618 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:38.310148  4038 consensus_queue.cc:1175] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "83f6f5810950417195f707ff308a8125" responder_term: 3 status { last_received { term: 3 index: 3 } last_committed_idx: 2 last_received_current_leader { term: 3 index: 3 } }
W20260709 13:00:38.323498  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 65599.2us
I20260709 13:00:38.342569  1509 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:38.343506  4040 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:38.343777  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:38.323943  1615 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.346882  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.347980  4056 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.325050  4093 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:38.349987  4093 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:38.325893  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 49580.3us
W20260709 13:00:38.350894  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 79969.4us
I20260709 13:00:38.351570  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:38.352777  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:38.353490  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 110397us
I20260709 13:00:38.327440  1534 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.274621  3346 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.354976  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:38.355605  1687 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:00:38.356858  1541 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:38.354226  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:38.357816  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:38.282564  3942 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:38.363725  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.311292  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:38.364440  1539 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:38.316234  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 86094.1us
I20260709 13:00:38.365836  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.366449  1508 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:38.366935  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.367154  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.368155  1584 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.321312  4067 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:00:38.341534  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:38.333186  1617 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1289) took 1728 ms (client timeout 1574 ms). Trace:
I20260709 13:00:38.354240  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:38.359145  1615 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:38.359855  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:38.302327  1616 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:38.361675  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.326926  1537 rpcz_store.cc:276] 0709 13:00:35.886079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:35.886357 (+   278us) service_pool.cc:224] Handling call
0709 13:00:36.231336 (+344979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.390868 (+1159532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.285670 (+894802us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b54001e4450. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:38.285820 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":792479616}
I20260709 13:00:38.277168  3727 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.283242  3916 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:38.477662  1507 leader_election.cc:336] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.528s (SENT)
I20260709 13:00:38.476680  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:38.458853  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 90751.5us
I20260709 13:00:38.478951  2235 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:38.479229  1584 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.475795  1618 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:38.475246  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:38.289170  4021 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:38.479489  1507 leader_election.cc:336] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.549s (SENT)
I20260709 13:00:38.493467  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.494253  1537 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:38.474978  1541 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:38.330142  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:38.495568  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:38.456297  3800 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.481571  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:38.476112  3727 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:38.489996  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 135826us
W20260709 13:00:38.490484  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.327304  3866 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6.
I20260709 13:00:38.491734  2234 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:38.470917  1534 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:38.333555  4044 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:38.373515  2203 leader_election.cc:400] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5.
I20260709 13:00:38.323676  4030 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:38.320676  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:38.478464  1617 rpcz_store.cc:269] 0709 13:00:36.604546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.202826 (+598280us) service_pool.cc:224] Handling call
0709 13:00:38.332883 (+1130057us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b54003c8cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:38.333080 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1804173184}
I20260709 13:00:38.278249  4017 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.370201  4035 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:38.364566  4054 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:38.446321  1687 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:00:38.370293  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:38.503386  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.459280  3833 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:38.504033  1539 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:38.552240  1539 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:38.552790  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1786) took 1256 ms. Trace:
I20260709 13:00:38.326418  3957 raft_consensus.cc:2764] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:00:38.326327  3820 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:38.504747  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:38.556944  1537 raft_consensus.cc:2368] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7.
W20260709 13:00:38.372813  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.371455  3802 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:38.505527  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:38.407393  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:38.456735  3599 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:38.476414  4076 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:38.368804  4038 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.362447  4081 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:38.542068  4050 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.507332  1690 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:38.510751  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:38.326771  2229 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:38.494889  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:38.536720  2203 leader_election.cc:403] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:38.549010  4035 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:38.614046  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 245645us
I20260709 13:00:38.492972  4073 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.493335  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:38.539579  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:38.618979  1534 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:38.476824  3984 consensus_queue.cc:1059] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:00:38.548997  1686 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.550922  1610 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
W20260709 13:00:38.553597  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 74432.4us
I20260709 13:00:38.631701  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.328087  4063 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.475922  4075 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:38.596793  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:38.593529  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:38.601524  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 652) took 1091 ms. Trace:
I20260709 13:00:38.595723  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:38.606860  2201 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.551095  3833 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:38.590787  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2087) took 2409 ms. Trace:
I20260709 13:00:38.613497  3957 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:38.644454  1691 rpcz_store.cc:276] 0709 13:00:36.181075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.477921 (+1296846us) service_pool.cc:224] Handling call
0709 13:00:37.478191 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.626302 (+148111us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:37.626333 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:37.626373 (+    40us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:37.626407 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.590420 (+964013us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54002cd654. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:38.590493 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:38.590694 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340063616}
I20260709 13:00:38.548364  4054 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:38.615814  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 105149us
I20260709 13:00:38.620193  3866 raft_consensus.cc:2764] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:00:38.621044  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.543654  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.651659  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.652443  2234 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:38.629952  3822 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.554757  3797 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:38.560822  3599 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:38.600257  3820 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:38.646843  3833 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.648526  3957 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.648064  3984 raft_consensus.cc:3055] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:38.657169  2231 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:00:38.614259  4031 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.741s	user 0.010s	sys 0.001s
I20260709 13:00:38.576381  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.634814  2237 rpcz_store.cc:276] 0709 13:00:37.510658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.514699 (+  4041us) service_pool.cc:224] Handling call
0709 13:00:38.601332 (+1086633us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54005eddd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:38.601458 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503948160}
I20260709 13:00:38.635525  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.647132  4101 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.635856  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.609731  4067 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:38.507222  3795 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.567144  3802 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:38.617761  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.780743   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:38.599110  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 448) took 1429 ms. Trace:
I20260709 13:00:38.819398  2228 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:38.623579  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 648) took 1147 ms. Trace:
I20260709 13:00:38.819296  1536 rpcz_store.cc:276] 0709 13:00:37.169215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.952635 (+783420us) service_pool.cc:224] Handling call
0709 13:00:37.953044 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:37.953316 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:38.094370 (+141054us) log.cc:844] Serialized 14 byte log entry
0709 13:00:38.598427 (+504057us) spinlock_profiling.cc:243] Waited 3.81 ms on lock 0x7b1800402a00. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08f6944bc 00007fe08c9e248b 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a
0709 13:00:38.598660 (+   233us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:38.598727 (+    67us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:38.598787 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.598874 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:38.599038 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":220773,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":7708,"raft.queue_time_us":34917,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":1529,"spinlock_wait_cycles":8772352,"thread_start_us":177718,"threads_started":2,"wal-append.queue_time_us":150341}
W20260709 13:00:38.603353  1538 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2268) took 1750 ms (client timeout 2189 ms). Trace:
I20260709 13:00:38.821870  2229 rpcz_store.cc:276] 0709 13:00:37.471534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.476628 (+  5094us) service_pool.cc:224] Handling call
0709 13:00:38.284794 (+808166us) spinlock_profiling.cc:243] Waited 1.06 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:38.619364 (+334570us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2441984}
I20260709 13:00:38.721805  3346 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:38.683113  3957 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:38.563221  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 155940us
I20260709 13:00:38.681756  3820 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.684449  2233 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:38.684180  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:38.611832  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 52839.6us
I20260709 13:00:38.605720  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1815) took 1436 ms. Trace:
W20260709 13:00:38.804647  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 325125us
I20260709 13:00:38.626044  1685 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.655768  1686 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:38.635043  1692 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:00:38.813944  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 163150us
W20260709 13:00:38.815444  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 208500us
I20260709 13:00:38.657261  3882 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:38.656690  3916 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:38.674221  3599 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.678265  3800 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.713541  3802 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.602645  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 461) took 1307 ms. Trace:
I20260709 13:00:38.729880  3924 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:38.555233  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 644) took 1086 ms. Trace:
I20260709 13:00:38.637888  1584 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.648449  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:38.726419  3822 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5.
I20260709 13:00:38.553197  1539 rpcz_store.cc:276] 0709 13:00:37.296793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.551793 (+1255000us) service_pool.cc:224] Handling call
0709 13:00:38.552079 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:38.552568 (+   489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:38.552737 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.824350  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:38.823401  1538 rpcz_store.cc:269] 0709 13:00:36.852608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.497756 (+645148us) service_pool.cc:224] Handling call
0709 13:00:38.603147 (+1105391us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54000e8dd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:38.603280 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":646506368}
I20260709 13:00:38.817170  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.828493  2237 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:38.680828  3984 raft_consensus.cc:740] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:38.714632  4021 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:38.830392  1583 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:38.875777  2235 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:38.538830  4030 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:38.619071  4035 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:38.870625  4110 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 13:00:38.879127  1613 rpcz_store.cc:276] 0709 13:00:37.169546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.475435 (+305889us) service_pool.cc:224] Handling call
0709 13:00:38.605512 (+1130077us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400389354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:38.605646 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428740352}
W20260709 13:00:38.714100  4036 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.534s	user 0.008s	sys 0.001s
I20260709 13:00:38.613296  4038 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.633512  4044 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:00:38.656518  4054 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.882275  3916 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.894857  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:38.896723  4093 consensus_meta.cc:287] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.546s	user 0.015s	sys 0.000s
I20260709 13:00:38.897409  4093 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.878417  4030 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.884223  4036 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:38.685632  4067 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:38.563963  4076 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:38.709925  4081 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.915499  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.916429  4005 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:38.916887  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 268412us
I20260709 13:00:38.917271  4005 raft_consensus.cc:697] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:00:38.712478  4098 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:00:38.920269  4017 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:38.722877  4099 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:00:38.744630  3942 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.915508  4067 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:38.927001  3942 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:38.683347  4031 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:38.870206  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.870995  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:38.929569  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.930248  1683 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 3: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:38.930423  4031 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:38.821228  1535 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2242) took 2356 ms (client timeout 1879 ms). Trace:
W20260709 13:00:38.931190  1535 rpcz_store.cc:269] 0709 13:00:36.464270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:36.470698 (+  6428us) service_pool.cc:224] Handling call
0709 13:00:38.820722 (+2350024us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b540012cfd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:38.820911 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3759736192}
I20260709 13:00:38.931831  1688 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:38.836745  3820 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:38.881354  1507 leader_election.cc:336] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.565s (SENT)
W20260709 13:00:38.888036  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 58091.8us
I20260709 13:00:38.934152  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.934440  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.936182  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:38.887527  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 249575us
W20260709 13:00:38.735743  4088 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.241s	user 0.005s	sys 0.000s
I20260709 13:00:38.920275  4076 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:38.941257  2232 rpcz_store.cc:276] 0709 13:00:37.468874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.473083 (+  4209us) service_pool.cc:224] Handling call
0709 13:00:38.329903 (+856820us) spinlock_profiling.cc:243] Waited 929 us on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:38.552369 (+222466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2137344}
I20260709 13:00:38.888375  4021 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:38.835325  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:38.930531  2228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 836) took 1179 ms. Trace:
I20260709 13:00:38.870554  3984 consensus_queue.cc:260] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:38.936446  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:38.937136  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:38.938298  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 57442.1us
I20260709 13:00:38.887859  4038 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:38.888008  4097 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:38.916723  2233 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:38.909286  2231 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:38.954656  3984 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:38.954770  4093 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:38.943338  1584 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.952421  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 843) took 1102 ms. Trace:
W20260709 13:00:38.958112  1507 leader_election.cc:336] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.610s (SENT)
I20260709 13:00:38.958339  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:38.959723  2201 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:38.959870  1507 leader_election.cc:336] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.640s (SENT)
I20260709 13:00:38.960427  1507 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:38.944047  1540 rpcz_store.cc:276] 0709 13:00:37.295380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:37.606812 (+311432us) service_pool.cc:224] Handling call
0709 13:00:38.602418 (+995606us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b54004182d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:38.602551 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1098957056}
I20260709 13:00:38.957037  2233 rpcz_store.cc:276] 0709 13:00:37.850295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.827302 (+977007us) service_pool.cc:224] Handling call
0709 13:00:38.952359 (+125057us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:38.960035  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.962854  4044 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:38.974845  4044 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:38.961261  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:38.978874  1686 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.980643  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:38.964115  3800 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:38.965279  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:38.965600  4063 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:38.967757  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.970964  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 834) took 1224 ms. Trace:
I20260709 13:00:38.985952  4075 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:38.986543  2235 rpcz_store.cc:276] 0709 13:00:37.746621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.549811 (+803190us) service_pool.cc:224] Handling call
0709 13:00:38.635363 (+ 85552us) spinlock_profiling.cc:243] Waited 8.43 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:38.970671 (+335308us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b5400475254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:38.970876 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433498624}
I20260709 13:00:38.988940  2234 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:38.989169  2235 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:38.990248  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:38.976507  4044 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.989310  1583 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.005864  4097 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:39.006332  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 54857.7us
I20260709 13:00:38.981555  4005 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:38.983748  4093 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:39.008548  4093 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:39.008636  1583 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:38.971134  3924 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:39.003583  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:39.003789  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 76639.8us
I20260709 13:00:39.009948  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 838) took 1249 ms. Trace:
I20260709 13:00:39.010401  2237 rpcz_store.cc:276] 0709 13:00:37.761021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.981942 (+1220921us) service_pool.cc:224] Handling call
0709 13:00:39.009863 (+ 27921us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:39.009951  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:38.946107  2228 rpcz_store.cc:276] 0709 13:00:37.751407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.929109 (+1177702us) service_pool.cc:224] Handling call
0709 13:00:38.930453 (+  1344us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:38.985320  3800 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.016011  3800 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:38.943341  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:39.009042  4093 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.029325  2201 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:39.050171  2236 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:39.051069  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 831) took 1344 ms. Trace:
I20260709 13:00:39.051481  2236 rpcz_store.cc:276] 0709 13:00:37.706739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.834723 (+1127984us) service_pool.cc:224] Handling call
0709 13:00:39.050976 (+216253us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:39.056216  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 94958.3us
I20260709 13:00:39.056928  3896 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:39.059993  3896 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:39.060441  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 79960.7us
I20260709 13:00:39.061132  3896 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:39.061842  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 102070us
W20260709 13:00:39.062636  1507 leader_election.cc:336] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.644s (SENT)
W20260709 13:00:39.063665  1507 leader_election.cc:336] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.647s (SENT)
I20260709 13:00:39.064095  1507 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:39.068800  4097 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:39.077008  3942 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:39.081696  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.082760  1688 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.089974  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.092212  1683 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 13:00:39.092543  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 45603.5us
I20260709 13:00:39.093750  1657 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:39.095748  3942 raft_consensus.cc:2764] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 13:00:39.097646  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 44337.8us
I20260709 13:00:39.104056  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.077172  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:39.116609  1686 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:39.113819  4097 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.052722  4031 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:39.133220  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56172.3us
I20260709 13:00:39.136078  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:39.136956  1540 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 7
W20260709 13:00:39.145256  4112 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.103s	user 0.004s	sys 0.001s
I20260709 13:00:39.146409  4067 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:39.147404  3843 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.147150  3599 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.148079  3843 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.148892  4067 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:39.121665  4093 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:39.149812  3984 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.150409  3843 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:39.150962  3843 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:39.151383  4093 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:39.152210  3843 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:39.150998  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:39.152712  3843 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:39.153892  3797 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:39.154049  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:00:39.154268  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 90676.1us
I20260709 13:00:39.154569  3797 raft_consensus.cc:697] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:00:39.155066  1507 leader_election.cc:336] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.649s (SENT)
I20260709 13:00:39.155405  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.155618  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:39.156773  4104 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:39.157271  4104 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:39.157641  3916 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:39.158187  3727 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.622s	user 0.008s	sys 0.009s
I20260709 13:00:39.156999  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:39.160650  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:39.158736  3727 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:39.161338  3866 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:00:39.161365  3916 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:39.165373  3916 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:00:39.166437  1507 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.680s (SENT)
I20260709 13:00:39.164525  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:39.166728  3916 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:39.168082  4102 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:00:39.170987  2237 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:39.173563  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.173769  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.155982  3797 consensus_queue.cc:237] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:39.167001  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.157747  4104 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:39.179706  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47777.1us
I20260709 13:00:39.126857  4038 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:39.129685  4097 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:39.178257  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 47186us
I20260709 13:00:39.185951  3942 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:39.178717  3599 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:39.186754  2232 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:39.187067  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:39.187508  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.191850  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:39.193097  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 131761us
I20260709 13:00:39.210217  1538 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:39.212666  1657 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:39.214001  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 47633.1us
I20260709 13:00:39.214300  1583 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:39.213913  3843 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:39.214849  1507 leader_election.cc:336] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.699s (SENT)
I20260709 13:00:39.215576  3843 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.215796  1583 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:39.216902  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:39.216001  1507 leader_election.cc:336] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.824s (SENT)
I20260709 13:00:39.219311  1688 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:39.123591  3924 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 13:00:39.193126  3727 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:39.157516  3957 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.195106  3942 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:39.235270  4031 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:39.236691  4063 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:39.237274  4063 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.237717  4063 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.237958  1535 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 13:00:39.238487  3833 consensus_meta.cc:287] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.553s	user 0.003s	sys 0.012s
I20260709 13:00:39.238899  3833 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:39.240232  3957 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:39.241019  3833 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:39.241779  1613 raft_consensus.cc:2393] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:39.235385  3866 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:39.242249  4081 raft_consensus.cc:2764] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:00:39.233757  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:39.221278  4038 raft_consensus.cc:2764] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 13:00:39.241843  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 55034.7us
I20260709 13:00:39.247777  1692 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:39.235359  3942 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.255888  1537 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:39.276751  4116 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:00:39.277235  1685 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:39.244444  4030 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.280129  4030 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:39.281479  4035 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.281728  4076 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:39.283735  4076 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:39.284339  4035 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:39.285284  1541 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:39.285723  4056 raft_consensus.cc:2764] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:00:39.230551  3984 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.286605  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 464) took 1196 ms. Trace:
I20260709 13:00:39.287014  1541 rpcz_store.cc:276] 0709 13:00:38.089934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.566491 (+476557us) service_pool.cc:224] Handling call
0709 13:00:38.566785 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:39.285706 (+718921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.286365 (+   659us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b54003444d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:39.286555 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651460352}
W20260709 13:00:39.288700  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 59569.1us
W20260709 13:00:39.290918  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 100114us
I20260709 13:00:39.290736  1508 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.260387  1584 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:39.303998  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.303579  1538 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:39.260697  4005 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:39.310359  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 94395.1us
I20260709 13:00:39.312690  1540 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:39.310756  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:39.290702  3727 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 13:00:39.328395  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.326821  2230 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:39.331290  1508 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:39.341588  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 79998.1us
I20260709 13:00:39.327446  3599 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:39.313967  4005 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:00:39.342808  1507 leader_election.cc:336] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.970s (SENT)
I20260709 13:00:39.341073  4035 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.354946  4035 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.356467  3795 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 13:00:39.355875  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 88659us
I20260709 13:00:39.316319  4081 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:39.340230  3833 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:39.365191  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:39.365407  4054 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:39.367833  4054 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:39.355355  1507 leader_election.cc:336] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.970s (SENT)
W20260709 13:00:39.370095  1507 leader_election.cc:336] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.988s (SENT)
W20260709 13:00:39.371186  1507 leader_election.cc:336] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.045s (SENT)
I20260709 13:00:39.339681  3957 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:39.351181  4005 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:39.361109  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 125253us
I20260709 13:00:39.373471  3942 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:39.405020  4035 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:39.363504  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:39.405515  4104 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.363855  3833 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.406780  3833 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:39.407966  3942 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:39.408711  4104 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:39.364044  3346 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
I20260709 13:00:39.359257  3797 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
W20260709 13:00:39.409512  1507 leader_election.cc:336] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.078s (SENT)
I20260709 13:00:39.409976  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.410362  3797 raft_consensus.cc:3055] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:39.410806  3797 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:00:39.411206  1507 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.232s (SENT)
I20260709 13:00:39.411957  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:39.411734  3797 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.388243  4081 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:39.411653  1507 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:39.413173  4054 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.412909  4081 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.414085  3924 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.414937  3924 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:39.415182  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.415314  4081 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:39.416034  1691 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:39.416567  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:39.417301  1612 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:39.417254  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:39.418437  1614 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:39.419435  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:39.388859  3727 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.423687  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:39.424386  1615 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:39.426395  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:00:39.375751  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 100434us
I20260709 13:00:39.427340  4054 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:39.427783  4054 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.427824  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.428222  4054 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.429605  3727 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.388067  2201 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:39.394712  4102 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:00:39.396188  1611 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:39.398329  3896 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:39.398756  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.360077  4067 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:39.399681  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.400461  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.401239  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.401837  1584 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:39.402338  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.403268  4036 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:39.364637  1609 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:00:39.406087  1610 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.412236  3957 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:39.436108  3797 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:39.470003  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 68274.1us
I20260709 13:00:39.471035  1584 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.472856  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.415927  4081 consensus_queue.cc:1059] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 13:00:39.474164  4081 raft_consensus.cc:3055] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:39.474599  4081 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:39.425249  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:39.475764  1615 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:39.394158  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:39.389213  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 846) took 1323 ms. Trace:
I20260709 13:00:39.476792  2230 rpcz_store.cc:276] 0709 13:00:38.065863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:38.826001 (+760138us) service_pool.cc:224] Handling call
0709 13:00:39.388710 (+562709us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b5400741154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:39.388900 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832980224}
W20260709 13:00:39.477373  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 83928.9us
W20260709 13:00:39.477723  1507 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.511s (SENT)
I20260709 13:00:39.477947  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:39.478157  3346 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.478338  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:39.479493  1507 leader_election.cc:336] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.544s (SENT)
I20260709 13:00:39.480161  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:39.481043  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:39.481554  3346 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:39.481789  1614 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:00:39.481271  1507 leader_election.cc:336] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.574s (SENT)
I20260709 13:00:39.483891  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:39.482618  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:39.484683  1612 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:39.485174  3346 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:39.485262  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:39.486123  3957 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.449046  1537 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
W20260709 13:00:39.451748  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 72242.4us
I20260709 13:00:39.457787  4036 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:39.469894  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.438498  4005 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.487318  3346 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:00:39.532661  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 59872us
I20260709 13:00:39.537554  2203 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:39.537979  1507 leader_election.cc:336] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.871s (SENT)
I20260709 13:00:39.538767  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:39.539784  2228 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:39.541313  4055 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:39.542477  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.543226  2230 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.549577  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:39.563244  1691 raft_consensus.cc:2368] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7.
I20260709 13:00:39.565725  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:39.566454  2236 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:39.569252  4055 raft_consensus.cc:697] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:39.570614  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.570528  4055 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.571358  3843 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:39.571602  1692 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:39.571986  3843 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:39.572973  3843 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:39.577366  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.577656  3802 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:39.578266  2228 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:39.561849  4103 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:39.579388  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.580873  4017 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:39.581707  4035 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:39.582168  3896 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:39.583279  4035 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:39.583667  4035 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:39.584694  4035 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.587236  4035 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:39.591194  1507 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.766s (SENT)
I20260709 13:00:39.591763  1507 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:39.593031  4126 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.586232  1610 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:39.585858  4036 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.503687  4038 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.504153  4067 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:39.555074  4075 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:39.593923  4076 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:39.602325  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.492461  3800 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:39.607072  1507 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.734s (SENT)
I20260709 13:00:39.614137  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.531553  3957 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.106s
I20260709 13:00:39.595633  4005 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:39.634213  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.606225  3346 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:39.608659  4075 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:39.611408  4102 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.642156  1584 leader_election.cc:400] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5.
W20260709 13:00:39.649132  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 52629.6us
I20260709 13:00:39.645542  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:39.652393  2235 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:39.655050  1688 raft_consensus.cc:2376] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6. Re-sending same reply.
W20260709 13:00:39.656488  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.639276  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.658795  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.660215  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.661381  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:39.661755  2230 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:39.661947  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.662518  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.662955  4093 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.663138  1583 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:39.663487  4130 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:39.664057  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.633289  4076 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.665078  4067 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.667114  4076 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:39.668481  4056 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.669065  4056 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.671162  4056 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:39.675843  3833 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:39.640167  4075 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:39.652612  4054 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.682739  1508 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.683645  4054 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:39.684094  1508 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.685298  4030 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:39.661180  1684 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:39.645052  4036 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.684273  4054 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.605808  4031 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:39.638806  3599 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:39.615499  4104 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:00:39.638482  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.696875  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.689885  3984 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:39.703698  3727 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:39.662894  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.639240  4097 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:39.720605  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 110883us
I20260709 13:00:39.697327  4076 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 13:00:39.723009  1507 leader_election.cc:336] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.845s (SENT)
I20260709 13:00:39.723083  3800 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:39.650015  4081 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:39.783986  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:39.785526  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:39.785753  4017 consensus_queue.cc:1059] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:39.725739  1690 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:00:39.726333  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.690454  4063 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.730469  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.685596  4102 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.710704  4031 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:39.725281  3727 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:39.734472  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:39.735224  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:39.735668  3833 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:39.714447  4054 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:39.737632  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:39.814437  2233 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:39.784016  4081 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.738904  2231 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:39.739573  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.740257  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.743383  3822 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.742676  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:39.746232  2235 raft_consensus.cc:2410] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:39.747937  1615 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:39.749831  4104 consensus_queue.cc:1243] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:00:39.738904  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.823359  1692 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:39.751286  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.823935  1618 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:39.752697  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.761379  1540 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:39.761946  1538 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:39.803224  3800 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:39.761663  4005 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:39.763190  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:39.763993  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:39.764416  1539 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:39.766213  1542 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: [5.7-7.8]
I20260709 13:00:39.765103  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.765894  1689 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:00:39.775494  1612 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:39.728905  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:39.781742  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 58783.5us
I20260709 13:00:39.790861  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:39.805953  2234 raft_consensus.cc:2410] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:00:39.741128  4075 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.737491  3346 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.849151  1537 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.853802  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.854997  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.855068  4063 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:39.855680  1692 raft_consensus.cc:2368] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6.
I20260709 13:00:39.856457  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.800302  3795 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:00:39.861725  4054 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:39.863605  4075 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:39.835098  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.865999  1690 raft_consensus.cc:2368] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 2. Current term is 3.
I20260709 13:00:39.866839  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.868027  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.868784  1690 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:39.836848  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.871704  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.872800  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.811674  3727 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:39.874547  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.839406  1616 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:39.840962  3822 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.842139  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:39.884328  1542 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.842788  1535 raft_consensus.cc:2376] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7. Re-sending same reply.
I20260709 13:00:39.884760  1536 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:39.879400  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.843636  1538 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:39.845049  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.845335  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.846376  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:39.842819  1540 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:00:39.848626  1684 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:39.885766  1538 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:39.792057  3802 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:00:39.877597  1610 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:39.878844  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.890875  1614 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:39.896062  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.897205  1692 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:39.888199  1688 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:39.889403  2237 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:39.899655  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.900391  2237 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:39.890615  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:39.902361  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 66843.2us
I20260709 13:00:39.903462  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.877316  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.813917  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:39.906294  1609 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:39.906643  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.907701  1686 raft_consensus.cc:2368] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4.
I20260709 13:00:39.908722  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.910005  4075 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:39.910926  4075 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:39.918442  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.920202  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.921206  1686 raft_consensus.cc:2368] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7.
I20260709 13:00:39.905062  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:39.924994  4063 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:39.933676  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:39.936076  4063 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:39.937456  4063 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:39.940992  4063 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:39.943665  4067 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:39.944252  4067 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.949851  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:39.894552  1538 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:39.922277  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:39.884058  4005 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:39.962098  1537 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:39.974774  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:40.011652  2237 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:40.027784  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:40.028460  2237 raft_consensus.cc:2376] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5. Re-sending same reply.
I20260709 13:00:40.030566  1543 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 13:00:40.027899  1507 leader_election.cc:336] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.619s (SENT)
I20260709 13:00:39.967073  1683 raft_consensus.cc:2368] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7.
I20260709 13:00:39.973320  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:40.022848  1686 raft_consensus.cc:2368] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7.
W20260709 13:00:39.988482  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 188588us
I20260709 13:00:39.960075  1687 raft_consensus.cc:2368] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6.
I20260709 13:00:39.954105  1584 leader_election.cc:400] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7.
I20260709 13:00:40.035954  1690 raft_consensus.cc:2376] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6. Re-sending same reply.
I20260709 13:00:40.036315  1584 leader_election.cc:403] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:40.038811  1538 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:40.007628  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.989990  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:39.977573  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:39.979228  4103 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.047214  3822 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:40.047904  4097 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:40.023927  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:40.023689  4044 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.034s
W20260709 13:00:40.050352  4138 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:00:40.052524  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:40.026358  4005 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:40.061718  4093 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7.
W20260709 13:00:40.062148  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 108575us
I20260709 13:00:40.063959  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:40.064908  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:40.066725  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:40.066639  4145 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:40.067795  4102 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:40.070490  4017 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:40.126287  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:40.072257  4075 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:40.127533  2231 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:40.127743  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:40.128391  4145 raft_consensus.cc:3055] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:40.128860  4145 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:40.075248  2235 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:40.130044  4145 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:40.075958  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:40.132577  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:40.133587  1687 raft_consensus.cc:2393] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:40.090693  2230 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:40.081705  1691 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:39.895346  3800 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:40.139935  3800 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:40.141842  4081 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.083076  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:40.142427  4081 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:40.143615  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:40.144645  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:40.089766  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.145272  4081 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:40.094763  4103 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:40.147742  4103 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:40.048874  4038 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:40.149374  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:40.150401  1540 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:40.151675  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:40.153039  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:40.092832  1542 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:40.155153  1540 raft_consensus.cc:2368] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6.
I20260709 13:00:40.156657  1692 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:40.073050  3727 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:40.171059  1538 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:40.121490  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:40.121848  1508 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:40.111809  1684 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:40.125347  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 58692.7us
I20260709 13:00:40.176542  1688 raft_consensus.cc:2368] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6.
I20260709 13:00:40.129057  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:40.179176  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:40.180260  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:40.077207  2234 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:40.136199  2237 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:40.134560  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:40.137850  1584 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:40.181358  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.183825  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 62073.3us
I20260709 13:00:40.114789  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:40.181439  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:40.181829  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:40.185901  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 111122us
I20260709 13:00:40.183514  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:40.189752  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:40.190896  1688 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:40.191155  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:40.191857  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:40.192075  1688 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:40.191898  1542 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:40.193699  1686 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:40.191345  2235 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.190596  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:40.187466  4030 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 6 current term: 6 is higher.
I20260709 13:00:40.197985  1539 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:40.199250  1686 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:40.199739  1690 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:40.200192  1692 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:40.201968  1688 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:40.192612  4130 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.192483  4102 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:40.209254  1539 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:40.204958  1541 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:40.209749  1535 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:40.202689  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:40.214322  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:40.189240  1683 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:40.401463  1540 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:40.403347  1683 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:40.406121  1683 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:00:40.262857  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 125109us
I20260709 13:00:40.206593  4103 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:40.410189  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 146088us
W20260709 13:00:40.447139  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 242483us
I20260709 13:00:40.450649  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:40.455531  1539 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:00:40.456359  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.456904  1507 leader_election.cc:336] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.964s (SENT)
I20260709 13:00:40.205843  4126 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:40.496978  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:40.499629  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.500401  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:40.500290  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 52039.6us
I20260709 13:00:40.442065  4103 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:40.504441  1541 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:40.501667  1657 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:40.511319  1535 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:40.331625  4130 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:40.516324  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:40.527251  1538 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:40.522020  4102 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:40.210530  4031 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.299s
I20260709 13:00:40.509567  4063 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.510732  4103 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:40.206277  4005 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:40.549225  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:40.549927  1538 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:40.529017  1685 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:40.537087  3942 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:40.524438  2230 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
W20260709 13:00:40.568349  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 111463us
I20260709 13:00:40.394737  4148 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.225s
W20260709 13:00:40.571719  1507 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.974s (SENT)
I20260709 13:00:40.575476  1507 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:40.572708  1541 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:00:40.509277  4081 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:00:40.576403  1541 pending_rounds.cc:85] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 1
I20260709 13:00:40.582338  1541 pending_rounds.cc:107] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 13:00:40.584779  1541 raft_consensus.cc:2889] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:40.585963  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:40.586107  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:40.586645  2228 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.587930  4102 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:40.586715  1541 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:40.565696  1685 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:40.590418  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:40.577595  4148 consensus_queue.cc:1243] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:00:40.579494  4005 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:40.578543  3802 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:40.579248  3727 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:40.607467  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:40.599339  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 97747.2us
I20260709 13:00:40.602129  1610 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:40.617480  1616 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:40.617878  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:40.617552  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:40.619396  1615 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:00:40.619590  1618 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:40.614593  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:40.620354  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:40.623234  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:40.624567  1615 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:40.624774  1618 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [4.3-6.4]
W20260709 13:00:40.625211  4153 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 13:00:40.625978  1612 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:40.626196  1615 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:40.627189  1612 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:40.629962  1685 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:40.631057  1685 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:40.635938  1610 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:40.638710  4130 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:40.630463  1613 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:40.589187  4063 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:40.642073  1616 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:40.643355  4063 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:40.648223  1541 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:40.648553  2235 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:40.651701  4036 raft_consensus.cc:3055] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:40.652168  4036 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:40.669508  1508 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:40.677589  1541 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:40.668771  1583 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:40.689718  1539 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:40.692615  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:40.694212  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 73132.7us
I20260709 13:00:40.694449  1541 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.696095  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:40.697058  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:40.698590  1539 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.697347  1583 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:40.703835  1538 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:40.627664  3942 raft_consensus.cc:697] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:40.710767  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:40.701196  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:40.697714  1684 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.500s	user 0.001s	sys 0.009s
I20260709 13:00:40.693334  4130 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:40.715786  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:40.699970  3727 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:40.724694  1614 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:40.713343  4166 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:00:40.712304  3833 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:40.733860  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:40.730177  3942 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:40.741355  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:40.748585  1542 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.741540  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:40.712819  1684 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:40.695958  4104 consensus_queue.cc:1059] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:00:40.751437  4104 raft_consensus.cc:3055] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:40.653069  4036 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:40.751820  4104 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:40.749078  2228 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:40.752722  4104 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:40.741808  4152 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:00:40.754943  3833 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:40.755915  3833 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:40.753954  4104 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:40.746675  1539 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:40.760800  1583 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:40.761371  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:40.723356  4162 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:40.762017  3896 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:40.767514  4067 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:40.769277  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:40.769629  1539 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:40.763631  1541 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:40.773355  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:40.776059  1538 raft_consensus.cc:2393] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:40.783417  1582 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:40.786353  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:40.787165  1618 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:40.788439  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:40.789168  1614 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:40.789865  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:40.790658  1539 raft_consensus.cc:2410] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:40.787035  1582 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:40.770558  4152 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:40.788672  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:40.792371  1610 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
W20260709 13:00:40.793278  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 221573us
I20260709 13:00:40.796660  1541 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:40.769068  4103 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:40.802608  1612 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:40.786592  4067 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:40.813092  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.814071  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.816718  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:40.817499  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:40.789558  2228 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:40.823863  1688 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:00:40.827858  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.781041  3896 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:40.833791  1691 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:40.834297  1692 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:00:40.842226  1609 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: [3.4-5.5]
I20260709 13:00:40.844775  1685 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:00:40.845283  1685 pending_rounds.cc:85] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 2
I20260709 13:00:40.848717  1685 pending_rounds.cc:107] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:00:40.849145  1685 raft_consensus.cc:2889] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:40.832863  3727 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:40.852941  3727 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:40.846319  1507 leader_election.cc:336] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.248s (SENT)
W20260709 13:00:40.854101  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:40.855083  1688 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:40.858608  1507 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:40.860190  1542 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:40.863335  2229 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:40.864470  4177 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:40.865813  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.866789  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 118089us
W20260709 13:00:40.869618  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 82464.5us
I20260709 13:00:40.868675  1584 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:40.866250  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2513609069999916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22120000000000004s for a response
I20260709 13:00:40.864956  4174 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:40.875519  1584 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:40.866762  4177 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:40.885689  4038 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:40.894054  4038 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:40.895676  4038 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:40.869067  4130 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:40.871691  4145 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:40.870399  4152 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:00:40.906174  4140 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.911s	user 0.006s	sys 0.009s
I20260709 13:00:40.882302  4036 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:40.906482  4152 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:40.912986  4167 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:00:40.853740  4067 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:40.922060  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:40.938205  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 67276.8us
I20260709 13:00:40.889117  4076 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:40.965167  4168 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:00:40.967186  4103 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:40.929296  4170 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:40.900027  4174 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:40.996045  4174 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:40.947592  4175 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:40.899433  4177 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:40.887156  3896 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:40.959702  4067 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:41.008297  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:41.020020  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 673) took 1390 ms. Trace:
W20260709 13:00:41.020926  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 174637us
I20260709 13:00:41.020939  1584 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:41.020928  2236 rpcz_store.cc:276] 0709 13:00:39.618230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.618373 (+   143us) service_pool.cc:224] Handling call
0709 13:00:39.674990 (+ 56617us) spinlock_profiling.cc:243] Waited 5.6 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:00:39.675180 (+   190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:39.675413 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:39.675438 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:39.675889 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:00:39.777089 (+101200us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:39.777140 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:39.777208 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:40.861002 (+1083794us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:00:40.907238 (+ 46236us) raft_consensus.cc:1685] finished
0709 13:00:40.907267 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:41.008466 (+101199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39502848,"thread_start_us":100803,"threads_started":1,"wal-append.queue_time_us":285293}
I20260709 13:00:41.028172  1584 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:41.023826  4170 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:41.028085  4067 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.031188  4067 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:41.014524  4104 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.035069  1584 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:41.014878  4038 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.038306  4093 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:41.029476  4076 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:41.020370  4175 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:00:41.045258  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:41.046264  1685 pending_rounds.cc:85] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 5
I20260709 13:00:41.050310  1685 pending_rounds.cc:107] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:00:41.050704  1685 raft_consensus.cc:2889] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:41.050861  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:41.047430  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:41.036763  4104 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:41.053388  4170 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:41.057317  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:41.058023  4063 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:41.054664  4172 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:00:41.058820  4067 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:41.058874  1584 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:41.047624  4093 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:41.064661  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:41.066360  4162 consensus_queue.cc:1059] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:41.067802  4162 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:41.068192  4162 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:41.061000  4076 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:41.069020  4162 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:41.065181  1507 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:41.070055  4162 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:41.066399  4093 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:41.052260  1688 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:00:41.073024  1688 raft_consensus.cc:606] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:41.073370  1688 raft_consensus.cc:614] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:41.052138  4173 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:41.077615  4076 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:41.078641  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 674) took 1446 ms. Trace:
I20260709 13:00:41.078862  4172 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:41.079209  2229 rpcz_store.cc:276] 0709 13:00:39.631799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:39.638617 (+  6818us) service_pool.cc:224] Handling call
0709 13:00:39.668484 (+ 29867us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:40.863869 (+1195385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.078271 (+214402us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400730fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:41.078553 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2747371776}
I20260709 13:00:41.080032  4172 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:41.080470  4172 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:41.082715  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:41.085255  4165 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:41.085556  1507 leader_election.cc:400] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6.
I20260709 13:00:41.086186  1507 leader_election.cc:403] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:00:41.087409  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:41.088071  4031 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6.
I20260709 13:00:41.081271  4172 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:41.091594  3833 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:41.093372  2201 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:41.096932  3727 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:41.097572  4174 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:41.099527  4104 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:41.100939  3896 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:00:41.103652  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.089547  4067 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:41.153074  4067 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:41.154671  4067 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:41.158900  4175 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:41.159606  4175 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:41.187983  1440 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 340) took 12600 ms (client timeout 15000 ms). Trace:
I20260709 13:00:41.090793  3797 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:41.199564  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:41.200965  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:41.136718  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 54086us
I20260709 13:00:41.201648  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:41.203321  2231 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:41.204282  4162 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:41.204840  2237 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:41.204957  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:41.205646  1689 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:41.205586  4005 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:41.206316  3833 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:41.207252  4081 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:41.207726  3833 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:41.208155  4081 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:41.208798  3833 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:41.201678  2232 raft_consensus.cc:2376] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5. Re-sending same reply.
I20260709 13:00:41.211097  4173 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:41.211359  4144 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:00:41.214533  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.215246  4148 raft_consensus.cc:3055] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:41.215363  1690 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:41.215837  4148 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:41.214071  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:41.218155  1692 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:41.217065  4148 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:41.220703  4148 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:41.223655  1617 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:00:41.223769  4150 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:00:41.225579  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:41.226397  1613 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
W20260709 13:00:41.228699  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.232520  3795 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:41.234519  3795 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:41.236083  4055 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:41.237155  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:41.237785  2236 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
W20260709 13:00:41.238155  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:41.240547  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 153214us
I20260709 13:00:41.242074  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:41.242844  2230 raft_consensus.cc:2410] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:00:41.246157  3822 consensus_queue.cc:1059] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:41.246493  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 426) took 1067 ms. Trace:
I20260709 13:00:41.136543  3942 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:41.241412  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:41.236578  4055 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:41.250290  2235 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:41.241653  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:41.212924  4173 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:41.135375  4149 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:41.253363  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:41.295054  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 230453us
I20260709 13:00:41.295858  1584 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:41.340022  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:41.344002  1541 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:41.303457  1689 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:00:41.350867  1689 raft_consensus.cc:606] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:00:41.341225  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:41.304164  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.354506  1689 raft_consensus.cc:614] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:41.252966  4172 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.249246  4165 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:41.361873  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:41.362838  1542 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:41.246929  1609 rpcz_store.cc:276] 0709 13:00:40.179115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:40.183668 (+  4553us) service_pool.cc:224] Handling call
0709 13:00:40.264021 (+ 80353us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:40.843992 (+579971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.246189 (+402197us) spinlock_profiling.cc:243] Waited 551 ms on lock 0x7b5400104354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:41.246409 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1267310976}
I20260709 13:00:41.105402  4177 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:41.369930  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:41.374114  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:41.375268  1539 raft_consensus.cc:2410] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:41.303267  3984 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.303048  4031 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:41.378001  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 136653us
I20260709 13:00:41.378981  1538 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:00:41.380136  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:41.383700  1536 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:41.237020  3727 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:41.380384  1543 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:00:41.273940  4173 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:41.336591  4063 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:41.253129  3795 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 13:00:41.336619  3957 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.402585  3957 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:41.402778  4063 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:41.405652  3957 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:41.254240  4150 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 13:00:41.346581  4104 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.348133  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:41.347033  4149 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:41.450428  1542 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:41.450819  1539 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:41.363754  4126 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.373265  4076 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:41.362195  3797 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:41.455058  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 251506us
I20260709 13:00:41.455224  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:41.377367  4130 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:41.461699  1538 raft_consensus.cc:2410] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:41.457551  1539 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:41.418453  4159 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:00:41.358676  4144 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
W20260709 13:00:41.464597  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 217083us
I20260709 13:00:41.471448  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:41.471315  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 198336us
I20260709 13:00:41.417673  4162 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:00:41.480419  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 239995us
I20260709 13:00:41.483678  2203 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:41.454826  3957 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:41.460371  4076 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.461817  4165 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.464963  4130 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.512023  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:41.500263  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 138507us
I20260709 13:00:41.514247  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:41.486176  4170 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:41.416014  3984 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:41.477161  4162 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:41.484409  3833 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:41.531332  1683 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:00:41.447862  4031 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:41.188617  1440 rpcz_store.cc:269] 0709 13:00:28.588129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:28.588447 (+   318us) service_pool.cc:224] Handling call
0709 13:00:28.984466 (+396019us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:00:41.187638 (+12203172us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:37.694435 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:37.695006 (+   571us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:37.695040 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:00:37.695058 (+    18us) tablet.cc:662] Decoding operations
0709 13:00:37.778008 (+ 82950us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:37.778061 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:37.778092 (+    31us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:00:37.778597 (+   505us) tablet.cc:701] Row locks acquired
0709 13:00:37.778616 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:00:37.778768 (+   152us) write_op.cc:270] Start()
0709 13:00:37.778889 (+   121us) write_op.cc:276] Timestamp: P: 1783602037778722 usec, L: 0
0709 13:00:37.778913 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:00:37.781510 (+  2597us) log.cc:844] Serialized 2457 byte log entry
0709 13:00:38.122740 (+341230us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08c9e20ac 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b
0709 13:00:38.732913 (+610173us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:00:39.161825 (+428912us) op_driver.cc:464] REPLICATION: finished
0709 13:00:39.486721 (+324896us) write_op.cc:301] APPLY: starting
0709 13:00:39.486839 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:39.487090 (+   251us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:39.487127 (+    37us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:39.488173 (+  1046us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:39.488505 (+   332us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:00:39.488548 (+    43us) write_op.cc:312] APPLY: finished
0709 13:00:39.488988 (+   440us) log.cc:844] Serialized 53 byte log entry
0709 13:00:39.489331 (+   343us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:39.495025 (+  5694us) write_op.cc:454] Released schema lock
0709 13:00:39.532581 (+ 37556us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:39.539531 (+  6950us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:39.539563 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":25,"spinlock_wait_cycles":889844096,"child_traces":[["op",{"apply.queue_time_us":300019,"apply.run_cpu_time_us":3545,"apply.run_wall_time_us":52764,"num_ops":6,"prepare.queue_time_us":105436,"prepare.run_cpu_time_us":81725,"prepare.run_wall_time_us":535990,"raft.queue_time_us":182295,"raft.run_cpu_time_us":23014,"raft.run_wall_time_us":115555,"replication_time_us":1382672,"spinlock_wait_cycles":41761792,"thread_start_us":632504,"threads_started":4,"wal-append.queue_time_us":572610,"wal-append.run_cpu_time_us":17129,"wal-append.run_wall_time_us":2130360}]]}
I20260709 13:00:41.543692  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:41.542577  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:41.568249  2233 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:41.510787  4076 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:41.583918  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.448143  4055 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:41.486078  4104 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:41.527926  4130 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.599988  1535 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:41.465783  4126 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:41.609241  1534 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:00:41.530638  4165 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:41.634687  4174 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:41.636298  4187 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 13:00:41.638178  4189 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:00:41.592234  4005 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:41.639282  4185 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.446s	user 0.001s	sys 0.000s
W20260709 13:00:41.650385  4184 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:00:41.652796  4197 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:41.653250  4194 consensus_queue.cc:1059] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:41.653827  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:41.654239  4200 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:41.657945  4126 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:41.658934  3727 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:41.659957  4031 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:41.793555  1684 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:41.660414  4055 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:41.661051  4163 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:00:41.807096  4165 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:41.807853  4031 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:41.662048  4162 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:41.530347  4173 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:41.666276  4177 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:41.667575  3984 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:41.668457  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:41.669387  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:41.653766  1540 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:41.639202  4206 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:41.639868  4196 raft_consensus.cc:3055] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:41.640872  4170 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:41.577214  4179 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.774161  4005 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:41.820631  4038 consensus_meta.cc:287] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.776s	user 0.007s	sys 0.003s
W20260709 13:00:41.768066  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 243340us
W20260709 13:00:41.768510  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:41.770287  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 474467us
W20260709 13:00:41.771867  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:41.772603  4152 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.007s
W20260709 13:00:41.772505  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 288951us
I20260709 13:00:41.692313  4175 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:41.777060  4201 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:41.784162  4190 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.269s	user 0.001s	sys 0.001s
I20260709 13:00:41.726280  4030 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:00:41.795113  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:41.804487  4194 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:41.806687  4200 raft_consensus.cc:3055] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:41.821945  1690 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:41.678655  4149 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:41.841785  4104 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:41.732618  1686 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
W20260709 13:00:41.837361  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:41.749061  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:41.841459  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.856397  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:41.865582  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:00:41.869692  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 326055us
W20260709 13:00:41.863634  4152 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.000s	sys 0.007s
I20260709 13:00:41.863899  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:41.887864  4175 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:41.874305  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:41.886950  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.880282  1509 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:41.831557  4196 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:41.865635  4173 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:41.833105  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:41.889983  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:41.864562  4162 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.890794  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:41.831444  4206 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:41.897624  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:41.904325  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:41.897310  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:41.879706  4194 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:41.879591  4200 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:41.883299  1684 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.905867  1616 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:41.905213  1618 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:41.906915  1658 leader_election.cc:400] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6.
I20260709 13:00:41.895649  4201 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:41.916505  4227 consensus_queue.cc:1059] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:00:41.916808  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:41.931778  1617 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:41.917160  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:41.938719  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:41.933713  3984 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:41.942837  4186 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:00:41.943106  4192 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.942687  3984 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:41.943629  4192 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:41.937074  4103 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:41.946578  4103 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:41.895769  4212 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:00:41.948947  4103 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:41.949034  1687 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:00:41.906598  4206 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:41.950215  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:41.945185  3984 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:41.954253  1687 raft_consensus.cc:606] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:00:41.962656  1687 raft_consensus.cc:614] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:41.916741  4031 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:41.949668  4165 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.894168  4175 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:00:41.844688  4179 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:41.967031  4103 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.968089  3727 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:41.967031  4031 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:41.968972  4197 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:41.984548  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 119107us
I20260709 13:00:41.962414  3795 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:41.990757  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:41.997066  1613 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:41.983726  4036 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:41.989763  4031 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:41.942955  4205 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 13:00:41.993925  3795 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:41.995481  4172 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.003s
I20260709 13:00:42.018074  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:41.980535  4165 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.026473  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:41.989471  3727 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.028759  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:42.005330  4192 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.009976  4227 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:42.006625  4036 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:42.037227  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.013191  4210 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 13:00:42.061076  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:42.065642  4244 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:00:42.066540  3942 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:42.067147  4198 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.026072  4145 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:00:42.068209  4202 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.117s
I20260709 13:00:42.068594  4227 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:42.073670  4226 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:42.070142  4222 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:00:42.070626  4224 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:42.070866  4215 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:41.987201  4103 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.031706  1683 raft_consensus.cc:2410] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:42.028319  3795 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.100737  3957 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.100541  4031 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.103782  3727 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.032320  4212 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.096697  4224 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:42.105674  4212 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.106082  4224 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:42.106164  4212 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.082033  1536 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:42.067718  4232 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:42.088464  4177 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:41.994747  4197 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.090471  4198 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:42.097739  4229 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.096715  4227 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.099903  4103 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.091684  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:42.045902  4239 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.073755  3822 consensus_queue.cc:1059] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:00:42.090605  3942 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:42.068962  4221 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:00:42.107010  4148 consensus_meta.cc:287] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.886s	user 0.010s	sys 0.003s
I20260709 13:00:42.002664  4196 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.021226  4200 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:42.101262  3957 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.106896  4226 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.106736  4036 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:42.257315  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 249005us
W20260709 13:00:42.285998  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 164694us
I20260709 13:00:42.303683  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2142) took 1049 ms. Trace:
W20260709 13:00:42.326272  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.340114  1690 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:42.115350  4244 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:42.361447  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.371050  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:42.147333  4202 consensus_queue.cc:1243] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 7.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:42.382949  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:42.260838  4076 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.000s
I20260709 13:00:42.304407  1691 rpcz_store.cc:276] 0709 13:00:41.228073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.228248 (+   175us) service_pool.cc:224] Handling call
0709 13:00:41.228706 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:41.474803 (+246097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:41.474849 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:41.499006 (+ 24157us) log.cc:844] Serialized 26 byte log entry
0709 13:00:41.720264 (+221258us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:41.720323 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:41.720425 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:42.277566 (+557141us) raft_consensus.cc:1685] finished
0709 13:00:42.277588 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:42.277796 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":405442,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":122,"thread_start_us":365538,"threads_started":2,"wal-append.queue_time_us":326523}
I20260709 13:00:42.391311  3822 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.367791  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1796) took 1030 ms. Trace:
I20260709 13:00:42.391978  1535 rpcz_store.cc:276] 0709 13:00:41.337821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.355870 (+ 18049us) service_pool.cc:224] Handling call
0709 13:00:41.380920 (+ 25050us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:41.600516 (+219596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.367456 (+766940us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b5400411c54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:42.367707 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":502826368}
I20260709 13:00:42.392431  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:42.274415  4179 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:42.409678  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:42.410528  2230 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:42.277463  4201 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.386338  4204 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:42.150906  4198 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.262786  4248 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.149897  4229 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.349226  4241 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:42.415467  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 342375us
I20260709 13:00:42.155272  4232 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.456249  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:42.306597  4239 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.457062  4239 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.457885  4232 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.422328  1583 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:42.391988  4130 consensus_meta.cc:287] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.773s	user 0.012s	sys 0.000s
I20260709 13:00:42.148784  4210 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.246s
I20260709 13:00:42.306265  1684 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:42.428831  2230 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:42.387087  4103 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.148303  4177 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.276826  4246 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.231s
W20260709 13:00:42.470917  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:42.471704  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:42.449596  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 66736.1us
I20260709 13:00:42.394191  4173 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:42.410473  4202 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:42.454952  1535 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:42.530850  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:42.408039  3822 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.459055  4232 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:42.502216  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:42.535163  2230 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:42.527096  1658 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:42.423833  4165 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.307098  4172 consensus_queue.cc:1059] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.146s
I20260709 13:00:42.409561  4076 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:42.453903  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:42.564375  1611 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:00:42.573315  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 269544us
I20260709 13:00:42.574426  2236 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:00:42.569284  4162 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.672s	user 0.007s	sys 0.003s
I20260709 13:00:42.451699  4229 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.551041  1688 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:00:42.579393  1688 pending_rounds.cc:85] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 4
I20260709 13:00:42.579663  1688 pending_rounds.cc:107] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:00:42.579890  1688 raft_consensus.cc:2889] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:42.551829  4195 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 13:00:42.552307  4214 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:00:42.556234  3346 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.582254  3346 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.583498  3795 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.584414  3346 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:42.537569  4031 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.533288  4250 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:42.525787  4103 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.558142  4209 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.559583  4076 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:42.564303  4227 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:42.453006  3942 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:42.591591  3942 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.591807  4005 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.585254  4104 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:42.566443  4226 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.567425  3957 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.501029  4247 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.574507  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:42.545969  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:42.459374  4130 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.526261  3727 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.048s
W20260709 13:00:42.576689  4055 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.767s	user 0.007s	sys 0.004s
I20260709 13:00:42.575412  2228 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.589008  2234 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 4: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:42.575304  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:42.593184  1507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.139s	user 0.013s	sys 0.027s
I20260709 13:00:42.593081  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:42.534646  4173 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:42.576632  4197 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.576398  4063 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:42.593531  4206 consensus_meta.cc:287] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.621s	user 0.012s	sys 0.000s
I20260709 13:00:42.591849  4177 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.593582  4104 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:42.599617  4224 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:42.603349  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:42.614693  1507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.160s	user 0.016s	sys 0.027s
I20260709 13:00:42.615026  2233 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
W20260709 13:00:42.621752  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.622658  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:42.607582  4243 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
W20260709 13:00:42.594784  4221 consensus_queue.cc:1175] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "83f6f5810950417195f707ff308a8125" responder_term: 5 status { last_received { term: 4 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:00:42.607069  3727 consensus_queue.cc:1243] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:00:42.628235  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 1.17386e+06us
I20260709 13:00:42.594376  4235 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.317s
I20260709 13:00:42.623592  2232 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
W20260709 13:00:42.630229  1509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.004s	sys 0.002s
W20260709 13:00:42.649974  1509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.004s	sys 0.003s
I20260709 13:00:42.451407  4248 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.387665  4244 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.672421  4244 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:42.673072  2235 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:42.588482  4250 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:42.678228  4250 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:42.678521  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:42.679173  1688 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:42.619067  4263 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:42.535068  4241 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:42.605566  4209 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.438674  4204 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:42.654206  4126 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:42.652575  4165 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
W20260709 13:00:42.655925  4253 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:00:42.656164  4248 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:42.657023  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:42.637923  4224 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:42.636035  4173 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:42.627635  4206 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.707028  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:42.707751  1685 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.707913  4206 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:42.634137  4104 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:42.658864  4144 raft_consensus.cc:3055] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:42.708745  4144 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:42.709203  4104 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.709746  4144 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.606359  4247 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.714584  3822 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:42.714975  4247 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:42.667524  1540 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1795) took 1330 ms (client timeout 1702 ms). Trace:
I20260709 13:00:42.666872  4229 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:42.715768  1540 rpcz_store.cc:269] 0709 13:00:41.337499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.337645 (+   146us) service_pool.cc:224] Handling call
0709 13:00:42.667290 (+1329645us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54004191d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:42.667432 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":637410688}
I20260709 13:00:42.670328  1537 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:42.625746  3795 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:42.673496  4226 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:42.717374  4229 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:42.679872  1617 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:42.683148  4263 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.719446  4263 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:42.681171  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:42.721104  4175 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:42.721975  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:42.722772  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:42.723189  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:42.723498  1687 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:42.689904  4031 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:42.724510  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:42.726112  4212 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.000s
I20260709 13:00:42.726177  4232 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:42.726847  4232 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:42.726822  4212 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.726377  4177 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:42.728771  4177 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:42.729147  4177 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:42.731751  1583 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:42.692584  4198 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:42.685765  3984 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:42.694370  4204 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:42.695214  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:42.697078  1688 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 13:00:42.662794  4130 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.697425  1582 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:42.642849  4103 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.701951  3896 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.726218  4179 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:42.691403  4231 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.727422  4232 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:42.691805  4241 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:42.626583  4063 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:42.948513  1685 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:42.917307  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2153) took 1027 ms. Trace:
I20260709 13:00:43.082618  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1803) took 1724 ms. Trace:
I20260709 13:00:42.733870  1687 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:00:42.968853  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 1.08767e+06us
W20260709 13:00:43.091943  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.099958  1688 pending_rounds.cc:85] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 2
W20260709 13:00:42.822719  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.104926  1688 pending_rounds.cc:107] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 4.3
W20260709 13:00:42.793246  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:43.102272  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 370588us
I20260709 13:00:43.109898  1583 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:42.979665  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1843) took 1008 ms. Trace:
W20260709 13:00:42.829010  1507 leader_election.cc:336] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.816s (SENT)
I20260709 13:00:43.105239  1688 raft_consensus.cc:2889] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:43.102602  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 405204us
I20260709 13:00:42.994233  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2155) took 1094 ms. Trace:
I20260709 13:00:43.101212  1537 rpcz_store.cc:276] 0709 13:00:41.342765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.342947 (+   182us) service_pool.cc:224] Handling call
0709 13:00:41.369036 (+ 26089us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:42.670183 (+1301147us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b540035b9a0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa63dc0 00007fe08faee1c0 00007fe08faef231 00007fe08faebd8f 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c
0709 13:00:42.716916 (+ 46733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.067212 (+350296us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b540035b9a0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa63dc0 00007fe08faefa02 00007fe08faebdd6 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:00:43.067514 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46256640}
W20260709 13:00:43.113581  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:43.106982  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:43.110868  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 281916us
I20260709 13:00:43.110496  1617 rpcz_store.cc:276] 0709 13:00:41.970873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.971032 (+   159us) service_pool.cc:224] Handling call
0709 13:00:42.679697 (+708665us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:42.719376 (+ 39679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.979279 (+259903us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b54003c87d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:42.979580 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1581850880}
W20260709 13:00:42.912252  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.116556  1688 pending_rounds.cc:107] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:00:43.116710  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.116847  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:43.117579  2234 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:43.116935  1688 raft_consensus.cc:2889] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:43.119019  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:43.116864  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:43.120759  1507 leader_election.cc:336] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.828s (SENT)
W20260709 13:00:43.089957  4267 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
W20260709 13:00:43.121310  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 594354us
I20260709 13:00:43.121608  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:43.121703  1583 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:42.855495  4269 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 13:00:43.062323  4239 consensus_meta.cc:287] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.605s	user 0.010s	sys 0.000s
I20260709 13:00:43.122334  1582 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:43.099728  1690 rpcz_store.cc:276] 0709 13:00:41.889180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.365602 (+476422us) service_pool.cc:224] Handling call
0709 13:00:42.366101 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:42.366317 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:42.409061 (+ 42744us) log.cc:844] Serialized 14 byte log entry
0709 13:00:42.915925 (+506864us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b1800194e60. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c979c0a 00007fe0960b9219 00007fe08ca01a40 00007fe08c9fc4e4 00007fe08c9fc455 00007fe08fae4060 00007fe08faf6716 00007fe08fb07287 00007fe08fb07208
0709 13:00:42.916376 (+   451us) log.cc:844] Serialized 14 byte log entry
0709 13:00:42.916572 (+   196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:42.916627 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:42.916693 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.916766 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:42.916882 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2675,"prepare.queue_time_us":51244,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":2906,"raft.queue_time_us":631576,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":4088,"spinlock_wait_cycles":8288256}
I20260709 13:00:43.123191  1685 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:43.125079  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.133371  2235 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:43.051646  4212 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.137370  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:42.925124  4177 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.139698  4177 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.121663  4204 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:43.140441  4204 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:43.140869  4204 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:43.144636  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.021106  4202 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:43.128024  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:43.145599  2230 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.145694  4275 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:43.146457  4209 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:43.146878  4207 raft_consensus.cc:3055] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:43.147454  4207 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:42.913791  4270 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:00:42.840562  4038 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.966455  4166 raft_consensus.cc:3055] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:43.019536  4260 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:42.999351  3727 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.165820  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:43.166462  1610 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:43.166194  3727 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.163303  4166 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:43.168527  3727 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:42.794173  4031 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:42.786654  4175 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:43.111729  4188 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.135138  1688 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:00:42.831486  4076 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.169466  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:43.135826  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:42.686816  4250 consensus_queue.cc:1059] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.435s
I20260709 13:00:42.794942  4228 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:43.124625  4093 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:43.112052  3896 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.124655  4130 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:42.924892  2228 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:43.144474  4232 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.146456  4275 consensus_queue.cc:1243] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:43.128700  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.187696  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.168248  4166 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.112478  1691 rpcz_store.cc:276] 0709 13:00:41.899821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.391191 (+491370us) service_pool.cc:224] Handling call
0709 13:00:42.391457 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:42.391680 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:42.439586 (+ 47906us) log.cc:844] Serialized 14 byte log entry
0709 13:00:42.993845 (+554259us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:42.993901 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:42.993954 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.994028 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:42.994167 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":571547,"prepare.queue_time_us":36138,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":579504,"raft.run_cpu_time_us":51,"raft.run_wall_time_us":51}
I20260709 13:00:43.157127  1616 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [4.3-6.4]
W20260709 13:00:43.162503  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.163617  1612 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:43.163015  4263 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.165526  4196 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:43.178838  4170 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:43.178325  4212 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:43.170854  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:43.175374  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 719132us
I20260709 13:00:43.134752  3957 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.128044  4173 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:43.248198  1534 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:00:43.181070  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 60345.3us
I20260709 13:00:43.183502  1687 pending_rounds.cc:85] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 5
I20260709 13:00:43.249572  1687 pending_rounds.cc:107] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:00:43.249987  1687 raft_consensus.cc:2889] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:43.117610  3984 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.188898  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:43.261279  1543 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:00:42.966130  3800 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:42.794256  4205 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:43.191481  1658 connection.cc:256] RPC call timeout handler was delayed by -1.5614723310000613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
W20260709 13:00:43.265300  1658 connection.cc:256] RPC call timeout handler was delayed by -1.5452723310000351s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17200000000000001s for a response
I20260709 13:00:43.149379  4239 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.213826  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:43.177404  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:43.223383  2233 pending_rounds.cc:85] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 5
I20260709 13:00:43.176120  4177 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:43.276744  2233 pending_rounds.cc:107] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:00:43.277186  2233 raft_consensus.cc:2889] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:43.148319  4207 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.265077  3800 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:43.284832  4207 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:43.285485  4232 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.285265  3800 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.286386  3800 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:43.288947  1538 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:43.224005  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:43.227918  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.226533  1540 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:43.180351  4260 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:00:43.265724  1658 connection.cc:256] RPC call timeout handler was delayed by -1.4399723310000354s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
W20260709 13:00:43.291425  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2797723310000606s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2015s for a response
I20260709 13:00:43.292132  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:43.266330  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:43.153105  2230 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
W20260709 13:00:43.293458  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.165534  4038 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.117849  4231 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.190325  4249 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.176044  4246 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.284085  1609 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:43.290974  1536 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:43.192549  4202 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.305922  4202 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.306396  4224 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:00:43.307080  4224 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:43.307583  4224 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.308230  4202 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:43.270200  1541 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:00:43.309195  4269 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:00:43.309474  3984 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.309689  4224 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.294514  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.310186  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 500) took 1274 ms. Trace:
I20260709 13:00:43.311133  3896 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:43.178043  4175 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.293257  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:43.312026  1688 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:43.312860  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:43.312992  4038 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.313400  4175 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:43.168905  4204 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.189924  4093 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:43.315171  4093 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.315925  4204 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.317011  1535 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 7. Ops: []
I20260709 13:00:43.266716  4130 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.324627  4130 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.299705  3957 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.327723  4231 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.328645  3957 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.180102  4228 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.331715  4228 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:43.334216  4260 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.226749  4104 raft_consensus.cc:3055] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:43.337713  4104 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:43.338860  4104 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.340314  4104 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:43.247228  4173 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:43.341709  4173 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:43.316063  4239 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.179270  4188 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.347954  1618 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:43.349188  4188 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:43.180649  4076 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.299743  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.303488  4249 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.304399  4246 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.306245  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:43.308837  4063 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:43.361475  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.190325  4031 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.254266  4170 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:43.368942  4263 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:43.370036  4273 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:00:43.369395  4170 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.370925  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:43.314100  2236 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:43.300582  4250 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:43.302778  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.297801  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:43.385094  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:43.298393  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.385746  1691 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:43.358600  1543 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:43.360270  2230 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:43.310696  1539 rpcz_store.cc:276] 0709 13:00:42.035442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.035655 (+   213us) service_pool.cc:224] Handling call
0709 13:00:42.036077 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:42.046373 (+ 10296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:42.046402 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:42.066542 (+ 20140us) log.cc:844] Serialized 26 byte log entry
0709 13:00:42.584091 (+517549us) spinlock_profiling.cc:243] Waited 8 ms on lock 0x7b1800087fa0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:00:42.627611 (+ 43520us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:42.650494 (+ 22883us) log.cc:844] Serialized 14 byte log entry
0709 13:00:42.650695 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:42.650756 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:42.691180 (+ 40424us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b1800087fa0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff522 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:00:43.021325 (+330145us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b1800087fa0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff522 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:00:43.309994 (+288669us) raft_consensus.cc:1685] finished
0709 13:00:43.310013 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.310127 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9819,"prepare.queue_time_us":25109,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":22674,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":5727,"spinlock_wait_cycles":79269888,"thread_start_us":43317,"threads_started":1,"wal-append.queue_time_us":628792}
I20260709 13:00:43.363811  4249 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:43.365296  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 59133us
I20260709 13:00:43.367411  2231 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:00:43.368136  4031 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:43.373626  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.351979  2235 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:43.192677  4276 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.396603  1534 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:43.398578  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2145) took 1912 ms. Trace:
I20260709 13:00:43.401948  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:43.402861  1541 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:43.372087  4005 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:43.403035  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:43.424561  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:43.300935  4205 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.401070  3942 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:43.405129  4215 consensus_queue.cc:1059] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:43.402609  4275 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:00:43.407414  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.424535  1689 rpcz_store.cc:276] 0709 13:00:41.486533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.494651 (+  8118us) service_pool.cc:224] Handling call
0709 13:00:41.495009 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:41.726468 (+231459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:41.726530 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:42.726926 (+1000396us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b2c00144828. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:00:42.727375 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:00:42.842773 (+115398us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:42.910345 (+ 67572us) log.cc:844] Serialized 14 byte log entry
0709 13:00:43.131585 (+221240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.131665 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:43.371507 (+239842us) raft_consensus.cc:1685] finished
0709 13:00:43.398380 (+ 26873us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.398518 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":266816,"prepare.queue_time_us":68398,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":379807,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":3240,"spinlock_wait_cycles":52851456,"thread_start_us":345300,"threads_started":2,"wal-append.queue_time_us":449646}
I20260709 13:00:43.408430  1538 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 13:00:43.407500  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.414727  4076 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:43.384272  4263 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:43.440759  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:43.444046  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.190073  4241 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.420248  4246 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:43.423728  1508 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:43.423553  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.450295  1692 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:00:43.450708  1692 pending_rounds.cc:85] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 5
I20260709 13:00:43.450496  2230 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:43.451016  1692 pending_rounds.cc:107] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:00:43.451328  1692 raft_consensus.cc:2889] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:43.426800  2234 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:43.428550  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.452329  1543 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:43.428929  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.385905  4248 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:43.430290  1691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:00:43.432952  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:43.434875  1537 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:43.449215  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.449465  1685 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:43.477310  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44493.8us
I20260709 13:00:43.453256  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.484032  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.484858  1543 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:43.429051  4249 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:43.456054  2236 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:43.474004  1691 raft_consensus.cc:606] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:00:43.556356  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:43.556833  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 435176us
W20260709 13:00:43.563315  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 441002us
W20260709 13:00:43.567118  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.439352  4263 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:43.571923  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:43.572881  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.429713  4224 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.127s
I20260709 13:00:43.456019  4031 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:43.578908  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:43.576606  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 140523us
I20260709 13:00:43.587680  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:43.589201  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:43.590430  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.439777  4005 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.134s
I20260709 13:00:43.586752  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.597575  1537 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:43.596755  4076 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:00:43.598815  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.578361  4263 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:43.600329  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.429865  4276 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:43.603155  4276 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:43.605453  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:43.571538  4248 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.561340  4241 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.608203  4248 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:43.580789  4274 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:43.609097  4274 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.609597  4274 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.577111  4081 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:00:43.613726  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 511) took 1679 ms. Trace:
I20260709 13:00:43.571528  4177 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:43.615129  1613 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:43.614210  1683 rpcz_store.cc:276] 0709 13:00:41.934470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.416152 (+481682us) service_pool.cc:224] Handling call
0709 13:00:42.416591 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:42.970028 (+553437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:42.970058 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:43.118347 (+148289us) log.cc:844] Serialized 26 byte log entry
0709 13:00:43.373674 (+255327us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:43.373713 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.373780 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:43.613497 (+239717us) raft_consensus.cc:1685] finished
0709 13:00:43.613518 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.613655 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":552951,"raft.queue_time_us":602085,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"thread_start_us":236569,"threads_started":1,"wal-append.queue_time_us":305669}
I20260709 13:00:43.580749  4173 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.615990  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:43.617062  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:43.617760  4173 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.571202  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.620285  4205 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:43.580958  4250 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:43.621529  4188 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:00:43.437858  3942 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.596217  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:43.677033  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:43.678248  1688 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:43.679598  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:43.680658  1688 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:43.585896  4031 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:43.681975  4031 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:43.625700  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:43.686000  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.685675  4194 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.687175  4194 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:43.629204  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.689618  4076 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.690348  4076 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:43.691291  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.630172  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:43.691919  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:43.692430  2234 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:43.692555  4076 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.698697  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.700317  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:43.642478  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:43.693161  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:43.628368  1582 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:43.692540  1538 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.707250  3984 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.708015  3984 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.708559  4198 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.709877  4179 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:43.710371  4179 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:43.710726  4200 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:43.710790  4179 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:43.647432  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.714010  2230 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:43.715279  2230 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:43.716244  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:43.720727  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:43.722841  1689 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:00:43.724139  4179 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.726085  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2149) took 2058 ms. Trace:
I20260709 13:00:43.726653  1686 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:43.727029  1538 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:43.678126  1617 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:43.730192  1684 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:43.731106  2203 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:00:43.731747  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.731990  4144 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:43.732940  3984 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:43.733103  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 104838us
I20260709 13:00:43.734305  1582 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:43.735386  4076 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:43.736449  4227 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:43.736900  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:43.738008  4076 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:43.738220  4215 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:43.738552  4076 raft_consensus.cc:697] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:43.739639  4076 consensus_queue.cc:237] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.726629  1692 rpcz_store.cc:276] 0709 13:00:41.668151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.834286 (+166135us) service_pool.cc:224] Handling call
0709 13:00:41.866070 (+ 31784us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.469863 (+1603793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.725793 (+255930us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b540028ebd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:43.726014 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3615336320}
I20260709 13:00:43.609177  4241 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.738591  4215 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:43.631323  4030 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.712507  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:43.777866  1543 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:43.728056  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:43.778254  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.768400  2234 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:43.778779  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:43.830399  1685 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:43.829945  4031 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.777232  3984 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:43.832826  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.830541  2230 raft_consensus.cc:2368] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
I20260709 13:00:43.841289  1538 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:00:43.845721  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 108958us
I20260709 13:00:43.850281  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.745199  4246 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.859313  1688 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:00:43.737004  4227 raft_consensus.cc:697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:43.866735  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.866999  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.871297  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:43.876034  1536 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:43.879473  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 507) took 1731 ms. Trace:
I20260709 13:00:43.879931  1535 rpcz_store.cc:276] 0709 13:00:42.147754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.563196 (+415442us) service_pool.cc:224] Handling call
0709 13:00:42.563530 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.317510 (+753980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.879220 (+561710us) spinlock_profiling.cc:243] Waited 753 ms on lock 0x7b54000e92d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:43.879413 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1732284288}
I20260709 13:00:43.880357  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:43.875290  2203 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.841s (SENT) [suppressed 19 similar messages]
I20260709 13:00:43.883021  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.891847  1538 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
W20260709 13:00:43.890923  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.883713  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:43.895236  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.895864  1541 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:43.896730  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:43.862412  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 82342.4us
I20260709 13:00:43.899470  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:43.820498  4198 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.902338  1537 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:43.904605  2234 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
W20260709 13:00:43.774597  4103 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.017s	sys 0.007s
I20260709 13:00:43.897382  1541 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:43.774672  4241 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:43.909722  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 131668us
I20260709 13:00:43.875605  4246 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.876940  3984 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:00:43.920186  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 53306.4us
I20260709 13:00:43.849913  4130 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:43.922446  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.926370  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:43.886230  4036 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:43.853327  4204 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 13:00:43.819717  4030 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.939081  4274 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.994825  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.995488  1538 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:43.996222  4030 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.923043  4198 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:43.998402  4198 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:43.925954  3896 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 13:00:43.945353  3942 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:44.000121  4256 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:43.953714  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:44.000527  4256 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:44.001092  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 47486.7us
I20260709 13:00:44.001382  4256 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:44.001950  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:44.002693  4256 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.003078  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:43.954852  4248 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:43.956791  1692 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:44.006731  1692 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:44.007863  1692 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:43.957676  4227 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:43.973114  4103 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.018s	sys 0.008s
W20260709 13:00:43.979238  4260 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.005s	sys 0.009s
W20260709 13:00:44.010334  4260 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.005s	sys 0.010s
I20260709 13:00:44.011775  4257 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:43.958943  4239 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:44.012370  4257 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:43.960784  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.965632  2235 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:43.863529  4175 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:00:44.013943  1536 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
W20260709 13:00:43.966648  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.014904  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:43.745601  4250 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:44.015477  1536 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.017155  4250 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:44.017731  4250 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:44.018091  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:44.018368  4250 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:43.967661  1539 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.050s	sys 0.111s
W20260709 13:00:44.019676  1539 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.051s	sys 0.111s
I20260709 13:00:44.019527  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:43.909487  4179 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:43.970485  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.970695  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 707) took 1242 ms. Trace:
W20260709 13:00:43.975200  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.919262  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:43.977941  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 79901.2us
I20260709 13:00:44.021819  1584 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:43.929920  4063 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 13:00:43.996405  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:44.027045  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:44.028268  1538 raft_consensus.cc:2410] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:44.021062  2233 rpcz_store.cc:276] 0709 13:00:42.728241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.728371 (+   130us) service_pool.cc:224] Handling call
0709 13:00:42.728756 (+   385us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:00:43.296106 (+567350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.296135 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:00:43.302910 (+  6775us) log.cc:844] Serialized 78 byte log entry
0709 13:00:43.593718 (+290808us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:43.593777 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:43.714448 (+120671us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54002a35d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:43.714493 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:43.966882 (+252389us) raft_consensus.cc:1685] finished
0709 13:00:43.966902 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:43.970628 (+  3726us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":3334,"spinlock_wait_cycles":1136942848,"thread_start_us":290353,"threads_started":1,"wal-append.queue_time_us":625883}
I20260709 13:00:44.020677  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:44.029655  1543 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:44.030501  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:44.030779  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:44.031571  1543 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:44.032680  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:44.034014  4031 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:44.034312  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:44.035123  1538 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:44.036912  1692 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:44.045805  1543 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:44.047631  1543 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 13:00:44.048112  1543 pending_rounds.cc:85] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 4
I20260709 13:00:44.048386  1543 pending_rounds.cc:107] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 13:00:44.048254  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:44.049068  1687 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 2.
I20260709 13:00:44.050264  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:44.050851  1537 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:44.051196  1687 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:44.052168  1537 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:00:44.052240  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:44.054020  4282 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:44.055213  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:44.055946  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 78830.2us
I20260709 13:00:44.057662  4280 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:44.060154  4280 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.061627  4173 consensus_queue.cc:1059] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:44.062968  4173 raft_consensus.cc:3055] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:00:44.063413  4173 raft_consensus.cc:740] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:44.064427  4173 consensus_queue.cc:260] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:44.068658  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:44.069571  2234 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
W20260709 13:00:44.072439  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.054266  2201 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:44.076869  1536 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:44.079386  1537 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:43.978348  4104 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.638s	user 0.011s	sys 0.001s
W20260709 13:00:43.987546  4207 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.702s	user 0.009s	sys 0.002s
I20260709 13:00:44.089206  1543 raft_consensus.cc:2889] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:44.082564  1535 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:44.031921  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:44.093442  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.124506  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 102768us
I20260709 13:00:44.127341  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:44.129313  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.130625  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:44.131479  1691 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:44.132649  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:44.135027  1534 raft_consensus.cc:2410] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:44.136308  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:44.080701  4170 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.005s	sys 0.033s
I20260709 13:00:44.060804  4280 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:44.084817  1692 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:00:44.086464  1541 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:00:44.056005  1687 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
W20260709 13:00:44.087896  1690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.024s	sys 0.031s
W20260709 13:00:44.192548  1690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.028s	sys 0.036s
I20260709 13:00:44.193713  1537 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:43.929240  4215 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:44.195664  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 121243us
I20260709 13:00:44.196903  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:44.204468  1537 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:44.197929  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:44.129405  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:44.206198  1687 raft_consensus.cc:2410] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:44.198074  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:44.137861  4170 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.005s	sys 0.034s
I20260709 13:00:44.137881  3800 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:44.275641  1543 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:00:44.276381  1687 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8377b7234e4472ea27005f16dede669"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
W20260709 13:00:44.212023  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:43.977468  4241 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:44.195808  4093 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:43.994165  4130 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:44.209867  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.163005  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.137964  4248 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:00:44.123278  4198 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:44.287405  2237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.049s	sys 0.039s
W20260709 13:00:44.286630  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.288973  2237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.049s	sys 0.039s
W20260709 13:00:44.140750  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 221625us
W20260709 13:00:44.200232  1540 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.004s	sys 0.023s
I20260709 13:00:44.245010  1691 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:44.191929  4063 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:44.291294  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:44.278230  1687 raft_consensus.cc:606] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:00:44.299116  1687 raft_consensus.cc:614] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:44.207661  4081 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 13:00:44.083127  4257 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:44.253225  4173 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:44.210493  4281 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:44.123576  3942 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.252822  4228 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:44.129715  4274 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:44.381546  1534 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [4.3-6.4]
W20260709 13:00:44.383167  4196 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
W20260709 13:00:44.291716  1540 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.903s	user 0.004s	sys 0.023s
I20260709 13:00:44.254940  4260 raft_consensus.cc:3055] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:44.351967  1538 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:44.275426  3800 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:44.195647  4188 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.109s
I20260709 13:00:44.195495  4177 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.110s
I20260709 13:00:44.347206  4198 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:44.352674   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:00:44.298098  4063 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.290827  4248 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:44.359405  1543 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [4.3-6.4]
W20260709 13:00:44.362399  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.439725  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.440263  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.429786   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:00:44.293658  3833 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.015s	sys 0.012s
I20260709 13:00:44.350044  4281 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:44.371913  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.392915  4104 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:44.370886  4263 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.771s	user 0.008s	sys 0.003s
I20260709 13:00:44.352888  4093 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:44.353871  1583 leader_election.cc:400] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
I20260709 13:00:44.450230  1583 leader_election.cc:403] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:00:44.352993  4282 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:44.405154  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.363529  4257 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:44.398797  4194 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.711s	user 0.005s	sys 0.006s
I20260709 13:00:44.385982  4228 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:44.400602  3800 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:44.392663  4260 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:44.488474  1683 raft_consensus.cc:2410] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:44.439414  4248 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:44.492113  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2154) took 2588 ms. Trace:
W20260709 13:00:44.482710  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 129014us
I20260709 13:00:44.493943  1583 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:44.446069  3833 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.016s	sys 0.013s
I20260709 13:00:44.439538  4063 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:44.499192  1508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.005s	sys 0.037s
I20260709 13:00:44.445844  4196 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.450203  4281 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.492656  1686 rpcz_store.cc:276] 0709 13:00:41.892823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:42.347969 (+455146us) service_pool.cc:224] Handling call
0709 13:00:42.348291 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.727090 (+1378799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.481098 (+754008us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54000f6754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:44.481357 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3169170688}
I20260709 13:00:44.394491  3942 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:44.500509  1508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.005s	sys 0.037s
W20260709 13:00:44.508172  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 608750us
W20260709 13:00:44.510937  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.511710  1685 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.005s	sys 0.017s
W20260709 13:00:44.512182  1685 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.005s	sys 0.017s
W20260709 13:00:44.512954  1542 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.003s	sys 0.047s
W20260709 13:00:44.516897  1542 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.003s	sys 0.047s
I20260709 13:00:44.518461  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2314) took 1330 ms. Trace:
W20260709 13:00:44.514446  2203 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.006s	sys 0.016s
I20260709 13:00:44.449625  4175 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.054s
I20260709 13:00:44.489301  4228 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:44.521876  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 310160us
I20260709 13:00:44.484275  4093 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:44.498837  4170 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:44.495940  4204 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:44.496133  4200 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.510852  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:44.473709  4104 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:44.540342  4148 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.004s	sys 0.011s
W20260709 13:00:44.521013  2203 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.006s	sys 0.016s
I20260709 13:00:44.471009  4263 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:44.502243  3984 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:44.573992  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 57955.6us
W20260709 13:00:44.575810  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 700491us
W20260709 13:00:44.576622  2236 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.945s	user 0.038s	sys 0.099s
I20260709 13:00:44.519057  1542 rpcz_store.cc:276] 0709 13:00:43.187633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.192515 (+  4882us) service_pool.cc:224] Handling call
0709 13:00:43.253779 (+ 61264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:43.254263 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.254305 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:43.271744 (+ 17439us) log.cc:844] Serialized 26 byte log entry
0709 13:00:43.642039 (+370295us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:44.517517 (+875478us) log.cc:844] Serialized 14 byte log entry
0709 13:00:44.517930 (+   413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.518124 (+   194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.518183 (+    59us) raft_consensus.cc:1685] finished
0709 13:00:44.518201 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.518380 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13275,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":22353,"thread_start_us":1082491,"threads_started":2,"wal-append.queue_time_us":987194}
I20260709 13:00:44.538837  4030 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:44.576376  2228 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.011s	sys 0.007s
I20260709 13:00:44.573582  4031 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:44.579237  4260 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:44.595033  4031 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.595494  4260 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:44.574467  4148 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.004s	sys 0.012s
W20260709 13:00:44.577248  4076 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.832s	user 0.012s	sys 0.024s
W20260709 13:00:44.598308  4076 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.854s	user 0.012s	sys 0.024s
I20260709 13:00:44.572628  4200 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:44.598791  2228 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.011s	sys 0.007s
I20260709 13:00:44.595697  4031 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:44.581393  1508 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:44.606642  3984 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:44.607292  3984 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:44.608457  3984 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:44.613503  1508 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:44.582859  2236 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.952s	user 0.039s	sys 0.099s
I20260709 13:00:44.512876  4196 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:44.627282  2230 raft_consensus.cc:2368] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6.
W20260709 13:00:44.622743  4286 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 13:00:44.623775  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.577328  4239 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.021s	sys 0.009s
W20260709 13:00:44.623153  1658 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.006s	sys 0.032s
W20260709 13:00:44.636202  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.581555  4227 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.022s	sys 0.009s
W20260709 13:00:44.636848  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.623430  4130 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:44.579509  4241 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:44.637306  4239 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.021s	sys 0.009s
W20260709 13:00:44.639394  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.636158  1658 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.006s	sys 0.032s
W20260709 13:00:44.722112  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 751142us
W20260709 13:00:44.723016  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.623042  4280 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:44.644536  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.731775  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:44.682725  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.721665  4239 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:00:44.732921  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.734000  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.640332  4036 consensus_meta.cc:287] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.647s	user 0.007s	sys 0.003s
I20260709 13:00:44.730996  4076 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:44.738492  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:44.739741  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:44.742041  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:44.740636  2233 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:44.748853  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:44.749166  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:44.749692  2233 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.738585  4239 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:44.753119  4194 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:44.752625  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:44.754182  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.755954  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.733948  4256 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.731s	user 0.012s	sys 0.000s
I20260709 13:00:44.757712  1658 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:44.780192  1507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.018s	sys 0.046s
W20260709 13:00:44.780601  1507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.018s	sys 0.046s
W20260709 13:00:44.781322  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 778244us
I20260709 13:00:44.783032  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:44.735579  4241 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:44.773312  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:44.803503  1584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.007s	sys 0.017s
W20260709 13:00:44.843847  1584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.009s	sys 0.017s
W20260709 13:00:44.844463  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 715085us
I20260709 13:00:44.845674  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:44.824254  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.846619  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:44.804224  1509 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.008s	sys 0.016s
W20260709 13:00:44.848001  1509 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.010s	sys 0.017s
W20260709 13:00:44.848436  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 613755us
W20260709 13:00:44.856077  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 73167.5us
I20260709 13:00:44.765322  4194 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:44.779706  4205 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:44.803133  2231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.838s	user 0.016s	sys 0.046s
W20260709 13:00:44.858436  1541 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.012s	sys 0.039s
I20260709 13:00:44.858206  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:44.855216  1536 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.017s	sys 0.025s
I20260709 13:00:44.746454  4280 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.110s
I20260709 13:00:44.779805  4201 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:44.779882  4197 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:44.861608  1690 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.023s	sys 0.047s
W20260709 13:00:44.866334  1536 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.017s	sys 0.026s
I20260709 13:00:44.857010  3896 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:00:44.876989  1690 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.024s	sys 0.047s
W20260709 13:00:44.858650  1537 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.017s	sys 0.033s
W20260709 13:00:44.878744  1537 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.018s	sys 0.033s
W20260709 13:00:44.865912  2231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.901s	user 0.018s	sys 0.047s
I20260709 13:00:44.800038  4241 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:44.867650  1507 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:44.845674  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:44.863804  4205 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:44.890376  1687 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.010s	sys 0.038s
W20260709 13:00:44.893463  4291 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.384s	user 0.001s	sys 0.000s
W20260709 13:00:44.894953  4288 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 13:00:44.869357  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1330) took 1384 ms. Trace:
W20260709 13:00:44.869185  1541 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.012s	sys 0.039s
W20260709 13:00:44.870819  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.721995  4093 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:44.896381  1688 rpcz_store.cc:276] 0709 13:00:43.482477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.692943 (+210466us) service_pool.cc:224] Handling call
0709 13:00:44.864385 (+1171442us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54000aa5d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:44.866792 (+  2407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280106240}
I20260709 13:00:44.787397  4198 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:44.898545  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:44.800403  2229 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:44.873257  4130 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:44.861392  4256 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
W20260709 13:00:44.893573  1687 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.011s	sys 0.038s
W20260709 13:00:44.898301  1692 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.023s	sys 0.043s
I20260709 13:00:44.899933  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 716) took 1029 ms. Trace:
W20260709 13:00:44.640872  4227 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.023s	sys 0.009s
I20260709 13:00:44.912052  2230 rpcz_store.cc:276] 0709 13:00:43.870834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.870972 (+   138us) service_pool.cc:224] Handling call
0709 13:00:44.899734 (+1028762us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b540065a9d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:44.899872 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1734482304}
W20260709 13:00:44.902374  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56755.3us
I20260709 13:00:44.938988  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:44.889761  4241 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:44.915324  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 352980us
I20260709 13:00:44.892899  3896 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:44.948377  2234 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:44.876979  4197 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:44.888266  2232 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:44.914765  1692 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.023s	sys 0.043s
W20260709 13:00:44.941493  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 73946.1us
W20260709 13:00:44.966454  4289 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.382s	user 0.001s	sys 0.000s
I20260709 13:00:44.941170  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:44.969257  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:44.970587  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:44.971714  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:44.954934  4306 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:44.898826  4205 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.974050  4306 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:44.975109  4188 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:44.975049  4306 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:44.976668  3833 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:44.977141  3833 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.977708  3833 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.941061  4274 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:44.902807  3942 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:44.981365  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1346) took 1083 ms. Trace:
I20260709 13:00:44.981701  4312 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:44.981835  1692 rpcz_store.cc:276] 0709 13:00:43.897788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.300360 (+402572us) service_pool.cc:224] Handling call
0709 13:00:44.300631 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:44.301008 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:44.301037 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:44.334714 (+ 33677us) log.cc:844] Serialized 26 byte log entry
0709 13:00:44.965898 (+631184us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:44.965977 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.966057 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.981101 (+ 15044us) raft_consensus.cc:1685] finished
0709 13:00:44.981128 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.981294 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3700,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":6396,"thread_start_us":607779,"threads_started":1,"wal-append.queue_time_us":621301}
W20260709 13:00:44.983568  4295 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.353s	user 0.001s	sys 0.000s
I20260709 13:00:44.984715  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 880) took 1294 ms. Trace:
W20260709 13:00:44.984923  4293 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.345s	user 0.001s	sys 0.000s
I20260709 13:00:44.986109  4310 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:44.985170  2237 rpcz_store.cc:276] 0709 13:00:43.690260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.692944 (+  2684us) service_pool.cc:224] Handling call
0709 13:00:43.693225 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:43.693632 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.693659 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:43.694153 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 13:00:44.292610 (+598457us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:44.292654 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.292724 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.984426 (+691702us) raft_consensus.cc:1685] finished
0709 13:00:44.984447 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.984637 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18239,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":130,"spinlock_wait_cycles":9472,"thread_start_us":598073,"threads_started":1,"wal-append.queue_time_us":870349}
I20260709 13:00:44.986553  4310 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:44.986922  4310 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:44.987093  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 708) took 1394 ms. Trace:
I20260709 13:00:44.987449  2236 rpcz_store.cc:276] 0709 13:00:43.593143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.593391 (+   248us) service_pool.cc:224] Handling call
0709 13:00:43.593747 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:43.593947 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.593972 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:43.626759 (+ 32787us) log.cc:844] Serialized 26 byte log entry
0709 13:00:44.619230 (+992471us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:44.619288 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.619379 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.986869 (+367490us) raft_consensus.cc:1685] finished
0709 13:00:44.986886 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.987036 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":987916,"threads_started":1,"wal-append.queue_time_us":1258837}
I20260709 13:00:44.988276  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:44.901990  4104 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:44.945312  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1329) took 1463 ms. Trace:
I20260709 13:00:44.989355  1683 rpcz_store.cc:276] 0709 13:00:43.482134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.616842 (+134708us) service_pool.cc:224] Handling call
0709 13:00:44.945104 (+1328262us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b540003e154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:44.945255 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1981007104}
I20260709 13:00:44.990171  1684 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 13:00:44.990602  1684 pending_rounds.cc:85] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 2
I20260709 13:00:44.990491  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 511) took 1366 ms. Trace:
I20260709 13:00:44.990926  1684 pending_rounds.cc:107] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:00:44.991232  1684 raft_consensus.cc:2889] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:44.991775  1684 pending_rounds.cc:107] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:00:44.992101  1684 raft_consensus.cc:2889] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:44.992753  4321 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:44.904441  4259 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:44.952447  4076 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:44.932833  4309 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:44.993649  4259 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:44.994777  4309 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:44.991050  1536 rpcz_store.cc:276] 0709 13:00:43.623731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.193931 (+570200us) service_pool.cc:224] Handling call
0709 13:00:44.194350 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:44.194645 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:44.198497 (+  3852us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:44.199082 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 13:00:44.877254 (+678172us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:44.877301 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.877392 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.990270 (+112878us) raft_consensus.cc:1685] finished
0709 13:00:44.990291 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.990433 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":524757,"threads_started":1,"wal-append.queue_time_us":685935}
I20260709 13:00:44.993600  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 717) took 1113 ms. Trace:
W20260709 13:00:45.010493  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.012773  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1343) took 1267 ms. Trace:
W20260709 13:00:44.856860  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 83664.3us
I20260709 13:00:45.016789  1616 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:45.017608  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:45.018868  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 523) took 1153 ms. Trace:
I20260709 13:00:45.047179  1541 rpcz_store.cc:276] 0709 13:00:43.865232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.283017 (+417785us) service_pool.cc:224] Handling call
0709 13:00:44.283579 (+   562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.283827 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:44.900096 (+616269us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.018527 (+118431us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:45.018587 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:45.018637 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.018702 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.018816 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":519255,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":3492,"thread_start_us":713834,"threads_started":2}
I20260709 13:00:45.013442  1685 rpcz_store.cc:276] 0709 13:00:43.745126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.845103 (+ 99977us) service_pool.cc:224] Handling call
0709 13:00:43.845603 (+   500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:43.872069 (+ 26466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.872276 (+   207us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:43.879785 (+  7509us) log.cc:844] Serialized 26 byte log entry
0709 13:00:44.512735 (+632950us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:44.927544 (+414809us) log.cc:844] Serialized 14 byte log entry
0709 13:00:44.939078 (+ 11534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.939251 (+   173us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:45.012504 (+ 73253us) raft_consensus.cc:1685] finished
0709 13:00:45.012529 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.012700 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36735,"prepare.queue_time_us":466372,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":58838,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":25701,"thread_start_us":992998,"threads_started":2,"wal-append.queue_time_us":1111922}
I20260709 13:00:45.048545  1584 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:44.965757  4197 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:44.995189  4309 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:45.047183  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:45.056087  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:45.050608  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:45.001288  2231 rpcz_store.cc:276] 0709 13:00:43.879782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.890901 (+ 11119us) service_pool.cc:224] Handling call
0709 13:00:43.963997 (+ 73096us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:43.964438 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.964475 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:43.964968 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:00:44.903443 (+938475us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:44.911387 (+  7944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.942428 (+ 31041us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:44.993292 (+ 50864us) raft_consensus.cc:1685] finished
0709 13:00:44.993354 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:44.993540 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2219,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":4205,"thread_start_us":920673,"threads_started":1,"wal-append.queue_time_us":1025924}
I20260709 13:00:45.053242  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:45.001708  4321 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:45.057350  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:45.061123  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:45.064250  1614 raft_consensus.cc:2410] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:00:45.018618  4196 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:45.070655  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 519) took 1884 ms. Trace:
I20260709 13:00:45.021042  4170 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:00:45.086189  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.020421  4300 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:45.072438  1690 rpcz_store.cc:276] 0709 13:00:43.186373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.559233 (+372860us) service_pool.cc:224] Handling call
0709 13:00:43.559616 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.559823 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:43.578264 (+ 18441us) log.cc:844] Serialized 14 byte log entry
0709 13:00:44.878091 (+1299827us) log.cc:844] Serialized 14 byte log entry
0709 13:00:44.879837 (+  1746us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.070189 (+190352us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:45.070264 (+    75us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:45.070337 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.070441 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.070582 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14910,"prepare.queue_time_us":893124,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":110,"raft.queue_time_us":69593,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":35672,"spinlock_wait_cycles":101910016,"thread_start_us":1474069,"threads_started":3,"wal-append.queue_time_us":905786}
W20260709 13:00:45.106164  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 258325us
I20260709 13:00:45.055536  4263 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:45.144505  1540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.014s	sys 0.022s
W20260709 13:00:45.180022  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.180734  1542 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:00:45.180012  1540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.015s	sys 0.023s
I20260709 13:00:45.182035  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2321) took 1882 ms. Trace:
I20260709 13:00:45.182520  1540 rpcz_store.cc:276] 0709 13:00:43.300118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.302172 (+  2054us) service_pool.cc:224] Handling call
0709 13:00:43.372021 (+ 69849us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:43.372228 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:44.384776 (+1012548us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.181601 (+796825us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:45.181678 (+    77us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:45.181746 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.181836 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.181966 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1165726,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"thread_start_us":1791991,"threads_started":2}
W20260709 13:00:45.189702  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.072573  3942 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:45.211967  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.059497  4321 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:45.190686  1657 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:45.214619  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 521) took 1541 ms. Trace:
I20260709 13:00:45.218385  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:45.215123  1537 rpcz_store.cc:276] 0709 13:00:43.673147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.225636 (+552489us) service_pool.cc:224] Handling call
0709 13:00:44.226068 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.226425 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:44.879246 (+652821us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.214126 (+334880us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:45.214202 (+    76us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:45.214277 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.214382 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.214543 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31082,"prepare.queue_time_us":415627,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":80,"raft.queue_time_us":58597,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":36258,"thread_start_us":900119,"threads_started":3}
W20260709 13:00:45.233816  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.123456  4196 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:45.124459  4319 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:45.074538  4198 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.139s
I20260709 13:00:45.134032  4260 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:45.250571  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 202101us
I20260709 13:00:45.212642  4130 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:45.236820  4312 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:45.240278  4263 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:45.297969  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 236953us
I20260709 13:00:45.234827  1609 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:00:45.143234  4227 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.112s
I20260709 13:00:45.224323  3942 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:45.181880  4256 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.074s
W20260709 13:00:45.256160  4318 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 13:00:45.303426  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:45.306599  2233 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.002s
I20260709 13:00:45.297168  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1344) took 1516 ms. Trace:
I20260709 13:00:45.322613  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:45.341575  2233 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:45.301839  4148 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.300743  4198 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:45.320325  4326 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.265213  4260 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:45.338400  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 147805us
I20260709 13:00:45.182873  3896 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.341781  1691 rpcz_store.cc:276] 0709 13:00:43.780546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.296334 (+515788us) service_pool.cc:224] Handling call
0709 13:00:44.296805 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:44.297342 (+   537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:44.297384 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:44.348542 (+ 51158us) log.cc:844] Serialized 26 byte log entry
0709 13:00:44.881781 (+533239us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:45.124923 (+243142us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.296870 (+171947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.296941 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:45.296982 (+    41us) raft_consensus.cc:1685] finished
0709 13:00:45.296992 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.297109 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":97511,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":23,"raft.queue_time_us":112,"spinlock_wait_cycles":5430528,"thread_start_us":867029,"threads_started":3,"wal-append.queue_time_us":624887}
I20260709 13:00:45.253401  4196 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:45.356467  4031 consensus_meta.cc:287] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.755s	user 0.011s	sys 0.021s
W20260709 13:00:45.326679  4248 consensus_meta.cc:287] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.822s	user 0.023s	sys 0.006s
I20260709 13:00:45.324314  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:45.340180  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 532) took 1397 ms. Trace:
I20260709 13:00:45.257272  4319 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.339968  3942 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:45.418561  1687 rpcz_store.cc:276] 0709 13:00:43.942943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.301320 (+358377us) service_pool.cc:224] Handling call
0709 13:00:44.301627 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.301871 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:44.911995 (+610124us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.339805 (+427810us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:45.339860 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:45.339917 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.340002 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.340125 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":677061,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"thread_start_us":975168,"threads_started":3}
W20260709 13:00:45.413759  3800 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.911s	user 0.015s	sys 0.007s
W20260709 13:00:45.418260  4257 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.942s	user 0.014s	sys 0.009s
I20260709 13:00:45.418416  4227 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:45.369601  4246 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:45.437397  4063 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.925s	user 0.014s	sys 0.006s
W20260709 13:00:45.437964  4335 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:00:45.417933  2228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 713) took 1664 ms. Trace:
I20260709 13:00:45.451268  4246 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:00:45.444025  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 559) took 2328 ms. Trace:
I20260709 13:00:45.431332  4081 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.434671  4347 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:45.331350  4263 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.336570  4130 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.421705  4330 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.404976  4148 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.478338  4198 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:45.453603  4063 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.459857  4256 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.492072  4349 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:45.564623  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2332) took 2156 ms. Trace:
I20260709 13:00:45.490736  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:45.570071  1609 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:45.571221  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:45.454375  4257 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:00:45.495137  4287 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.997s	user 0.004s	sys 0.005s
I20260709 13:00:45.495999  4196 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:45.413722  4326 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:45.446591  4227 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:45.456679  2229 rpcz_store.cc:276] 0709 13:00:43.115648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.118145 (+  2497us) service_pool.cc:224] Handling call
0709 13:00:45.443798 (+2325653us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b540065bb54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:45.443933 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3775082368}
W20260709 13:00:45.500162  4188 consensus_meta.cc:287] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.008s
I20260709 13:00:45.503937  4309 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:45.505002  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.506008   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:45.451823  2228 rpcz_store.cc:276] 0709 13:00:43.753642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.755543 (+  1901us) service_pool.cc:224] Handling call
0709 13:00:43.916532 (+160989us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:43.943082 (+ 26550us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:44.601108 (+658026us) log.cc:844] Serialized 14 byte log entry
0709 13:00:44.951827 (+350719us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:45.069183 (+117356us) log.cc:844] Serialized 26 byte log entry
0709 13:00:45.069535 (+   352us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:45.179430 (+109895us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.341527 (+162097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.358434 (+ 16907us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:45.366269 (+  7835us) raft_consensus.cc:1685] finished
0709 13:00:45.368207 (+  1938us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.413208 (+ 45001us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27612,"prepare.queue_time_us":1103700,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":76,"raft.queue_time_us":152391,"raft.run_cpu_time_us":451,"raft.run_wall_time_us":83012,"thread_start_us":1077893,"threads_started":3,"wal-append.queue_time_us":378660}
I20260709 13:00:45.506908  4342 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.510382  4274 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:45.510737  4260 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:45.511066  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:45.629441  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.496001  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 525) took 2127 ms. Trace:
I20260709 13:00:45.326179  4170 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.421274  4031 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.421533  4248 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:45.517729  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:45.518270  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.445686  3800 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:45.530349  3984 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.018s	sys 0.032s
W20260709 13:00:45.532871  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 73229.4us
W20260709 13:00:45.515221  4205 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.008s
I20260709 13:00:45.540539  3942 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:45.524416  4204 consensus_meta.cc:287] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.951s	user 0.011s	sys 0.000s
I20260709 13:00:45.555199  1537 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:00:45.714346  1537 pending_rounds.cc:85] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 3
I20260709 13:00:45.714690  1537 pending_rounds.cc:107] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:00:45.714984  1537 raft_consensus.cc:2889] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:45.716485  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:45.558804  4263 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:45.559021  4282 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.075s	user 0.012s	sys 0.000s
W20260709 13:00:45.560976  4173 consensus_meta.cc:287] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.210s	user 0.011s	sys 0.000s
I20260709 13:00:45.514753  4319 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:45.561975  4198 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.562927  4148 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:45.551242  4330 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:45.564886  1617 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 6. Ops: [4.5-4.5]
I20260709 13:00:45.565025  1538 rpcz_store.cc:276] 0709 13:00:43.408269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.035907 (+627638us) service_pool.cc:224] Handling call
0709 13:00:44.036305 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.393575 (+357270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.564399 (+1170824us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54000e83d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:45.564567 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725064960}
I20260709 13:00:45.568082  4256 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.568524  4063 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:45.569142  4349 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:45.759292  4349 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:45.571187  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:45.761534  4148 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.571993  1540 raft_consensus.cc:2368] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7.
I20260709 13:00:45.762333  4148 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.545503  4347 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:45.605945  4103 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.614632  4344 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.615664  4321 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:45.617717  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:45.730103  4263 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:45.622195  1543 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:45.617774  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:45.584506  4241 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.607s	user 0.015s	sys 0.005s
W20260709 13:00:45.597848  4281 consensus_meta.cc:287] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.024s	user 0.009s	sys 0.005s
W20260709 13:00:45.617321  3833 consensus_meta.cc:287] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.012s
W20260709 13:00:45.685205  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 362649us
I20260709 13:00:45.799261  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:45.633172  1689 rpcz_store.cc:276] 0709 13:00:43.368581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.724842 (+356261us) service_pool.cc:224] Handling call
0709 13:00:44.864834 (+1139992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.865027 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:44.865051 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:44.865091 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:44.865124 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.865179 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.495826 (+630647us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54000aa5d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:45.495940 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2613721984}
W20260709 13:00:45.689572  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:45.690218  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.606329  4338 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.613883  4188 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:45.697190  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:45.807678  1690 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:45.629536  4274 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:45.719714  4205 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.717351  1537 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
W20260709 13:00:45.748163  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.708745  4076 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.711061  4030 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:45.761447  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:45.490212  4081 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:45.573345  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2323) took 2167 ms. Trace:
I20260709 13:00:45.822953  1539 rpcz_store.cc:276] 0709 13:00:43.405485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.405618 (+   133us) service_pool.cc:224] Handling call
0709 13:00:43.405951 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:43.406282 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:43.406315 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:43.406793 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 13:00:44.020086 (+613293us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:44.020123 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:44.020184 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:45.494528 (+1474344us) spinlock_profiling.cc:243] Waited 65.4 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:00:45.573069 (+ 78541us) raft_consensus.cc:1685] finished
0709 13:00:45.573088 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:45.573270 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2648,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":139,"spinlock_wait_cycles":150334336,"thread_start_us":612910,"threads_started":1,"wal-append.queue_time_us":1018259}
I20260709 13:00:45.725199  4204 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
W20260709 13:00:45.765017  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.713591  4207 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.732822  4282 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:45.630151  4342 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.868299  4207 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.869635  1612 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:45.870352  4342 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:45.773236  1541 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:45.776993  4344 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:45.871363  4207 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:45.872524  4344 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:45.619305  1611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1846) took 2464 ms (client timeout 2212 ms). Trace:
W20260709 13:00:45.874715  1611 rpcz_store.cc:269] 0709 13:00:43.154783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.154890 (+   107us) service_pool.cc:224] Handling call
0709 13:00:45.619067 (+2464177us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400389354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:45.619218 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2417170176}
I20260709 13:00:45.876202  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:45.734551  4312 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.878679  4312 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:45.880874  4312 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:45.732931  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:45.694795  4337 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.135s
W20260709 13:00:45.883081  4360 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:00:45.765607  4228 consensus_meta.cc:287] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 1.194s	user 0.011s	sys 0.002s
I20260709 13:00:45.885110  4228 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.886549  4321 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:45.887250  4228 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:45.791903  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.805778  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:45.889353  4343 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:00:45.703194  3984 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.091s	user 0.018s	sys 0.032s
I20260709 13:00:45.813802  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:45.890086  4358 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 13:00:45.814460  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:45.819821  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 54400.4us
I20260709 13:00:45.890836  1683 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:45.811177  4257 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.821177  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:45.825073  1690 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:45.771153  4347 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:45.711959  4227 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.864007  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:45.910916  1691 raft_consensus.cc:2368] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6.
W20260709 13:00:45.829150  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:45.822057  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:45.809599  4260 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.889467  1687 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:45.912824  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:45.913591  1687 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:45.897588  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:45.915741  1611 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:45.799117  4241 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.863517  4351 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:45.801106  4263 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.776916  4319 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:00:45.904174  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:45.812487  4326 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:45.905355  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:45.885146  3800 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:45.907711  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:45.807781  3833 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:45.802562  4281 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.933653  1687 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:45.891572  4246 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:45.897221  1541 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:45.779824  4103 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.953524  1683 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:45.906090  1613 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:00:45.923345  4260 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:45.922086  4256 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:45.972241  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 80894.4us
I20260709 13:00:45.905352  4228 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:45.981736  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 44547.3us
I20260709 13:00:45.921108  4227 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:45.908470  4030 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.811898  4338 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:45.977680  4310 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.990s	user 0.010s	sys 0.013s
I20260709 13:00:45.862237  4076 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.922446  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1333) took 2439 ms. Trace:
I20260709 13:00:45.977813  4309 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:46.006251  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 98713.9us
I20260709 13:00:45.808666  4349 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:46.008682  1610 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:45.910221  4257 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:46.031113  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:46.032866  4361 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:00:46.033068  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:45.986790  4342 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:45.916872  4352 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.144s
I20260709 13:00:46.010002  4031 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:46.039678  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:46.034260  4362 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
W20260709 13:00:46.041002  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 262086us
I20260709 13:00:46.057297  1611 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:45.917985  4357 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:46.051595  4263 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:46.053136  4281 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:45.890108  4343 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:46.060928  4342 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:46.062925  4343 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:46.060535  4370 log.cc:927] Time spent T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:00:46.060420  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:46.065882  1611 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.065943  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 438) took 1003 ms. Trace:
I20260709 13:00:46.066447  1609 rpcz_store.cc:276] 0709 13:00:45.062622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.996475 (+933853us) service_pool.cc:224] Handling call
0709 13:00:46.065862 (+ 69387us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:46.067502  4148 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:46.067747  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:46.069140  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:46.068328  4347 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:45.988052  4246 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:46.113258  1614 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
I20260709 13:00:46.113997  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 439) took 1046 ms. Trace:
I20260709 13:00:46.114452  1614 rpcz_store.cc:276] 0709 13:00:45.067658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.617563 (+549905us) service_pool.cc:224] Handling call
0709 13:00:46.113927 (+496364us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:46.083014  4347 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:46.092823  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:46.119386  1688 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:46.120760  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:46.121534  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:46.095716  4260 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:46.097012  4330 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:46.105060  4076 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:46.152287  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 121280us
I20260709 13:00:46.153247  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:46.106583  4030 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:46.034646  4310 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:46.110905  4205 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:46.111114  1509 connection.cc:256] RPC call timeout handler was delayed by -1.1950832420000097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32220000000000004s for a response
I20260709 13:00:46.054345  4248 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:46.111944  3984 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:46.094283  4292 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.557s	user 0.006s	sys 0.004s
I20260709 13:00:46.122165  1535 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:46.053194  4259 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:46.123294  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:46.124128  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:46.185185  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:46.185076  1614 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:46.185851  1538 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:46.187084  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:46.147859  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.148065  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:46.190178  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.190347  1536 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:46.155392  4367 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:46.035404  1684 rpcz_store.cc:276] 0709 13:00:43.483356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:43.731121 (+247765us) service_pool.cc:224] Handling call
0709 13:00:44.952240 (+1221119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.992742 (+ 40502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:45.922208 (+929466us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540003e150. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:45.922369 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2794812800}
I20260709 13:00:46.149015  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:46.156970  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:46.156970  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:46.198037  1541 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:46.198403  1537 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:46.199353  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:46.199553  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:46.122714  4197 consensus_meta.cc:287] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.045s	user 0.011s	sys 0.000s
I20260709 13:00:46.018939  4228 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:46.184046  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.193642  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:45.955152  4351 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:46.162156  4148 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:46.189710  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:46.194612  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:46.208748  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.195219  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:46.196627  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:46.197747  1539 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:46.210153  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:46.205996  1618 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:46.211392  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:46.212095  1537 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:46.212258  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:46.213044  1536 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:46.213268  1537 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:00:46.080488  4357 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:46.215292  1688 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:46.224459  2233 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:46.225531  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:46.221684  1614 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:46.219643  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:46.230028  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:46.230973  1614 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:46.232712  1611 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:46.235647  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:46.241063  1611 raft_consensus.cc:2368] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6.
I20260709 13:00:46.238302  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:46.230078  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:46.271973  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:46.272337  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:46.273243  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:46.171072  4379 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:00:46.258757  1538 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:46.267110  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:46.277495  4342 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:46.277967  4342 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:46.278328  4342 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:46.269287  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.275863  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:46.208114  4148 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:46.284379  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:46.268674  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 58583.2us
I20260709 13:00:46.289301  1690 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:46.193845  3984 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:46.220654  4260 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:46.296140  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:46.300609  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:46.300117  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:46.220849  4330 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:46.206676  4197 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:46.247444  4207 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:46.310102  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.320513  1536 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:46.284111  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:46.264158  4241 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:46.283464  4292 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:00:46.255904  4383 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:46.256572  4347 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
I20260709 13:00:46.205281  4367 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:46.825896  1658 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.002s	sys 0.007s
W20260709 13:00:46.378240  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 213809us
W20260709 13:00:47.274452  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:47.259231  1655 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.023s	sys 0.011s
W20260709 13:00:47.275128  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.274951  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:47.275183  1655 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.023s	sys 0.012s
I20260709 13:00:47.275769  1688 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 3.
I20260709 13:00:47.276372  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1355) took 1057 ms. Trace:
I20260709 13:00:47.276587  4166 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:47.276811  1688 rpcz_store.cc:276] 0709 13:00:46.218664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.651476 (+432812us) service_pool.cc:224] Handling call
0709 13:00:47.276308 (+624832us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:47.277752  4342 consensus_meta.cc:287] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.999s	user 0.011s	sys 0.000s
I20260709 13:00:46.424485  1541 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:46.640563  3942 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:46.802945  2201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.014s	sys 0.040s
I20260709 13:00:47.280155  4342 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:47.285501  1692 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:47.288565  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1838) took 1113 ms. Trace:
I20260709 13:00:47.289125  1536 rpcz_store.cc:276] 0709 13:00:46.175347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.211830 (+ 36483us) service_pool.cc:224] Handling call
0709 13:00:47.288455 (+1076625us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.290548  4342 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:46.333966  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:46.362202  1690 raft_consensus.cc:2410] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:00:47.273481  1658 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.002s	sys 0.019s
W20260709 13:00:47.280999  2201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.241s	user 0.018s	sys 0.047s
W20260709 13:00:47.302770  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 977573us
I20260709 13:00:47.303787  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:47.302814  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 1.27486e+06us
I20260709 13:00:47.280473  4383 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:46.297224  4148 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:46.642465  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:46.471763  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.555717  4338 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:46.431957  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.552716  1684 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:00:46.346081  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 445) took 1163 ms. Trace:
I20260709 13:00:46.348219  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:47.328398  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 434) took 2300 ms. Trace:
I20260709 13:00:47.329118  1611 rpcz_store.cc:276] 0709 13:00:45.182304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.270821 (+1088517us) service_pool.cc:224] Handling call
0709 13:00:46.346005 (+ 75184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.282739  2229 raft_consensus.cc:2410] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:00:46.296423  4376 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.992s
I20260709 13:00:46.372078  4330 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:46.802603  4368 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.015s	sys 0.074s
W20260709 13:00:46.751083  4063 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.006s	sys 0.006s
I20260709 13:00:47.314898  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:46.940743  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 428) took 1993 ms. Trace:
I20260709 13:00:47.314929  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:47.317526  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 2.195e+06us
I20260709 13:00:46.268148  4257 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:47.278666  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2184) took 1239 ms. Trace:
I20260709 13:00:47.356729  1688 rpcz_store.cc:276] 0709 13:00:46.038989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.277889 (+1238900us) service_pool.cc:224] Handling call
0709 13:00:47.278193 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.278364 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.278387 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:47.278426 (+    39us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:47.278461 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.278513 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.278618 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.358090  1618 rpcz_store.cc:276] 0709 13:00:44.946913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.947501 (+   588us) service_pool.cc:224] Handling call
0709 13:00:44.947906 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:46.211077 (+1263171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.940364 (+729287us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54000bf4d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:46.940647 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2892849024}
I20260709 13:00:47.358906  1688 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:46.285782  3833 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:47.323014  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 989038us
I20260709 13:00:47.359489  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2185) took 1296 ms. Trace:
I20260709 13:00:47.359856  3833 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:47.359896  1688 rpcz_store.cc:276] 0709 13:00:46.063164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.358109 (+1294945us) service_pool.cc:224] Handling call
0709 13:00:47.358531 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.359272 (+   741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.359440 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.360458  3833 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:46.641450  3984 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:47.329358  1537 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:47.329627  1612 rpcz_store.cc:276] 0709 13:00:45.017757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.017956 (+   199us) service_pool.cc:224] Handling call
0709 13:00:45.049252 (+ 31296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.870106 (+820854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.317651 (+1447545us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b54003c91d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:47.317842 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1886265216}
I20260709 13:00:47.329718  1543 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
W20260709 13:00:46.655212  4390 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:00:46.653144  4274 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.821s	user 0.008s	sys 0.000s
I20260709 13:00:47.340130  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:46.658447  4384 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 13:00:46.660060  4385 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:00:46.347041  4081 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:47.213858  4375 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.008s	user 0.040s	sys 0.059s
I20260709 13:00:47.440440  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.441612  1612 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:46.350839  4260 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:46.673627  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:47.462314  4197 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:47.463110  2228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 890) took 1831 ms (client timeout 1850 ms). Trace:
I20260709 13:00:47.463587  4197 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:47.463685  2228 rpcz_store.cc:269] 0709 13:00:45.631325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.689561 (+ 58236us) service_pool.cc:224] Handling call
0709 13:00:46.673354 (+983793us) spinlock_profiling.cc:243] Waited 5.95 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:47.463009 (+789655us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13673728}
I20260709 13:00:47.464649  4197 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:47.345225  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:46.333648  3896 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:47.466157  3896 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:47.468014  4260 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:47.468888  3896 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:46.675836  4246 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:46.643625  4256 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:47.469777  4256 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.470211  4256 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:47.471088  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 429) took 2523 ms. Trace:
I20260709 13:00:47.436846  4274 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:47.471477  1617 rpcz_store.cc:276] 0709 13:00:44.947291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.963207 (+ 15916us) service_pool.cc:224] Handling call
0709 13:00:44.989314 (+ 26107us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:45.764728 (+775414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.470806 (+1706078us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b54000875d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:47.471028 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1323089664}
I20260709 13:00:47.472675  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:47.473076  4274 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:47.362438  4342 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:47.362536  3942 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:47.355409  1609 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
W20260709 13:00:47.362799  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 59124us
I20260709 13:00:46.657369  4200 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:47.363440  1616 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:46.647167  4351 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:46.648844  4228 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:47.364655  1507 leader_election.cc:336] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.369s (SENT)
I20260709 13:00:47.508795  4330 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:47.508813  1616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1857) took 2479 ms (client timeout 1550 ms). Trace:
W20260709 13:00:47.509397  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 144757us
I20260709 13:00:47.359479  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:46.638759  4310 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:47.392257  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1841) took 1116 ms. Trace:
W20260709 13:00:47.511282  1609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 453) took 1645 ms (client timeout 1709 ms). Trace:
W20260709 13:00:47.351871  4368 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.294s	user 0.029s	sys 0.086s
I20260709 13:00:47.331331  4215 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:47.432438  1611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 454) took 1531 ms (client timeout 1803 ms). Trace:
W20260709 13:00:47.524608  1611 rpcz_store.cc:269] 0709 13:00:45.901539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.339866 (+1438327us) service_pool.cc:224] Handling call
0709 13:00:47.432369 (+ 92503us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.525702  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:47.526538  1611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 458) took 1556 ms (client timeout 1937 ms). Trace:
I20260709 13:00:47.526746  4383 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:47.526912  1611 rpcz_store.cc:269] 0709 13:00:45.970251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.525383 (+1555132us) service_pool.cc:224] Handling call
0709 13:00:47.526479 (+  1096us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.528079  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.511243  1543 rpcz_store.cc:276] 0709 13:00:46.275489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.282775 (+  7286us) service_pool.cc:224] Handling call
0709 13:00:47.392199 (+1109424us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.528936  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 459) took 1542 ms. Trace:
I20260709 13:00:47.338449  4380 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:47.529388  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:47.530289  1543 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
I20260709 13:00:47.531561  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:47.532152  1618 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:47.532709  4310 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:47.364374  4227 consensus_meta.cc:287] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 1.346s	user 0.006s	sys 0.006s
W20260709 13:00:47.354432  4063 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.401s	user 0.011s	sys 0.015s
I20260709 13:00:47.533201  4310 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:47.395685  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 537) took 1102 ms. Trace:
W20260709 13:00:47.444602  1612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1349) took 1875 ms (client timeout 2170 ms). Trace:
I20260709 13:00:47.354387  4198 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:47.331444  4103 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:47.534170  1537 rpcz_store.cc:276] 0709 13:00:46.292970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.642208 (+349238us) service_pool.cc:224] Handling call
0709 13:00:47.395609 (+753401us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:47.534354  1612 rpcz_store.cc:269] 0709 13:00:45.568874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.432120 (+1863246us) service_pool.cc:224] Handling call
0709 13:00:47.444535 (+ 12415us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.534971  4103 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.535342  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:47.535560  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.535488  4103 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:47.458931  4389 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:47.536429  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 455) took 1628 ms. Trace:
I20260709 13:00:47.536569  4389 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:47.362659  3833 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:47.537011  4389 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:47.465368  4197 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:47.473208  1617 raft_consensus.cc:2376] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4. Re-sending same reply.
I20260709 13:00:46.661715  4357 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:47.434042  3984 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:47.537794  1617 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 457) took 1596 ms (client timeout 1866 ms). Trace:
W20260709 13:00:47.538185  1617 rpcz_store.cc:269] 0709 13:00:45.941660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.472401 (+1530741us) service_pool.cc:224] Handling call
0709 13:00:47.537737 (+ 65336us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.538111  3984 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:47.538961  4197 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:47.539171  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.539778  4103 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:47.540217  3984 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:47.395287  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 891) took 1182 ms. Trace:
I20260709 13:00:47.479241  4194 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.352185  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:47.480747  4392 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:47.441005  4375 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.237s	user 0.042s	sys 0.061s
I20260709 13:00:47.395852  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2183) took 1381 ms. Trace:
I20260709 13:00:47.281814  4207 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:47.511757  1609 rpcz_store.cc:269] 0709 13:00:45.861920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.329335 (+467415us) service_pool.cc:224] Handling call
0709 13:00:47.507213 (+1177878us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46464}
I20260709 13:00:47.505801  4338 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:47.529635  4380 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:47.339838  4241 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:47.529354  1611 rpcz_store.cc:276] 0709 13:00:45.986842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.527789 (+1540947us) service_pool.cc:224] Handling call
0709 13:00:47.528878 (+  1089us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.535946  1537 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:00:47.458781  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2180) took 1517 ms. Trace:
I20260709 13:00:47.474208  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:47.540022  3942 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:47.478582  4256 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:47.539798  1617 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
W20260709 13:00:47.598196  1617 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 456) took 1678 ms (client timeout 2187 ms). Trace:
W20260709 13:00:47.598691  1617 rpcz_store.cc:269] 0709 13:00:45.919484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.538917 (+1619433us) service_pool.cc:224] Handling call
0709 13:00:47.598118 (+ 59201us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:47.510251  1507 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.602s (SENT)
I20260709 13:00:47.599830  1507 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:47.541483  1684 rpcz_store.cc:276] 0709 13:00:45.992660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.211150 (+218490us) service_pool.cc:224] Handling call
0709 13:00:46.328154 (+117004us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.334442 (+1006288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.373991 (+ 39549us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.536794  1612 rpcz_store.cc:276] 0709 13:00:45.908020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.535321 (+1627301us) service_pool.cc:224] Handling call
0709 13:00:47.536371 (+  1050us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.587188  1691 rpcz_store.cc:276] 0709 13:00:45.941271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.941499 (+   228us) service_pool.cc:224] Handling call
0709 13:00:45.941900 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:45.942341 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:45.942381 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:45.972027 (+ 29646us) log.cc:844] Serialized 26 byte log entry
0709 13:00:46.179567 (+207540us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:46.179622 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:46.179694 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:47.458508 (+1278814us) raft_consensus.cc:1685] finished
0709 13:00:47.458531 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.458702 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":23111,"thread_start_us":205632,"threads_started":1,"wal-append.queue_time_us":341335}
I20260709 13:00:47.545964  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:47.541105  4381 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.509958  4396 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.589382  1542 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:00:47.593458  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:47.620748  1609 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:47.594328  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:47.543260  1618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1342) took 2094 ms (client timeout 2285 ms). Trace:
W20260709 13:00:47.621354  1618 rpcz_store.cc:269] 0709 13:00:45.449894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.359262 (+1909368us) service_pool.cc:224] Handling call
0709 13:00:47.543180 (+183918us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:47.621647  1611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1350) took 2052 ms (client timeout 2662 ms). Trace:
W20260709 13:00:47.622403  1611 rpcz_store.cc:269] 0709 13:00:45.569143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.594084 (+2024941us) service_pool.cc:224] Handling call
0709 13:00:47.621566 (+ 27482us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:47.509508  1616 rpcz_store.cc:269] 0709 13:00:45.029200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.063005 (+ 33805us) service_pool.cc:224] Handling call
0709 13:00:46.934344 (+1871339us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:47.508731 (+574387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6140544}
I20260709 13:00:47.622818  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.623842  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.624593  1616 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:47.625633  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.540845  3984 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:47.626364  1616 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:47.627465  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1336) took 2254 ms. Trace:
I20260709 13:00:47.627948  1616 rpcz_store.cc:276] 0709 13:00:45.373222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.625332 (+2252110us) service_pool.cc:224] Handling call
0709 13:00:47.627397 (+  2065us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.614344  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.612221  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2188) took 1548 ms. Trace:
I20260709 13:00:47.629204  1691 rpcz_store.cc:276] 0709 13:00:46.064117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.611129 (+1547012us) service_pool.cc:224] Handling call
0709 13:00:47.611518 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.611748 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.611793 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:47.611854 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:47.611907 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.611990 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.612139 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.630970  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:47.632366  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 122149us
I20260709 13:00:47.585440  4380 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:47.633033  1507 connection.cc:256] RPC call timeout handler was delayed by -2.100982914000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17070000000000002s for a response
I20260709 13:00:47.636339  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:47.634058  1617 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1341) took 2196 ms (client timeout 2806 ms). Trace:
I20260709 13:00:47.637153  1618 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
W20260709 13:00:47.637352  1617 rpcz_store.cc:269] 0709 13:00:45.438997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.613962 (+2174965us) service_pool.cc:224] Handling call
0709 13:00:47.633990 (+ 20028us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.637180  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:47.638386  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:47.540850  4200 consensus_queue.cc:1059] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 13:00:47.579241  4194 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:47.645572  1537 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:47.655956  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:47.524875  4148 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:47.639364  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:47.652441  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:47.659305  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 465) took 1013 ms. Trace:
I20260709 13:00:47.659703  1617 rpcz_store.cc:276] 0709 13:00:46.646176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.652074 (+1005898us) service_pool.cc:224] Handling call
0709 13:00:47.659231 (+  7157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.660826  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:47.652878  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 463) took 1015 ms. Trace:
I20260709 13:00:47.661964  1611 rpcz_store.cc:276] 0709 13:00:46.645550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.636863 (+991313us) service_pool.cc:224] Handling call
0709 13:00:47.652795 (+ 15932us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.663087  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.642969  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:47.665035  1617 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:00:47.540800  2229 rpcz_store.cc:276] 0709 13:00:46.192394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.193645 (+  1251us) service_pool.cc:224] Handling call
0709 13:00:47.373724 (+1180079us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.639393  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 461) took 1002 ms. Trace:
I20260709 13:00:47.512658  4342 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:47.666754  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1354) took 2060 ms. Trace:
I20260709 13:00:47.667256  1618 rpcz_store.cc:276] 0709 13:00:46.636902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.638086 (+1001184us) service_pool.cc:224] Handling call
0709 13:00:47.639308 (+  1222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.667644  1611 rpcz_store.cc:276] 0709 13:00:45.606658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.662766 (+2056108us) service_pool.cc:224] Handling call
0709 13:00:47.666694 (+  3928us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.668331  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.668967  1618 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:47.669477  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1351) took 2100 ms. Trace:
I20260709 13:00:47.669780  1618 rpcz_store.cc:276] 0709 13:00:45.569444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.668076 (+2098632us) service_pool.cc:224] Handling call
0709 13:00:47.669397 (+  1321us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.670624  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:47.671188  1618 raft_consensus.cc:2376] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7. Re-sending same reply.
I20260709 13:00:47.640200  2232 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:47.619146  4197 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:47.672956  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.681859  1686 raft_consensus.cc:2410] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:47.674090  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.582047  4392 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:47.676652  1611 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [4.3-6.4]
I20260709 13:00:47.691753  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 1856) took 2662 ms. Trace:
I20260709 13:00:47.692281  1611 rpcz_store.cc:276] 0709 13:00:45.028914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.676039 (+2647125us) service_pool.cc:224] Handling call
0709 13:00:47.676396 (+   357us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:47.691335 (+ 14939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.691670 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:47.680212  1509 connection.cc:256] RPC call timeout handler was delayed by -1.4649164569999584s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
W20260709 13:00:47.657120  4369 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.728s	user 0.025s	sys 0.039s
I20260709 13:00:47.681802  1609 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:47.593947  4241 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:47.697790  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 440) took 2623 ms. Trace:
I20260709 13:00:47.674369  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:47.676118  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.618788  4330 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:47.697944  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 464) took 1052 ms. Trace:
I20260709 13:00:47.703908  1609 rpcz_store.cc:276] 0709 13:00:46.645930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.636033 (+990103us) service_pool.cc:224] Handling call
0709 13:00:47.697880 (+ 61847us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41216}
I20260709 13:00:47.672881  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.698575  1616 rpcz_store.cc:276] 0709 13:00:45.074039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.673789 (+2599750us) service_pool.cc:224] Handling call
0709 13:00:47.697694 (+ 23905us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.586371  4338 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:47.699640  1509 connection.cc:256] RPC call timeout handler was delayed by -1.3839164569999411s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1605s for a response
I20260709 13:00:47.700644  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.717803  1611 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:47.718735  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 435) took 2677 ms. Trace:
I20260709 13:00:47.719215  1611 rpcz_store.cc:276] 0709 13:00:45.041366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.717067 (+2675701us) service_pool.cc:224] Handling call
0709 13:00:47.717545 (+   478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.718375 (+   830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.718658 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.640499  4386 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:47.707269  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 466) took 1060 ms. Trace:
I20260709 13:00:47.720925  1612 rpcz_store.cc:276] 0709 13:00:46.646441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.644364 (+997923us) service_pool.cc:224] Handling call
0709 13:00:47.707180 (+ 62816us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.707945  1582 leader_election.cc:400] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6.
I20260709 13:00:47.721998  1611 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: [3.4-5.5]
I20260709 13:00:47.722096  1582 leader_election.cc:403] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:00:47.723219  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 436) took 2672 ms. Trace:
I20260709 13:00:47.723757  1611 rpcz_store.cc:276] 0709 13:00:45.050520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.720250 (+2669730us) service_pool.cc:224] Handling call
0709 13:00:47.721664 (+  1414us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:47.722823 (+  1159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.723146 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.725622  1611 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:00:47.701287  4369 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.772s	user 0.026s	sys 0.040s
W20260709 13:00:47.714663  2232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 892) took 1521 ms (client timeout 1634 ms). Trace:
W20260709 13:00:47.727042  2232 rpcz_store.cc:269] 0709 13:00:46.192832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.193658 (+   826us) service_pool.cc:224] Handling call
0709 13:00:47.335788 (+1142130us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7eee 0000000000375939 00007fe094a3b1e0 00007fe094b040c4 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:47.701818 (+366030us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5992320}
I20260709 13:00:47.727563  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:47.728863  4330 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:47.732673  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.734714  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:47.737236  1509 connection.cc:256] RPC call timeout handler was delayed by -1.3425653859999898s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33540000000000003s for a response
I20260709 13:00:47.738613  1616 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:00:47.739555  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1338) took 2319 ms. Trace:
I20260709 13:00:47.739948  1616 rpcz_store.cc:276] 0709 13:00:45.420333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.736089 (+2315756us) service_pool.cc:224] Handling call
0709 13:00:47.736594 (+   505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:47.739138 (+  2544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.739485 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.741932  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1339) took 2319 ms. Trace:
I20260709 13:00:47.742359  1616 rpcz_store.cc:276] 0709 13:00:45.422907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.740887 (+2317980us) service_pool.cc:224] Handling call
0709 13:00:47.741259 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.741513 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.741550 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:47.741607 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:47.741659 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.741728 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.741868 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.681694  4352 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:47.631963  4381 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:47.752362  1612 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:00:47.752285  1542 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:47.642005  3984 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:47.754055  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 441) took 2675 ms. Trace:
I20260709 13:00:47.756812  1612 rpcz_store.cc:276] 0709 13:00:45.078701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.721900 (+2643199us) service_pool.cc:224] Handling call
0709 13:00:47.752014 (+ 30114us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.753708 (+  1694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.753968 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.759416  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:47.761835  1509 connection.cc:256] RPC call timeout handler was delayed by -1.1130164569999579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
I20260709 13:00:47.635756  4396 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:47.765251  1509 connection.cc:256] RPC call timeout handler was delayed by -1.001416456999948s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20299999999999999s for a response
W20260709 13:00:47.770191  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.772753  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 449) took 2437 ms. Trace:
W20260709 13:00:47.775851  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 440704us
I20260709 13:00:47.775935  1611 rpcz_store.cc:276] 0709 13:00:45.335838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.724810 (+2388972us) service_pool.cc:224] Handling call
0709 13:00:47.725273 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:47.751739 (+ 26466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.772676 (+ 20937us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:47.773921  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.778602  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:47.793715  1543 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:47.797895  1612 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:00:47.799329  1537 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:47.802942  1683 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.007s	sys 0.011s
W20260709 13:00:47.805089  1683 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.007s	sys 0.011s
I20260709 13:00:47.807763  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2182) took 1838 ms. Trace:
I20260709 13:00:47.819880  1683 rpcz_store.cc:276] 0709 13:00:45.969579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.982596 (+ 13017us) service_pool.cc:224] Handling call
0709 13:00:45.982922 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.983138 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:46.018519 (+ 35381us) log.cc:844] Serialized 14 byte log entry
0709 13:00:47.264704 (+1246185us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b18003f42a0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a 00007fe08faf6a2a
0709 13:00:47.807293 (+542589us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:47.807368 (+    75us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:47.807435 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.807531 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:47.807687 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":28643,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":7660,"raft.queue_time_us":2890,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"spinlock_wait_cycles":2807552,"thread_start_us":542263,"threads_started":1}
I20260709 13:00:47.813601  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:47.822846  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:47.814548  4103 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:47.824222  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 45743.8us
I20260709 13:00:47.824489  1683 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:47.826844  2229 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:47.698519  4197 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:47.829559  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.830606  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:47.819124  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:47.808857  1537 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:00:47.821151  1617 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:47.813854  1609 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:00:47.831761  1509 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.515s (SENT)
I20260709 13:00:47.836227  4367 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:47.837854  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:47.823696  4368 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:47.838519  1539 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:47.840508  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1348) took 2271 ms. Trace:
I20260709 13:00:47.840962  1617 rpcz_store.cc:276] 0709 13:00:45.568598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.660518 (+2091920us) service_pool.cc:224] Handling call
0709 13:00:47.839770 (+179252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:47.841068  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:47.842218  1615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1855) took 2813 ms (client timeout 1798 ms). Trace:
W20260709 13:00:47.844135  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 99329.6us
I20260709 13:00:47.859742  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:47.749246  4386 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:47.954010  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:47.842697  1615 rpcz_store.cc:269] 0709 13:00:45.028441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.028588 (+   147us) service_pool.cc:224] Handling call
0709 13:00:47.840845 (+2812257us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:47.842142 (+  1297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11628800}
I20260709 13:00:47.925757  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:47.883594  1584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.004s	sys 0.030s
I20260709 13:00:47.949038  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:47.927714  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 147080us
W20260709 13:00:47.851136  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:47.918918  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:47.958284  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:47.958441  2237 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:47.950366  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:47.810895  4383 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:47.907932  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:47.997792  1543 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:47.995110  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:48.000288  1584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.008s	sys 0.030s
I20260709 13:00:47.975752  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:47.999202  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 48900.6us
W20260709 13:00:47.995772  1507 connection.cc:256] RPC call timeout handler was delayed by -1.5866355320000025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2662s for a response
I20260709 13:00:47.819667  4347 consensus_queue.cc:1243] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:00:47.995206  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:48.015949  2236 raft_consensus.cc:2410] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:00:47.995303  1539 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:48.017998  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:48.019505  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:48.020298  1539 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:48.027819  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:47.903177  4398 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 13:00:47.908195  4329 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 2.589s	user 0.013s	sys 0.008s
I20260709 13:00:47.832579  4403 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:48.034982  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:47.840246  4405 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:47.902451  4330 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:47.965723  4207 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:47.926195  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:47.845992  4396 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:47.902784  3984 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:47.949553  4410 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:48.055943  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:48.067564  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 297287us
I20260709 13:00:47.997090  4402 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:47.912672  4368 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:48.157401  1583 leader_election.cc:336] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.559s (SENT)
I20260709 13:00:48.108130  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:47.986732  4103 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6.
I20260709 13:00:47.870649  4413 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.275s
W20260709 13:00:48.039825  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 61929.4us
I20260709 13:00:47.831431  2229 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:48.132730  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:48.101089  1543 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:48.069382  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:48.069612  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.146301  2237 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
W20260709 13:00:48.028584  4401 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:00:48.038893  1539 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:00:48.158569  2231 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:48.133608  4338 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:48.029678  4256 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:48.169651  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:48.169283  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:48.180821  2237 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:48.181056  1543 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
W20260709 13:00:48.172708  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.172876  1615 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:48.173714  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:48.183202  1617 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:48.180231  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:48.187045  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:48.188017  2231 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:48.174544  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:48.175038  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:48.189792  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:48.082824  4347 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.191538  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:48.191521  2231 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:00:48.083070  4407 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.195115  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:48.032399  4383 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:48.206269  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:48.057319  4403 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:48.207229  2231 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:48.208402  2231 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:48.179651  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:48.181819  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:48.190326  1539 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:48.190805  1617 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:48.209553  2237 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:48.193104  2230 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:48.040212  4367 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.133s
I20260709 13:00:48.194123  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:48.178148  2236 raft_consensus.cc:2410] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:00:48.160369  3984 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:00:48.198195  4103 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.198771  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:48.199997  4198 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:48.200578  4418 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
I20260709 13:00:48.202000  4416 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:00:48.194880  4402 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:48.205762  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2181) took 2251 ms. Trace:
I20260709 13:00:48.238081  1687 rpcz_store.cc:276] 0709 13:00:45.954174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.975138 (+ 20964us) service_pool.cc:224] Handling call
0709 13:00:45.975482 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:46.456651 (+481169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:46.456698 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:47.280972 (+824274us) log.cc:844] Serialized 26 byte log entry
0709 13:00:47.672507 (+391535us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:47.674736 (+  2229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.674855 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.205557 (+530702us) raft_consensus.cc:1685] finished
0709 13:00:48.205577 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.205704 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":80079,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":6947,"thread_start_us":852841,"threads_started":2,"wal-append.queue_time_us":559071}
I20260709 13:00:48.240494  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:48.160840  4380 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:48.211561  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.227154  4427 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.213140  2235 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:48.219062  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:48.251966  1617 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:48.218027  1615 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:48.252580  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:48.221084  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 883) took 3498 ms. Trace:
I20260709 13:00:48.226606  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:48.235963  4389 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.696s	user 0.010s	sys 0.001s
I20260709 13:00:48.221973  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:48.098500  4320 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.097s
I20260709 13:00:48.211791  2228 raft_consensus.cc:2410] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:48.231573  4198 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:48.136888  4392 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:48.210755  2231 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:48.210204  4403 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:48.211639  2230 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:48.246397  1508 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:48.159657  4410 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:48.206938  1540 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:48.211234  4407 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:48.255254  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 223707us
I20260709 13:00:48.256004  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:48.282505  1543 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:48.168812  4381 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:48.253525  2234 rpcz_store.cc:276] 0709 13:00:44.722478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.722945 (+   467us) service_pool.cc:224] Handling call
0709 13:00:44.738344 (+ 15399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:44.949014 (+210670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.220812 (+3271798us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b5400648a54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:48.221024 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482435456}
I20260709 13:00:48.259310  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:48.205880  4347 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:48.297406  4386 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:48.260239  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:48.298790  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1337) took 2892 ms. Trace:
I20260709 13:00:48.271469  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:48.241293  1687 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:48.274127  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:48.237538  4413 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.284579  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:48.318303  1540 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:00:48.285809  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:48.288013  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:48.299217  1618 rpcz_store.cc:276] 0709 13:00:45.406363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.728723 (+2322360us) service_pool.cc:224] Handling call
0709 13:00:47.747446 (+ 18723us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:48.151443 (+403997us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.151476 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:48.151981 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 13:00:48.189577 (+ 37596us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:48.189628 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.189717 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.297356 (+107639us) spinlock_profiling.cc:243] Waited 9.12 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:00:48.298503 (+  1147us) raft_consensus.cc:1685] finished
0709 13:00:48.298523 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.298730 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1605,"raft.queue_time_us":96119,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":88423,"spinlock_wait_cycles":20966144,"thread_start_us":440645,"threads_started":2,"wal-append.queue_time_us":51759}
I20260709 13:00:48.319667  1543 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:48.288386  4410 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:48.260308  4427 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:48.321368  4413 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:48.237025  4368 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:48.232081  4103 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:48.291373  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1346) took 2727 ms. Trace:
I20260709 13:00:48.291764  3984 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:48.331389  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:48.294998  4402 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:48.295735  1690 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2178) took 2395 ms (client timeout 1898 ms). Trace:
I20260709 13:00:48.295961  4411 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:00:48.247684  4405 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.053s
I20260709 13:00:48.314675  1539 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:48.299062  4347 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:48.340618  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:48.315670  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:48.343155  1687 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:48.315907  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:48.298897  1615 raft_consensus.cc:2376] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7. Re-sending same reply.
I20260709 13:00:48.322759  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:48.326653  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:48.345587  2234 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:48.327411  1609 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:48.330217  1616 rpcz_store.cc:276] 0709 13:00:45.564225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.743490 (+2179265us) service_pool.cc:224] Handling call
0709 13:00:47.743865 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:48.037658 (+293793us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.037788 (+   130us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:48.103435 (+ 65647us) log.cc:844] Serialized 26 byte log entry
0709 13:00:48.206479 (+103044us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:48.206515 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.206583 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.291153 (+ 84570us) raft_consensus.cc:1685] finished
0709 13:00:48.291171 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.291308 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5120,"raft.queue_time_us":111223,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":17303,"thread_start_us":381959,"threads_started":2,"wal-append.queue_time_us":173882}
W20260709 13:00:48.333284  1690 rpcz_store.cc:269] 0709 13:00:45.900460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.903873 (+  3413us) service_pool.cc:224] Handling call
0709 13:00:48.295508 (+2391635us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b540003fa54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:48.295637 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":808536960}
I20260709 13:00:48.297017  4429 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:00:48.330219  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:48.336748  1683 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:48.347286  1582 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:48.264987  4389 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:48.306347  4403 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:48.654044  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:48.668259  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 321050us
I20260709 13:00:48.426338  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:48.669090  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2369) took 1226 ms. Trace:
I20260709 13:00:48.669464  1690 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:48.669580  1537 rpcz_store.cc:276] 0709 13:00:47.443203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.655673 (+212470us) service_pool.cc:224] Handling call
0709 13:00:48.668891 (+1013218us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54000ee2d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:48.669028 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328133376}
I20260709 13:00:48.514605  4380 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:48.599124  4103 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:48.671401  4103 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:48.671711  1686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2173) took 2858 ms (client timeout 2063 ms). Trace:
I20260709 13:00:48.587575  4407 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:48.672272  1686 rpcz_store.cc:269] 0709 13:00:45.813275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.813575 (+   300us) service_pool.cc:224] Handling call
0709 13:00:48.671480 (+2857905us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b540028f5d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:48.671637 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4118646528}
I20260709 13:00:48.673152  4429 raft_consensus.cc:3055] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:48.671820  4103 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:48.374408  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:48.393261  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.659348  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:48.373989  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:48.661739  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:48.347890  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:48.661248  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2359) took 1340 ms. Trace:
I20260709 13:00:48.328363  2232 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:00:48.601852  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 326672us
W20260709 13:00:48.367633  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 151259us
I20260709 13:00:48.367027  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2345) took 3343 ms. Trace:
W20260709 13:00:48.432868  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:48.682512  1690 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
W20260709 13:00:48.700676  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 454298us
I20260709 13:00:48.972764  2234 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
W20260709 13:00:48.504087  4431 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:00:48.657785  3984 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:48.975520  3984 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:00:48.503342  4408 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
W20260709 13:00:48.592913  4215 consensus_meta.cc:287] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.069s	user 0.009s	sys 0.000s
I20260709 13:00:48.977909  4196 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:48.954015  4416 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.007s	sys 0.009s
I20260709 13:00:48.977973  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2186) took 2914 ms. Trace:
W20260709 13:00:48.979700  4416 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.007s	sys 0.010s
I20260709 13:00:48.980731  4416 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.979809  1688 rpcz_store.cc:276] 0709 13:00:46.063510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.360753 (+1297243us) service_pool.cc:224] Handling call
0709 13:00:47.361074 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:47.361380 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.361406 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:47.361892 (+   486us) log.cc:844] Serialized 26 byte log entry
0709 13:00:47.762118 (+400226us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:47.762178 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.762275 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.306290 (+544015us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:00:48.977689 (+671399us) raft_consensus.cc:1685] finished
0709 13:00:48.977709 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.977918 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101921,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"spinlock_wait_cycles":10964736,"thread_start_us":399802,"threads_started":1,"wal-append.queue_time_us":794942}
I20260709 13:00:48.981465  4416 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:48.982800  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:48.978772  4196 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:48.983776  4416 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:48.984158  1541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1836) took 2809 ms (client timeout 1829 ms). Trace:
I20260709 13:00:48.984442  4416 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:00:48.984560  1541 rpcz_store.cc:269] 0709 13:00:46.174540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.198981 (+ 24441us) service_pool.cc:224] Handling call
0709 13:00:48.983978 (+2784997us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54000eddd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:48.984085 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":515456384}
I20260709 13:00:48.985379  4389 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:48.985715  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:48.986308  1541 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:48.988131  4196 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:48.588191  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:48.729169  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1362) took 1112 ms. Trace:
I20260709 13:00:48.733595  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:48.991384  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:48.797005  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2172) took 2984 ms. Trace:
W20260709 13:00:48.992421  1690 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2195) took 1245 ms (client timeout 1646 ms). Trace:
I20260709 13:00:48.997704  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:49.042841  1690 rpcz_store.cc:269] 0709 13:00:47.746872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.426002 (+679130us) service_pool.cc:224] Handling call
0709 13:00:48.992346 (+566344us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.059291  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:49.068169  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1360) took 1451 ms. Trace:
I20260709 13:00:49.068827  1690 rpcz_store.cc:276] 0709 13:00:47.616445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.058941 (+1442496us) service_pool.cc:224] Handling call
0709 13:00:49.068079 (+  9138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:48.589161  4413 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:49.043757  1689 rpcz_store.cc:276] 0709 13:00:45.812919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.813084 (+   165us) service_pool.cc:224] Handling call
0709 13:00:45.813384 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:45.873825 (+ 60441us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:45.874139 (+   314us) log.cc:844] Serialized 14 byte log entry
0709 13:00:45.984511 (+110372us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:45.984783 (+   272us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:45.984931 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.796636 (+2811705us) spinlock_profiling.cc:243] Waited 60 ms on lock 0x7b54000749d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:48.796722 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.796919 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3561,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":91741,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":206,"spinlock_wait_cycles":138108800,"thread_start_us":109298,"threads_started":1,"wal-append.queue_time_us":320388,"wal-append.run_cpu_time_us":1327,"wal-append.run_wall_time_us":1125241}
I20260709 13:00:49.045377  1691 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:48.373349  4369 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:49.043345  2235 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:48.893354  4197 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:48.904621  1613 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 430) took 3940 ms (client timeout 1841 ms). Trace:
W20260709 13:00:49.048620  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:49.076997  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.049108  1615 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:49.050606  1686 raft_consensus.cc:2368] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
I20260709 13:00:48.875000  1611 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:00:49.052088  1508 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 3.695s (SENT) [suppressed 62 similar messages]
I20260709 13:00:49.052474  1683 rpcz_store.cc:276] 0709 13:00:47.616999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.331086 (+714087us) service_pool.cc:224] Handling call
0709 13:00:48.729102 (+398016us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:49.045835  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 54736.2us
W20260709 13:00:48.685352  4173 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.822s	user 0.018s	sys 0.014s
I20260709 13:00:49.055275  1541 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:00:49.049901  1687 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
W20260709 13:00:48.994846  1507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.012s	sys 0.024s
I20260709 13:00:48.992821  1534 rpcz_store.cc:276] 0709 13:00:45.023409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.023594 (+   185us) service_pool.cc:224] Handling call
0709 13:00:45.056840 (+ 33246us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:45.311199 (+254359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:45.311226 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:45.341364 (+ 30138us) log.cc:844] Serialized 26 byte log entry
0709 13:00:45.756868 (+415504us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:47.903405 (+2146537us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b180042fe20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fc455 00007fe0934e43c7 00007fe08faf69a7 00007fe08fb060bf 00007fe08fb06038
0709 13:00:48.031764 (+128359us) log.cc:844] Serialized 14 byte log entry
0709 13:00:48.366504 (+334740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.366613 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:48.366669 (+    56us) raft_consensus.cc:1685] finished
0709 13:00:48.366763 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.366947 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5321,"prepare.queue_time_us":311583,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":692667,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":70120,"spinlock_wait_cycles":10690816,"thread_start_us":1066293,"threads_started":4,"wal-append.queue_time_us":386402,"wal-append.run_cpu_time_us":3343,"wal-append.run_wall_time_us":1390515}
I20260709 13:00:48.502605  4427 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:49.084961  1687 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1363) took 1467 ms (client timeout 1770 ms). Trace:
W20260709 13:00:49.085745  1687 rpcz_store.cc:269] 0709 13:00:47.617292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.374174 (+756882us) service_pool.cc:224] Handling call
0709 13:00:49.084902 (+710728us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:49.086613  1686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2196) took 1338 ms (client timeout 1760 ms). Trace:
W20260709 13:00:49.088183  1686 rpcz_store.cc:269] 0709 13:00:47.748771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.731925 (+983154us) service_pool.cc:224] Handling call
0709 13:00:49.086555 (+354630us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:48.472354  4410 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:48.690083  4429 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:48.426023  4347 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.042235  4103 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:48.966017  4198 consensus_meta.cc:287] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.681s	user 0.008s	sys 0.001s
W20260709 13:00:49.117842  1535 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.010s	sys 0.030s
I20260709 13:00:49.084271  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 534) took 4643 ms. Trace:
W20260709 13:00:49.084574  1507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.013s	sys 0.025s
W20260709 13:00:49.118412  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 831063us
I20260709 13:00:49.118258  1692 rpcz_store.cc:276] 0709 13:00:44.441132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.983438 (+542306us) service_pool.cc:224] Handling call
0709 13:00:44.983739 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:47.286085 (+2302346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.084032 (+1797947us) spinlock_profiling.cc:243] Waited 2.3 s on lock 0x7b540038aed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:49.084210 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5292925696}
I20260709 13:00:48.595050  2229 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:49.069921  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:49.086777  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:49.134204  1687 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:49.135440  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:49.136168  1687 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:49.138902  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.087496  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:49.044118  1543 rpcz_store.cc:276] 0709 13:00:47.320328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.320404 (+1000076us) service_pool.cc:224] Handling call
0709 13:00:48.660563 (+340159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.660806 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:48.660844 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:48.660901 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:48.660953 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:48.661019 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:48.661165 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.069340  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:49.075519  1613 rpcz_store.cc:269] 0709 13:00:44.963765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.964385 (+   620us) service_pool.cc:224] Handling call
0709 13:00:48.874426 (+3910041us) spinlock_profiling.cc:243] Waited 917 ms on lock 0x7b5400104d54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:48.904488 (+ 30062us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2109018880}
I20260709 13:00:49.081949  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:49.144388  1683 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:49.092927  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:49.097843  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:49.083240  4173 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.256s	user 0.024s	sys 0.029s
W20260709 13:00:49.106261  4388 log.cc:927] Time spent T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 2.459s	user 0.000s	sys 0.025s
I20260709 13:00:49.146756  1538 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:49.109654  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 539) took 1748 ms. Trace:
W20260709 13:00:49.114030  1610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 442) took 4017 ms (client timeout 1558 ms). Trace:
I20260709 13:00:49.083755  4369 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:00:49.147848  1610 rpcz_store.cc:269] 0709 13:00:45.097055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.134671 (+ 37616us) service_pool.cc:224] Handling call
0709 13:00:49.113795 (+3979124us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b54003219d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:49.113952 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1815097216}
I20260709 13:00:49.151027  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:49.151675  1610 raft_consensus.cc:2376] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6. Re-sending same reply.
I20260709 13:00:49.084998  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 722) took 1098 ms. Trace:
I20260709 13:00:49.090951  4403 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:49.119711  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:49.161875  1692 raft_consensus.cc:2410] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:00:49.120637  1535 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.010s	sys 0.031s
I20260709 13:00:49.073298  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:49.122956  1584 connection.cc:256] RPC call timeout handler was delayed by -1.9665269879999983s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
W20260709 13:00:49.163920  1584 connection.cc:256] RPC call timeout handler was delayed by -1.8032453549999445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
W20260709 13:00:49.164285  1584 connection.cc:256] RPC call timeout handler was delayed by -1.7438453549999622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
W20260709 13:00:49.164579  1584 connection.cc:256] RPC call timeout handler was delayed by -1.6687074929999994s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21080000000000002s for a response
W20260709 13:00:49.164951  1584 connection.cc:256] RPC call timeout handler was delayed by -1.667956452999988s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
W20260709 13:00:49.165344  1584 connection.cc:256] RPC call timeout handler was delayed by -1.030784696000012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
W20260709 13:00:49.165706  1584 connection.cc:256] RPC call timeout handler was delayed by -1.0300362699999823s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2785s for a response
I20260709 13:00:49.089509  4413 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:00:49.151811  1616 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:49.152084  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:49.152591  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:49.092794  2228 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:49.184775  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:49.185678  1507 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:49.152956  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:49.159482  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:49.183416  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:49.198679  1690 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:49.199201  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2201) took 1141 ms. Trace:
I20260709 13:00:49.199604  1690 rpcz_store.cc:276] 0709 13:00:48.057630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.159151 (+1101521us) service_pool.cc:224] Handling call
0709 13:00:49.199145 (+ 39994us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.201316  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:49.147773  1534 rpcz_store.cc:276] 0709 13:00:47.360751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.089733 (+1728982us) service_pool.cc:224] Handling call
0709 13:00:49.108594 (+ 18861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.108835 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:49.108874 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:49.109145 (+   271us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:49.109225 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.109330 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.109557 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.168905  4081 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:49.170409  4148 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:49.156224  4369 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:49.219457  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 53165.6us
I20260709 13:00:49.220594  1584 leader_election.cc:400] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7.
I20260709 13:00:49.220980  1584 leader_election.cc:403] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:49.182235  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:49.183285  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:49.175028  1689 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:00:49.204264  2237 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.018s	sys 0.008s
W20260709 13:00:49.223664  2237 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.018s	sys 0.008s
I20260709 13:00:49.204701  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:49.150576  4375 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:49.096120  4438 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:00:49.216653  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:49.237591  1692 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:49.225364  1690 raft_consensus.cc:2410] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:49.226329  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1361) took 1609 ms. Trace:
I20260709 13:00:49.239735  1689 rpcz_store.cc:276] 0709 13:00:47.616716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.073000 (+1456284us) service_pool.cc:224] Handling call
0709 13:00:49.226241 (+153241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.241151  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:49.242018  1689 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:49.242668  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2202) took 1075 ms. Trace:
I20260709 13:00:49.243093  1689 rpcz_store.cc:276] 0709 13:00:48.167784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.240767 (+1072983us) service_pool.cc:224] Handling call
0709 13:00:49.242593 (+  1826us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.244411  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:49.156561  4342 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:49.163493  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 574) took 1331 ms. Trace:
I20260709 13:00:49.246567  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 550) took 1020 ms. Trace:
I20260709 13:00:49.247269  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 726) took 1171 ms. Trace:
I20260709 13:00:49.247392  1689 rpcz_store.cc:276] 0709 13:00:48.226685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.244049 (+1017364us) service_pool.cc:224] Handling call
0709 13:00:49.246503 (+  2454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.215554  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:49.249752  1686 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:49.225487  1615 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:49.250656  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1373) took 1010 ms. Trace:
I20260709 13:00:49.251016  1615 rpcz_store.cc:276] 0709 13:00:48.240576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.183064 (+942488us) service_pool.cc:224] Handling call
0709 13:00:49.250593 (+ 67529us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4480}
I20260709 13:00:49.118909  4347 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:49.213318  4411 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:49.150307  4198 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:49.253731  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2205) took 1031 ms. Trace:
I20260709 13:00:49.211364  4081 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:49.263855  1690 rpcz_store.cc:276] 0709 13:00:48.222343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.204410 (+982067us) service_pool.cc:224] Handling call
0709 13:00:49.253654 (+ 49244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.116928  4410 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:49.247534  4403 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7.
I20260709 13:00:49.164614  4440 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
I20260709 13:00:49.265964  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:49.273752  1689 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:49.274490  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2198) took 1521 ms. Trace:
I20260709 13:00:49.275189  1689 rpcz_store.cc:276] 0709 13:00:47.753143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.249469 (+1496326us) service_pool.cc:224] Handling call
0709 13:00:49.274408 (+ 24939us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.247816  2237 rpcz_store.cc:276] 0709 13:00:48.075501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.210065 (+134564us) service_pool.cc:224] Handling call
0709 13:00:48.229333 (+ 19268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.229571 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:48.372985 (+143414us) log.cc:844] Serialized 14 byte log entry
0709 13:00:49.246097 (+873112us) log.cc:844] Serialized 14 byte log entry
0709 13:00:49.246607 (+   510us) log.cc:844] Serialized 14 byte log entry
0709 13:00:49.246887 (+   280us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:49.246935 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:49.246993 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.247058 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.247207 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8144,"prepare.queue_time_us":73793,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":65,"raft.queue_time_us":6270,"raft.run_wall_time_us":4107,"spinlock_wait_cycles":32269056,"thread_start_us":656951,"threads_started":2}
I20260709 13:00:49.276094  1691 raft_consensus.cc:2410] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:49.281363  4130 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:49.282030  4352 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:49.282639  4352 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:49.291149  4349 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:49.294478  4320 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:00:49.295089  4320 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:49.296128  4320 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:49.300500  4417 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.053s	user 0.015s	sys 0.010s
I20260709 13:00:49.303743  4235 raft_consensus.cc:3055] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:49.304381  4235 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:49.306207  4235 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:49.307744  4235 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:49.311187  4435 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 13:00:49.248265  4173 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:00:49.165544  3984 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:49.266422  4437 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:49.260054  4411 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:00:49.265031  4420 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.033s	sys 0.026s
I20260709 13:00:49.318572  4320 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:00:49.279995  4439 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:00:49.278720  4338 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.018s	user 0.019s	sys 0.038s
I20260709 13:00:49.249823  4429 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:49.462544  4420 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.116s	user 0.033s	sys 0.026s
W20260709 13:00:49.484057  4442 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 13:00:49.474618  4439 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:49.491801  4443 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:00:49.498623  4439 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:49.439105  4411 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:49.502947  4411 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:49.506069  4445 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:00:49.517059  4369 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:49.523959  4369 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:49.616735  1509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.000s	sys 0.026s
W20260709 13:00:49.816351  1509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.000s	sys 0.027s
W20260709 13:00:49.816860  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 839588us
I20260709 13:00:49.203745  1534 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:00:49.617184  1537 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.013s	sys 0.011s
W20260709 13:00:49.820528  1537 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.940s	user 0.014s	sys 0.011s
I20260709 13:00:49.821036  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2368) took 2386 ms. Trace:
W20260709 13:00:49.617966  1540 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.824s	user 0.000s	sys 0.028s
I20260709 13:00:49.618620  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2357) took 2298 ms. Trace:
W20260709 13:00:49.821852  1540 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.000s	sys 0.029s
W20260709 13:00:49.619109  1583 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.000s	sys 0.018s
I20260709 13:00:49.618652  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 538) took 3325 ms. Trace:
W20260709 13:00:49.620337  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:49.620260  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 397436us
I20260709 13:00:49.622180  1538 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:49.179615  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 540) took 1818 ms. Trace:
I20260709 13:00:49.622835  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:49.840353  1584 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:49.841089  1612 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:00:49.841743  1612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1874) took 1514 ms (client timeout 1871 ms). Trace:
W20260709 13:00:49.842206  1612 rpcz_store.cc:269] 0709 13:00:48.327165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.167699 (+840534us) service_pool.cc:224] Handling call
0709 13:00:49.841629 (+673930us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.843880  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:49.845043  1612 raft_consensus.cc:2410] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:49.252017  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:49.626292  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:49.628150  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 426889us
I20260709 13:00:49.629541  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:49.248701  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:49.638196  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.662439  1688 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:00:49.663668  1691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2200) took 1888 ms (client timeout 1831 ms). Trace:
W20260709 13:00:49.667316  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.160589  2232 rpcz_store.cc:276] 0709 13:00:47.977861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.211528 (+233667us) service_pool.cc:224] Handling call
0709 13:00:48.263534 (+ 52006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.043874 (+780340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.076403 (+ 32529us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.853893  4420 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:49.854805  4420 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:49.318679  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 735) took 1240 ms. Trace:
I20260709 13:00:49.246944  2229 rpcz_store.cc:276] 0709 13:00:47.828719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.829312 (+   593us) service_pool.cc:224] Handling call
0709 13:00:49.160263 (+1330951us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.856305  2236 rpcz_store.cc:276] 0709 13:00:48.079790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.704509 (+624719us) service_pool.cc:224] Handling call
0709 13:00:48.704894 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.705156 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:48.764688 (+ 59532us) log.cc:844] Serialized 14 byte log entry
0709 13:00:49.318309 (+553621us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:49.318363 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:49.318414 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.318493 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.318622 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":58956,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":198,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":12949,"thread_start_us":322882,"threads_started":1,"wal-append.queue_time_us":674511}
I20260709 13:00:49.858091  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:49.859364  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:49.686131  4081 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:49.861573  4081 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.861752  4427 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:49.282068  4130 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:00:49.480120  4338 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.221s	user 0.020s	sys 0.039s
I20260709 13:00:49.524561  4429 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:49.872784  1688 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 548) took 1662 ms (client timeout 1505 ms). Trace:
I20260709 13:00:49.874047  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1878) took 1324 ms. Trace:
I20260709 13:00:49.874439  1615 rpcz_store.cc:276] 0709 13:00:48.549949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.251774 (+701825us) service_pool.cc:224] Handling call
0709 13:00:49.873864 (+622090us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.875586  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:49.876417  1615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1374) took 1635 ms (client timeout 2170 ms). Trace:
W20260709 13:00:49.876808  1615 rpcz_store.cc:269] 0709 13:00:48.240865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.875252 (+1634387us) service_pool.cc:224] Handling call
0709 13:00:49.876336 (+  1084us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.535274  4432 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:49.526679  4410 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:49.722555  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.878104  4432 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:00:49.716811  1683 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.572s	user 0.006s	sys 0.009s
I20260709 13:00:49.700835  4173 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:49.661134  1686 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 3.
I20260709 13:00:49.720551  4198 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:49.707266  4347 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:49.802832  1507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.014s	sys 0.033s
W20260709 13:00:49.846462  1691 rpcz_store.cc:269] 0709 13:00:47.774760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.730240 (+955480us) service_pool.cc:224] Handling call
0709 13:00:49.663347 (+933107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.793284  4194 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:49.803668  1539 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.012s	sys 0.020s
I20260709 13:00:49.821511  1534 rpcz_store.cc:276] 0709 13:00:47.434856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.202786 (+1767930us) service_pool.cc:224] Handling call
0709 13:00:49.203319 (+   533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.820462 (+617143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.820848 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:49.838979  1583 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.000s	sys 0.019s
W20260709 13:00:49.839211  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.831696  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 542) took 2469 ms. Trace:
W20260709 13:00:49.852480  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.839783  1543 rpcz_store.cc:276] 0709 13:00:47.361091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.141844 (+1780753us) service_pool.cc:224] Handling call
0709 13:00:49.178870 (+ 37026us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.179150 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:49.179188 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:49.179246 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:49.179300 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.179373 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.179528 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.583858  4439 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:49.821905  1535 rpcz_store.cc:276] 0709 13:00:47.319988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.305957 (+985969us) service_pool.cc:224] Handling call
0709 13:00:48.312095 (+  6138us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:48.329925 (+ 17830us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:48.334836 (+  4911us) log.cc:844] Serialized 14 byte log entry
0709 13:00:49.166889 (+832053us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:49.215851 (+ 48962us) log.cc:844] Serialized 26 byte log entry
0709 13:00:49.216094 (+   243us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:49.216146 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.216246 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:49.617584 (+401338us) raft_consensus.cc:1685] finished
0709 13:00:49.617682 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.618214 (+   532us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":391797,"prepare.queue_time_us":5348,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":748894,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":6820,"thread_start_us":831633,"threads_started":1,"wal-append.queue_time_us":967389}
I20260709 13:00:49.910805  4432 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:49.912258  4432 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:49.912952  1535 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 13:00:49.913870  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1857) took 1489 ms. Trace:
I20260709 13:00:49.823467  1536 rpcz_store.cc:276] 0709 13:00:46.293262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.290244 (+996982us) service_pool.cc:224] Handling call
0709 13:00:47.290723 (+   479us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:47.682914 (+392191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.682957 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:47.683725 (+   768us) log.cc:844] Serialized 52 byte log entry
0709 13:00:47.959388 (+275663us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:47.959438 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:47.959500 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:49.299722 (+1340222us) spinlock_profiling.cc:243] Waited 4.21 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:00:49.617703 (+317981us) raft_consensus.cc:1685] finished
0709 13:00:49.617755 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.618405 (+   650us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1545,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":119,"spinlock_wait_cycles":9683712,"thread_start_us":615083,"threads_started":2,"wal-append.queue_time_us":466722}
I20260709 13:00:49.881340  4173 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:49.914353  1535 rpcz_store.cc:276] 0709 13:00:48.424730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.911850 (+1487120us) service_pool.cc:224] Handling call
0709 13:00:49.912492 (+   642us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:49.913492 (+  1000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.913772 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.914996  4173 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:49.879236  1616 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:49.796300  1541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.004s	sys 0.029s
W20260709 13:00:49.919353  1541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.829s	user 0.005s	sys 0.030s
W20260709 13:00:49.898881  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 842722us
I20260709 13:00:49.920509  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2364) took 2567 ms. Trace:
I20260709 13:00:49.921084  1541 rpcz_store.cc:276] 0709 13:00:47.352844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.089345 (+1736501us) service_pool.cc:224] Handling call
0709 13:00:49.089701 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:49.090106 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:49.090134 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:49.090592 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:00:49.919999 (+829407us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:49.920066 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.920158 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:49.920217 (+    59us) raft_consensus.cc:1685] finished
0709 13:00:49.920233 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.920429 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17477,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":129,"thread_start_us":828942,"threads_started":1,"wal-append.queue_time_us":684221}
I20260709 13:00:49.880336  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1867) took 1613 ms. Trace:
I20260709 13:00:49.803403  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2187) took 3727 ms. Trace:
I20260709 13:00:49.926658  1684 rpcz_store.cc:276] 0709 13:00:46.063807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.611045 (+1547238us) service_pool.cc:224] Handling call
0709 13:00:47.640933 (+ 29888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:47.641445 (+   512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:47.641479 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:49.233748 (+1592269us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b2c00144828. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:00:49.252059 (+ 18311us) log.cc:844] Serialized 26 byte log entry
0709 13:00:49.692147 (+440088us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:49.692272 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.692483 (+   211us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:49.740593 (+ 48110us) raft_consensus.cc:1685] finished
0709 13:00:49.740612 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.791350 (+ 50738us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9379,"raft.queue_time_us":11907,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":19306,"spinlock_wait_cycles":8635776,"thread_start_us":54316,"threads_started":1,"wal-append.queue_time_us":54976}
W20260709 13:00:49.928351  1687 consensus_meta.cc:287] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.792s	user 0.008s	sys 0.006s
I20260709 13:00:49.928862  1687 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:49.884081  1582 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:49.884439  1538 rpcz_store.cc:276] 0709 13:00:47.362249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.180414 (+1818165us) service_pool.cc:224] Handling call
0709 13:00:49.621599 (+441185us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.829206 (+207607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.831605 (+  2399us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.929692  1612 rpcz_store.cc:276] 0709 13:00:48.236560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.843459 (+1606899us) service_pool.cc:224] Handling call
0709 13:00:49.849972 (+  6513us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.934968  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2193) took 2188 ms. Trace:
W20260709 13:00:49.873275  1688 rpcz_store.cc:269] 0709 13:00:48.209844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.982481 (+772637us) service_pool.cc:224] Handling call
0709 13:00:49.872512 (+890031us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b540028d7d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:49.872686 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1559611648}
W20260709 13:00:49.895087  1507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.689s	user 0.015s	sys 0.034s
W20260709 13:00:49.938802  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 753121us
W20260709 13:00:49.898356  1539 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.013s	sys 0.021s
W20260709 13:00:49.902645  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:49.903595  1686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 545) took 1728 ms (client timeout 2176 ms). Trace:
I20260709 13:00:49.905974  3942 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:49.907732  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:49.908254  1613 raft_consensus.cc:2368] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6.
I20260709 13:00:49.909612  1534 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:49.897142  1683 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:49.935614  1684 rpcz_store.cc:276] 0709 13:00:47.746307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.928912 (+2182605us) service_pool.cc:224] Handling call
0709 13:00:49.929345 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.934406 (+  5061us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:49.934436 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:49.934482 (+    46us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:49.934676 (+   194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.934738 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:49.934867 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:49.940486  1686 rpcz_store.cc:269] 0709 13:00:48.175163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.215209 (+1040046us) service_pool.cc:224] Handling call
0709 13:00:49.896935 (+681726us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:49.941238  1687 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 543) took 1773 ms (client timeout 2073 ms). Trace:
I20260709 13:00:49.953101  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1846) took 2382 ms. Trace:
I20260709 13:00:49.953660  1534 rpcz_store.cc:276] 0709 13:00:47.558579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.908737 (+2350158us) service_pool.cc:224] Handling call
0709 13:00:49.909116 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.940983 (+ 31867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:49.941245 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.933531  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:49.897289  1508 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.001s	sys 0.004s
I20260709 13:00:49.956614  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:49.957782  1615 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:00:49.958360  1615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1868) took 1703 ms (client timeout 2220 ms). Trace:
I20260709 13:00:49.958424  1507 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:49.958758  1615 rpcz_store.cc:269] 0709 13:00:48.254571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.880130 (+1625559us) service_pool.cc:224] Handling call
0709 13:00:49.958252 (+ 78122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.959995  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:49.960955  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1879) took 1232 ms. Trace:
W20260709 13:00:49.961531  1508 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.001s	sys 0.005s
W20260709 13:00:49.962327  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 910199us
W20260709 13:00:49.966979  1613 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1369) took 1727 ms (client timeout 2037 ms). Trace:
W20260709 13:00:49.967499  1613 rpcz_store.cc:269] 0709 13:00:48.239382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.224944 (+985562us) service_pool.cc:224] Handling call
0709 13:00:49.966844 (+741900us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.968667  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:49.953142  1687 rpcz_store.cc:269] 0709 13:00:48.168671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.135101 (+966430us) service_pool.cc:224] Handling call
0709 13:00:49.929388 (+794287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.970855  1615 rpcz_store.cc:276] 0709 13:00:48.729004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.959577 (+1230573us) service_pool.cc:224] Handling call
0709 13:00:49.960870 (+  1293us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:49.974601  1683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 542) took 1833 ms (client timeout 1745 ms). Trace:
I20260709 13:00:49.987687  1691 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:49.982396  1683 rpcz_store.cc:269] 0709 13:00:48.155876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.081632 (+925756us) service_pool.cc:224] Handling call
0709 13:00:49.952970 (+871338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:49.977866  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1881) took 1005 ms. Trace:
I20260709 13:00:49.989077  1613 rpcz_store.cc:276] 0709 13:00:48.965855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.968297 (+1002442us) service_pool.cc:224] Handling call
0709 13:00:49.971199 (+  2902us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:49.989300  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 130118us
W20260709 13:00:50.007433  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 43937.5us
I20260709 13:00:49.956616  4411 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:50.031101  1616 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:00:50.023597  1612 raft_consensus.cc:2376] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7. Re-sending same reply.
W20260709 13:00:50.034287  1612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1370) took 1794 ms (client timeout 2285 ms). Trace:
W20260709 13:00:50.035939  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 175860us
W20260709 13:00:50.039357  1612 rpcz_store.cc:269] 0709 13:00:48.239676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.932524 (+1692848us) service_pool.cc:224] Handling call
0709 13:00:50.034217 (+101693us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:50.028460  1509 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.580s (SENT)
I20260709 13:00:50.004220  4427 raft_consensus.cc:697] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:50.038381  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2370) took 2541 ms. Trace:
I20260709 13:00:50.049926  1686 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:00:50.050453  1616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1875) took 1711 ms (client timeout 1897 ms). Trace:
I20260709 13:00:50.051996  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.035108  4320 raft_consensus.cc:2764] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
W20260709 13:00:50.053617  1616 rpcz_store.cc:269] 0709 13:00:48.339346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.184265 (+844919us) service_pool.cc:224] Handling call
0709 13:00:50.040595 (+856330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.059015  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2203) took 1837 ms. Trace:
W20260709 13:00:50.058944  1615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 473) took 1822 ms (client timeout 2348 ms). Trace:
W20260709 13:00:50.060117  1615 rpcz_store.cc:269] 0709 13:00:48.236343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.010093 (+1773750us) service_pool.cc:224] Handling call
0709 13:00:50.058856 (+ 48763us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.059988  1686 rpcz_store.cc:276] 0709 13:00:48.216066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.977956 (+1761890us) service_pool.cc:224] Handling call
0709 13:00:49.981799 (+  3843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.053268 (+ 71469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.053568 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.039335  4130 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:50.064754  1684 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:50.068444  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.069028  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 544) took 1899 ms. Trace:
I20260709 13:00:50.069496  1684 rpcz_store.cc:276] 0709 13:00:48.169231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.952772 (+1783541us) service_pool.cc:224] Handling call
0709 13:00:50.008481 (+ 55709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.068747 (+ 60266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.068956 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.072110  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2360) took 2743 ms. Trace:
I20260709 13:00:50.074632  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 549) took 1857 ms. Trace:
I20260709 13:00:50.075141  1687 rpcz_store.cc:276] 0709 13:00:48.217555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.970632 (+1753077us) service_pool.cc:224] Handling call
0709 13:00:50.033398 (+ 62766us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.070601 (+ 37203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.070647 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.074244 (+  3597us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:50.074310 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.074422 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.074563 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.076931  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.073275  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2362) took 2733 ms. Trace:
I20260709 13:00:50.066563  4130 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:50.085181  1583 leader_election.cc:336] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.588s (SENT)
I20260709 13:00:50.085924  1583 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:50.086262  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:50.086822  1613 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1871) took 1797 ms (client timeout 2302 ms). Trace:
I20260709 13:00:50.087260  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1883) took 1040 ms. Trace:
W20260709 13:00:50.087419  1613 rpcz_store.cc:269] 0709 13:00:48.283932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.027399 (+1743467us) service_pool.cc:224] Handling call
0709 13:00:50.080815 (+ 53416us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.087790  1612 rpcz_store.cc:276] 0709 13:00:49.046587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.042670 (+996083us) service_pool.cc:224] Handling call
0709 13:00:50.087190 (+ 44520us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:50.089643  1614 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.806s	user 0.011s	sys 0.004s
I20260709 13:00:50.089797  4450 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:00:50.090306  1614 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
W20260709 13:00:50.091123  1692 consensus_meta.cc:287] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.853s	user 0.004s	sys 0.012s
W20260709 13:00:50.091205  1614 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 451) took 4741 ms (client timeout 2139 ms). Trace:
I20260709 13:00:50.091667  1692 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
W20260709 13:00:50.091909  1614 rpcz_store.cc:269] 0709 13:00:45.349906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:46.256012 (+906106us) service_pool.cc:224] Handling call
0709 13:00:46.647893 (+391881us) spinlock_profiling.cc:243] Waited 892 us on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:49.174485 (+2526592us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7e9c 00007fe08c9c7a5b 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:50.091052 (+916567us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63865344}
W20260709 13:00:50.093055  1692 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2197) took 2344 ms (client timeout 2504 ms). Trace:
W20260709 13:00:50.093515  1692 rpcz_store.cc:269] 0709 13:00:47.749137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.175453 (+1426316us) service_pool.cc:224] Handling call
0709 13:00:50.092970 (+917517us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.093593  4338 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:00:50.092628  1689 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:50.096952  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 552) took 1863 ms. Trace:
I20260709 13:00:50.099107  4439 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:50.073462  4411 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:50.101092  4411 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:50.101838  4352 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.819s	user 0.012s	sys 0.004s
I20260709 13:00:50.097362  1692 rpcz_store.cc:276] 0709 13:00:48.233674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.095639 (+1861965us) service_pool.cc:224] Handling call
0709 13:00:50.096242 (+   603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.096614 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.096639 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.096686 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:00:50.096718 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.096781 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.096879 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.105052  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2361) took 2770 ms. Trace:
I20260709 13:00:50.105523  1540 rpcz_store.cc:276] 0709 13:00:47.334794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.349141 (+1014347us) service_pool.cc:224] Handling call
0709 13:00:48.373772 (+ 24631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.373979 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:49.822430 (+1448451us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.102132 (+279702us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.104226 (+  2094us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.104608 (+   382us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.104669 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:50.104769 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.104868 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.104997 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":366152,"prepare.queue_time_us":507546,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":116,"raft.queue_time_us":117113,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":151,"spinlock_wait_cycles":38690048,"thread_start_us":1178956,"threads_started":3,"wal-append.queue_time_us":204501}
I20260709 13:00:50.089184  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:50.083703  1616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1371) took 1838 ms (client timeout 2436 ms). Trace:
W20260709 13:00:50.112615  1616 rpcz_store.cc:269] 0709 13:00:48.239954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.067996 (+1828042us) service_pool.cc:224] Handling call
0709 13:00:50.078554 (+ 10558us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:50.097322  1507 connection.cc:256] RPC call timeout handler was delayed by -1.0953947699999844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
W20260709 13:00:50.113945  1507 connection.cc:256] RPC call timeout handler was delayed by -1.0737947699999495s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
I20260709 13:00:50.089148  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.118527  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2199) took 2355 ms. Trace:
I20260709 13:00:50.119036  1688 rpcz_store.cc:276] 0709 13:00:47.762804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.938115 (+2175311us) service_pool.cc:224] Handling call
0709 13:00:49.938825 (+   710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:49.952344 (+ 13519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:49.952388 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:49.970075 (+ 17687us) log.cc:844] Serialized 26 byte log entry
0709 13:00:50.055797 (+ 85722us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:00:50.055838 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.055911 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:50.118241 (+ 62330us) raft_consensus.cc:1685] finished
0709 13:00:50.118272 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.118433 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17443,"raft.queue_time_us":10951,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":4593,"thread_start_us":85269,"threads_started":1,"wal-append.queue_time_us":145624}
I20260709 13:00:50.079229  4420 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:50.115243  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:50.125018  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55829.1us
I20260709 13:00:50.123598  4369 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:50.073540  1539 rpcz_store.cc:276] 0709 13:00:47.329076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.341978 (+1012902us) service_pool.cc:224] Handling call
0709 13:00:48.342324 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.342573 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:00:49.129713 (+787140us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.004964 (+875251us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.022887 (+ 17923us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:50.040727 (+ 17840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.049343 (+  8616us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.072060 (+ 22717us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":538445,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":74238,"thread_start_us":1265266,"threads_started":2,"wal-append.queue_time_us":642196}
I20260709 13:00:50.127840  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1381) took 1148 ms. Trace:
I20260709 13:00:50.128410  1614 rpcz_store.cc:276] 0709 13:00:48.974906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.093302 (+1118396us) service_pool.cc:224] Handling call
0709 13:00:50.122931 (+ 29629us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.131330  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2204) took 1912 ms. Trace:
I20260709 13:00:50.131779  1683 rpcz_store.cc:276] 0709 13:00:48.218656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.010288 (+1791632us) service_pool.cc:224] Handling call
0709 13:00:50.043226 (+ 32938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.129702 (+ 86476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.129732 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.131009 (+  1277us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:50.131066 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.131135 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.131263 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.111229  4130 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:50.136554  4235 consensus_meta.cc:287] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.828s	user 0.012s	sys 0.002s
I20260709 13:00:50.082206  1537 rpcz_store.cc:276] 0709 13:00:47.339917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.670590 (+1330673us) service_pool.cc:224] Handling call
0709 13:00:48.670919 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:48.671100 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:49.821081 (+1149981us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.029014 (+207933us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.040229 (+ 11215us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:50.066351 (+ 26122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.068799 (+  2448us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.073211 (+  4412us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123151,"prepare.queue_time_us":210442,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":43835,"raft.run_cpu_time_us":352,"raft.run_wall_time_us":22002,"spinlock_wait_cycles":21105920,"thread_start_us":1124680,"threads_started":2,"wal-append.queue_time_us":236891}
I20260709 13:00:50.138862  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 555) took 1883 ms. Trace:
I20260709 13:00:50.139438  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 551) took 1908 ms. Trace:
I20260709 13:00:50.132069  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1877) took 1671 ms. Trace:
I20260709 13:00:50.139997  1684 rpcz_store.cc:276] 0709 13:00:48.230918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.108527 (+1877609us) service_pool.cc:224] Handling call
0709 13:00:50.138605 (+ 30078us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.138991 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.139025 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.139102 (+    77us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:50.139147 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.139219 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.139344 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:50.142849  4103 consensus_meta.cc:287] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.992s	user 0.016s	sys 0.000s
I20260709 13:00:50.143500  4103 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:50.139276  1683 rpcz_store.cc:276] 0709 13:00:48.255431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.137676 (+1882245us) service_pool.cc:224] Handling call
0709 13:00:50.138169 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.138485 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.138511 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.138568 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:50.138598 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.138655 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.138798 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.145210  1610 raft_consensus.cc:2368] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5.
I20260709 13:00:50.146567  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 557) took 1870 ms. Trace:
I20260709 13:00:50.147028  1683 rpcz_store.cc:276] 0709 13:00:48.276072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.145361 (+1869289us) service_pool.cc:224] Handling call
0709 13:00:50.145819 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.146157 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.146185 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.146230 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:50.146303 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.146373 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.146507 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.148983  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 558) took 1862 ms. Trace:
I20260709 13:00:50.140237  1613 rpcz_store.cc:276] 0709 13:00:48.460556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.088616 (+1628060us) service_pool.cc:224] Handling call
0709 13:00:50.132009 (+ 43393us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:50.160008  4412 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.914s	user 0.028s	sys 0.047s
W20260709 13:00:50.163300  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 143238us
I20260709 13:00:50.165376  4427 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.166980  1684 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:50.167853  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2206) took 1912 ms. Trace:
I20260709 13:00:50.168285  1684 rpcz_store.cc:276] 0709 13:00:48.255488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.166101 (+1910613us) service_pool.cc:224] Handling call
0709 13:00:50.166678 (+   577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.167481 (+   803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.167759 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.169831  4466 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:50.175185  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:50.165150  1509 leader_election.cc:336] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.605s (SENT)
I20260709 13:00:50.126003  4369 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:50.176770  4369 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.177707  4310 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:50.102546  4352 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:50.181358  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.182133  1613 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:50.182646  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1880) took 1224 ms. Trace:
I20260709 13:00:50.183135  4352 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:50.182960  1613 rpcz_store.cc:276] 0709 13:00:48.958248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.154387 (+1196139us) service_pool.cc:224] Handling call
0709 13:00:50.181200 (+ 26813us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:50.182583 (+  1383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46774656}
I20260709 13:00:50.184379  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.185165  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1382) took 1210 ms. Trace:
W20260709 13:00:50.193349  4424 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.947s	user 0.012s	sys 0.027s
W20260709 13:00:50.194574  4349 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.903s	user 0.013s	sys 0.003s
W20260709 13:00:50.198004  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 112907us
I20260709 13:00:50.207357  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.209456  1687 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:00:50.213577  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.214655  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 739) took 1964 ms. Trace:
I20260709 13:00:50.216557  1613 rpcz_store.cc:276] 0709 13:00:48.975240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.184029 (+1208789us) service_pool.cc:224] Handling call
0709 13:00:50.185112 (+  1083us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.149302  1683 rpcz_store.cc:276] 0709 13:00:48.286144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.148015 (+1861871us) service_pool.cc:224] Handling call
0709 13:00:50.148391 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.148660 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.148684 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.148723 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:50.148757 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.148805 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.148911 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.218251  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1375) took 1977 ms. Trace:
I20260709 13:00:50.220322  1616 rpcz_store.cc:276] 0709 13:00:48.241148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.123131 (+1881983us) service_pool.cc:224] Handling call
0709 13:00:50.218185 (+ 95054us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.219388  2233 rpcz_store.cc:276] 0709 13:00:48.249324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.888406 (+639082us) service_pool.cc:224] Handling call
0709 13:00:50.212716 (+1324310us) spinlock_profiling.cc:243] Waited 8.15 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:00:50.213107 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.213541 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.213592 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.213669 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:50.213724 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.213829 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.214006 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18740608}
W20260709 13:00:50.224407  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 182868us
I20260709 13:00:50.217480  1582 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:50.230372  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2208) took 1919 ms. Trace:
I20260709 13:00:50.230832  1687 rpcz_store.cc:276] 0709 13:00:48.300375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.208655 (+1908280us) service_pool.cc:224] Handling call
0709 13:00:50.209059 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:50.219002 (+  9943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.219220 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.210856  4173 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:50.237123  1609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1363) took 2000 ms (client timeout 1665 ms). Trace:
I20260709 13:00:50.237535  1685 raft_consensus.cc:2410] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:00:50.237356  4429 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:50.238147  4413 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.239817  4241 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:50.248427  1609 rpcz_store.cc:269] 0709 13:00:48.236665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.260048 (+ 23383us) service_pool.cc:224] Handling call
0709 13:00:50.217750 (+1957702us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b5400389854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:50.236870 (+ 19120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5648128}
I20260709 13:00:50.240257  4310 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.241880  4434 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.242539  3984 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.243028  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.243325  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:50.244643  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:50.247192  1542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1843) took 2732 ms (client timeout 1672 ms). Trace:
W20260709 13:00:50.247480  1583 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.040s (SENT)
I20260709 13:00:50.258615  1615 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:50.264628  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1376) took 2023 ms. Trace:
I20260709 13:00:50.265143  1615 rpcz_store.cc:276] 0709 13:00:48.241492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.206933 (+1965441us) service_pool.cc:224] Handling call
0709 13:00:50.264543 (+ 57610us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:50.262712  1542 rpcz_store.cc:269] 0709 13:00:47.514238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.620151 (+105913us) service_pool.cc:224] Handling call
0709 13:00:50.218452 (+2598301us) spinlock_profiling.cc:243] Waited 87.6 ms on lock 0x7b54000ad054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:50.246960 (+ 28508us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201495296}
I20260709 13:00:50.268267  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:50.270274  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.271080  1613 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 3.
I20260709 13:00:50.271904  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1882) took 1282 ms. Trace:
I20260709 13:00:50.272550  1616 rpcz_store.cc:276] 0709 13:00:48.989610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.220900 (+1231290us) service_pool.cc:224] Handling call
0709 13:00:50.271850 (+ 50950us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:50.272702  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 107628us
I20260709 13:00:50.264065  4201 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.260622  4103 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:50.275849  1541 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:50.271557  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1886) took 1180 ms. Trace:
I20260709 13:00:50.281575  1613 rpcz_store.cc:276] 0709 13:00:49.099363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.233879 (+1134516us) service_pool.cc:224] Handling call
0709 13:00:50.271502 (+ 37623us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.258229  4432 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:50.284752  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:50.293583  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.296677  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:50.296010  2203 leader_election.cc:336] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.674s (SENT)
I20260709 13:00:50.298975  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2207) took 2001 ms. Trace:
I20260709 13:00:50.296626  1535 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:50.285846  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.299939  1684 rpcz_store.cc:276] 0709 13:00:48.297480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.173795 (+1876315us) service_pool.cc:224] Handling call
0709 13:00:50.174435 (+   640us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:50.186045 (+ 11610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.186090 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:50.186781 (+   691us) log.cc:844] Serialized 26 byte log entry
0709 13:00:50.261988 (+ 75207us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:50.263691 (+  1703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.263784 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:50.298768 (+ 34984us) raft_consensus.cc:1685] finished
0709 13:00:50.298784 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.298919 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12595,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"thread_start_us":74701,"threads_started":1,"wal-append.queue_time_us":106828}
I20260709 13:00:50.302608  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1887) took 1186 ms. Trace:
I20260709 13:00:50.303161  1539 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:50.302989  1616 rpcz_store.cc:276] 0709 13:00:49.116408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.273350 (+1156942us) service_pool.cc:224] Handling call
0709 13:00:50.302551 (+ 29201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.304214  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.301808  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1383) took 1326 ms. Trace:
I20260709 13:00:50.305101  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.305207  1613 rpcz_store.cc:276] 0709 13:00:48.975544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.282257 (+1306713us) service_pool.cc:224] Handling call
0709 13:00:50.301748 (+ 19491us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.294188  4173 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:50.306360  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.306664  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:50.306789  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 59380.7us
I20260709 13:00:50.307111  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1384) took 1331 ms. Trace:
I20260709 13:00:50.307531  1613 rpcz_store.cc:276] 0709 13:00:48.975844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.306053 (+1330209us) service_pool.cc:224] Handling call
0709 13:00:50.307057 (+  1004us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:00:50.308179  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.308537  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:50.308974  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.313330  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:50.298180  4413 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.314987  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:50.316279  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:50.317788  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:50.318733  1609 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:50.319053  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:50.319247  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1379) took 2076 ms. Trace:
I20260709 13:00:50.319679  1609 rpcz_store.cc:276] 0709 13:00:48.242425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.261291 (+2018866us) service_pool.cc:224] Handling call
0709 13:00:50.319182 (+ 57891us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.320057  1536 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:00:50.320823  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:50.322597  1610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1368) took 2083 ms (client timeout 1908 ms). Trace:
W20260709 13:00:50.323076  1610 rpcz_store.cc:269] 0709 13:00:48.239091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.152336 (+913245us) service_pool.cc:224] Handling call
0709 13:00:50.322365 (+1170029us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b54000884d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:50.322527 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2208834944}
I20260709 13:00:50.324257  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:50.324951  1610 raft_consensus.cc:2376] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7. Re-sending same reply.
I20260709 13:00:50.307552  1537 raft_consensus.cc:2393] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 5: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:50.326268  1539 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:50.326443  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1358) took 2710 ms. Trace:
I20260709 13:00:50.326799  1689 rpcz_store.cc:276] 0709 13:00:47.615876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.662896 (+2047020us) service_pool.cc:224] Handling call
0709 13:00:50.092298 (+429402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.095043 (+  2745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.326222 (+231179us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b540041e6d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:50.326363 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":985705344}
I20260709 13:00:50.328127  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 553) took 2090 ms. Trace:
I20260709 13:00:50.328558  1686 rpcz_store.cc:276] 0709 13:00:48.237395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.124591 (+1887196us) service_pool.cc:224] Handling call
0709 13:00:50.179530 (+ 54939us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:50.179929 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.179973 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:50.180543 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:00:50.263284 (+ 82741us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:50.263319 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.263387 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:50.327903 (+ 64516us) raft_consensus.cc:1685] finished
0709 13:00:50.327924 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.328059 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":80331,"threads_started":1,"wal-append.queue_time_us":116109}
I20260709 13:00:50.331707  4413 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:50.334517  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.334197  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1385) took 1358 ms. Trace:
I20260709 13:00:50.335439  1610 rpcz_store.cc:276] 0709 13:00:48.976128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.323906 (+1347778us) service_pool.cc:224] Handling call
0709 13:00:50.334138 (+ 10232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.334230  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:50.337278  1613 raft_consensus.cc:2410] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:50.286973  4369 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:50.292830  4429 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:50.343758  1616 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
W20260709 13:00:50.344430  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44762.4us
I20260709 13:00:50.286612  4201 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.341945  4130 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:50.350528  1509 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.707s (SENT)
I20260709 13:00:50.351051  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:50.351575  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 55777.2us
I20260709 13:00:50.352164  4130 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:50.352439  2203 leader_election.cc:336] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.709s (SENT)
I20260709 13:00:50.353010  4413 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:00:50.353876  2203 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.803s (SENT)
W20260709 13:00:50.355197  2203 leader_election.cc:336] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.866s (SENT)
I20260709 13:00:50.355630  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:50.285115  4310 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.357227  1542 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:00:50.357897  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.358428  4310 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:50.358716  4241 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:50.360106  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:50.360872  1538 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:50.361799  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.362681  1616 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:50.363163  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1387) took 1385 ms. Trace:
I20260709 13:00:50.363507  1616 rpcz_store.cc:276] 0709 13:00:48.977486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.345146 (+1367660us) service_pool.cc:224] Handling call
0709 13:00:50.363101 (+ 17955us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.364532  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:50.358188  4413 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:00:50.366011  4413 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:50.366362  4413 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:50.366973  4148 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:50.367731  4413 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:50.356869  1509 leader_election.cc:336] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.879s (SENT)
I20260709 13:00:50.358835  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 472) took 2122 ms. Trace:
W20260709 13:00:50.373477  1509 leader_election.cc:336] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.908s (SENT)
I20260709 13:00:50.373677  1615 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:50.373919  1509 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:50.375408  4130 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:00:50.375664  1509 leader_election.cc:336] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.993s (SENT)
I20260709 13:00:50.376242  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:50.371706  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:50.377151  1538 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:50.378403  4241 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:50.379101  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.380079  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:50.380736  1542 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:50.386361  1538 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:00:50.386482  4419 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:50.373711  1613 rpcz_store.cc:276] 0709 13:00:48.236031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.346819 (+2110788us) service_pool.cc:224] Handling call
0709 13:00:50.358764 (+ 11945us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.387059  4419 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:50.388195  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.388356  4419 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.315171  4432 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:50.391840  4432 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:50.296208  3984 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:50.392976  1685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2171) took 4587 ms (client timeout 3339 ms). Trace:
I20260709 13:00:50.393159  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.394745  1618 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:50.394994  3984 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:50.313633  4434 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.395473  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1378) took 2153 ms. Trace:
I20260709 13:00:50.395831  1618 rpcz_store.cc:276] 0709 13:00:48.242131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.393935 (+2151804us) service_pool.cc:224] Handling call
0709 13:00:50.394373 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.395208 (+   835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.395395 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.397328  4434 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:50.398377  1690 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:00:50.398761  1690 pending_rounds.cc:85] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 6
I20260709 13:00:50.398988  1690 pending_rounds.cc:107] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:00:50.399233  1690 raft_consensus.cc:2889] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:50.400815  1685 rpcz_store.cc:269] 0709 13:00:45.805356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:45.805530 (+   174us) service_pool.cc:224] Handling call
0709 13:00:50.392701 (+4587171us) spinlock_profiling.cc:243] Waited 4.33 s on lock 0x7b54002ccc54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:50.392891 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9959519104}
I20260709 13:00:50.403669  1612 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:50.404332  1534 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:50.394236  1613 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 4.
W20260709 13:00:50.405139  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.433596  1541 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:50.418426  1609 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:50.419603  1617 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:00:50.419638  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:50.438182  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.439219  1613 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:50.433156  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.403041  4369 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:50.398304  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2191) took 2652 ms. Trace:
I20260709 13:00:50.443045  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.432749  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:50.445178  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:50.408969  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.434321  1542 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:50.435477  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:50.446719  1537 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:50.417958  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.454473  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:50.443286  1691 rpcz_store.cc:276] 0709 13:00:47.745664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.918039 (+2172375us) service_pool.cc:224] Handling call
0709 13:00:49.941547 (+ 23508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.990455 (+ 48908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.397986 (+407531us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b5400058f50. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:50.398198 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52908288}
W20260709 13:00:50.460450  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:50.460989  1614 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1872) took 2144 ms (client timeout 2671 ms). Trace:
W20260709 13:00:50.461413  1614 rpcz_store.cc:269] 0709 13:00:48.309308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.129266 (+1819958us) service_pool.cc:224] Handling call
0709 13:00:50.417794 (+288528us) spinlock_profiling.cc:243] Waited 5.03 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:00:50.451046 (+ 33252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11563392}
I20260709 13:00:50.424567  4201 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:50.403669  4419 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:50.467475  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:50.467796  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 110578us
I20260709 13:00:50.472864  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1386) took 1495 ms. Trace:
I20260709 13:00:50.473340  1610 rpcz_store.cc:276] 0709 13:00:48.977204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.337129 (+1359925us) service_pool.cc:224] Handling call
0709 13:00:50.472798 (+135669us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.473592  1539 raft_consensus.cc:2376] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6. Re-sending same reply.
I20260709 13:00:50.466078  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1885) took 1380 ms. Trace:
I20260709 13:00:50.476032  1609 rpcz_store.cc:276] 0709 13:00:49.084447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.320456 (+1236009us) service_pool.cc:224] Handling call
0709 13:00:50.454860 (+134404us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:50.476928  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:50.410543  4383 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:50.427848  4130 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:00:50.475492  1583 leader_election.cc:336] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.108s (SENT)
I20260709 13:00:50.480357  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:50.487072  1542 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:50.488488  1617 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:50.488534  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:50.494000  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:50.500383  1537 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:50.500819  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 125180us
I20260709 13:00:50.502651  1541 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:50.502837  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:50.503948  1534 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
W20260709 13:00:50.505939  1658 connection.cc:256] RPC call timeout handler was delayed by -1.74174782099999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2671s for a response
I20260709 13:00:50.509092  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:50.510234  4383 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:50.511065  4383 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:50.511514  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:50.506374  1610 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:50.510294  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:50.515105  1534 raft_consensus.cc:2393] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:50.529132  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:50.530002  1541 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:00:50.530969  1658 connection.cc:256] RPC call timeout handler was delayed by -1.5142105490000404s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15500000000000003s for a response
W20260709 13:00:50.531381  1658 connection.cc:256] RPC call timeout handler was delayed by -1.50510932800006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3069s for a response
W20260709 13:00:50.531680  1658 connection.cc:256] RPC call timeout handler was delayed by -1.4044105490000334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 13:00:50.531970  1658 connection.cc:256] RPC call timeout handler was delayed by -1.301810549000038s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
W20260709 13:00:50.532240  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2946105490000264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 13:00:50.532528  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2946105490000264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 13:00:50.532804  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2910105490000205s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
W20260709 13:00:50.533097  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2721105490000468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
W20260709 13:00:50.533363  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2721105490000468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
W20260709 13:00:50.533711  1658 connection.cc:256] RPC call timeout handler was delayed by -1.225310549000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18710000000000002s for a response
W20260709 13:00:50.533993  1658 connection.cc:256] RPC call timeout handler was delayed by -1.225310549000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18710000000000002s for a response
W20260709 13:00:50.537096  1658 connection.cc:256] RPC call timeout handler was delayed by -1.201910549000047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1897s for a response
W20260709 13:00:50.526520  1509 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.037s (SENT)
I20260709 13:00:50.537839  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:50.542644  2203 connection.cc:256] RPC call timeout handler was delayed by -2.5128891360000125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21760000000000002s for a response
I20260709 13:00:50.536026  4451 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:00:50.543251  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:50.526314  4201 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:50.544142  1539 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:50.544063  4201 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:50.546056  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:50.546412  1612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1884) took 1485 ms (client timeout 1798 ms). Trace:
I20260709 13:00:50.547080  4369 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.547724  4369 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:50.551676  1691 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:50.553164  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:50.546770  1612 rpcz_store.cc:269] 0709 13:00:49.061195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.088769 (+1027574us) service_pool.cc:224] Handling call
0709 13:00:50.546235 (+457466us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b54003c87d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:50.546358 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":669926656}
I20260709 13:00:50.554093  4383 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:50.554656  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2189) took 4490 ms. Trace:
I20260709 13:00:50.555197  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 470) took 2319 ms. Trace:
I20260709 13:00:50.555639  4201 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:50.555640  1611 rpcz_store.cc:276] 0709 13:00:48.235464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:48.235605 (+   141us) service_pool.cc:224] Handling call
0709 13:00:48.874586 (+638981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.079196 (+204610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.535042 (+1455846us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b5400104d50. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:50.555001 (+ 19959us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1468732800}
W20260709 13:00:50.559167  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.552745  1437 catalog_manager.cc:5697] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 3 to 4, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 4 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:50.555125  1690 rpcz_store.cc:276] 0709 13:00:46.064398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.273801 (+3209403us) service_pool.cc:224] Handling call
0709 13:00:50.397843 (+1124042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:50.399709 (+  1866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.554467 (+154758us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54002ccc50. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:50.554602 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2584055168}
W20260709 13:00:50.561805  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 107353us
I20260709 13:00:50.565817  4421 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:50.570034  1613 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:50.571143  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1377) took 2329 ms. Trace:
I20260709 13:00:50.571586  1615 rpcz_store.cc:276] 0709 13:00:48.241810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.269852 (+2028042us) service_pool.cc:224] Handling call
0709 13:00:50.570969 (+301117us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b54000889d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:50.571081 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155868544}
W20260709 13:00:50.537441  1658 connection.cc:256] RPC call timeout handler was delayed by -1.201910549000047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1897s for a response
W20260709 13:00:50.573796  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0513441510000234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1946s for a response
W20260709 13:00:50.574118  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0418281690000413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
I20260709 13:00:50.566108  4402 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:50.577407  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.577606  4369 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:50.578667  1685 raft_consensus.cc:2410] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:50.582680  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.583685  1685 raft_consensus.cc:2410] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:00:50.577641  2203 leader_election.cc:336] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.937s (SENT)
I20260709 13:00:50.577409  1437 catalog_manager.cc:5697] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 3 to 5, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 5 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:50.589201  1616 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:50.591097  1542 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:00:50.592365  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.583477  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:50.594558  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:50.594353  1535 raft_consensus.cc:2376] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8. Re-sending same reply.
I20260709 13:00:50.593957  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:50.599040  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.599288  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:50.601229  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 83434us
I20260709 13:00:50.053119  1543 rpcz_store.cc:276] 0709 13:00:47.491464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:49.905454 (+2413990us) service_pool.cc:224] Handling call
0709 13:00:49.905968 (+   514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:49.906471 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:49.906872 (+   401us) log.cc:844] Serialized 14 byte log entry
0709 13:00:50.032516 (+125644us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:50.032586 (+    70us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:50.032643 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.032727 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.032884 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36402,"prepare.queue_time_us":2917,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"thread_start_us":83241,"threads_started":1,"wal-append.queue_time_us":147700}
I20260709 13:00:50.604228  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.608901  1691 raft_consensus.cc:2410] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:00:50.608752  1583 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:50.576368  4402 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:50.612742  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:50.613777  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.614658  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.614107  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.608582  1437 catalog_manager.cc:5697] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.620147  1537 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:50.614841  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.623136  1687 raft_consensus.cc:2410] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:00:50.625662  4479 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.107s	user 0.004s	sys 0.000s
I20260709 13:00:50.632349  4468 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:50.632956  4468 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.634297  1437 catalog_manager.cc:5697] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 4 to 5, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:00:50.635432  1509 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.170s (SENT)
W20260709 13:00:50.636516  1509 leader_election.cc:336] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.285s (SENT)
I20260709 13:00:50.635891  1437 catalog_manager.cc:5697] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 1 to 2, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:50.636941  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:50.633322  4468 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.613785  1685 raft_consensus.cc:2376] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6. Re-sending same reply.
I20260709 13:00:50.641367  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:50.648792  2201 connection.cc:256] RPC call timeout handler was delayed by -1.1012878090000413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
W20260709 13:00:50.651376  2201 connection.cc:256] RPC call timeout handler was delayed by -1.0193878090000226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15960000000000002s for a response
I20260709 13:00:50.650620  1690 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:50.656594  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.657269  1689 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:50.657603  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:50.657691  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 52793.4us
W20260709 13:00:50.659119  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 62055us
I20260709 13:00:50.659775  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:50.661741  1687 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:50.666642  4481 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:50.654808  4148 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.668130  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.668447  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:50.669092  1683 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:50.670315  1683 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:50.672070  1683 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:50.674300  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:50.675354  1689 raft_consensus.cc:2393] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:50.669281  1685 raft_consensus.cc:2410] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 13:00:50.677942  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 220699us
W20260709 13:00:50.678647  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 106477us
I20260709 13:00:50.649992  4450 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:50.696341  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.655400  1437 catalog_manager.cc:5697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:00:50.646327  4434 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:50.677102  4402 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.668152  4148 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.702284  4235 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.702862  4235 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.705044  1685 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:50.707227  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.708812  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.710474  1687 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:50.713105  1686 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:00:50.714231  4484 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:00:50.714512  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.715034  4235 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:50.715508  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1388) took 1737 ms. Trace:
I20260709 13:00:50.715889  1618 rpcz_store.cc:276] 0709 13:00:48.977755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.396775 (+1419020us) service_pool.cc:224] Handling call
0709 13:00:50.397155 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:50.403075 (+  5920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:50.403108 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:50.404084 (+   976us) log.cc:844] Serialized 26 byte log entry
0709 13:00:50.538966 (+134882us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:50.539013 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:50.587302 (+ 48289us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:50.715247 (+127945us) raft_consensus.cc:1685] finished
0709 13:00:50.715268 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:50.715423 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":81,"thread_start_us":134410,"threads_started":1,"wal-append.queue_time_us":188712}
W20260709 13:00:50.718114  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:50.718603  2203 leader_election.cc:336] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.977s (SENT)
W20260709 13:00:50.718642  1657 connection.cc:256] RPC call timeout handler was delayed by -1.5567379460000552s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 13:00:50.719336  1657 connection.cc:256] RPC call timeout handler was delayed by -1.0778397800000334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
W20260709 13:00:50.719769  1657 connection.cc:256] RPC call timeout handler was delayed by -1.067039780000016s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
W20260709 13:00:50.719946  2203 leader_election.cc:336] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.068s (SENT)
I20260709 13:00:50.720433  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:50.721587  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.721668  1657 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:50.722299  4413 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:50.722944  4235 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:50.724426  4411 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:50.726967  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:50.733611  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:50.746657  1437 catalog_manager.cc:5697] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:50.749518  1437 catalog_manager.cc:5697] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 4 to 6, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 6 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.751358  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:50.752285  1686 raft_consensus.cc:2410] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:50.748260  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:50.750751  2203 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.137s (SENT)
I20260709 13:00:50.736678  4235 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:50.754853  4235 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:00:50.755345  4480 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.215s	user 0.002s	sys 0.000s
W20260709 13:00:50.755654  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.736460  4371 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:50.761652  4482 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.222s	user 0.002s	sys 0.005s
W20260709 13:00:50.762791  2203 connection.cc:256] RPC call timeout handler was delayed by -1.8088593900000092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
W20260709 13:00:50.763342  2203 connection.cc:256] RPC call timeout handler was delayed by -1.8088593900000092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1525s for a response
I20260709 13:00:50.761159  4130 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:00:50.763984  2203 connection.cc:256] RPC call timeout handler was delayed by -1.0681593900000053s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2348s for a response
W20260709 13:00:50.764444  2203 connection.cc:256] RPC call timeout handler was delayed by -1.0116109570000162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16970000000000002s for a response
W20260709 13:00:50.764896  2203 connection.cc:256] RPC call timeout handler was delayed by -1.0116109570000162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16970000000000002s for a response
W20260709 13:00:50.751020  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46433.7us
I20260709 13:00:50.752315  1437 catalog_manager.cc:5697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 3 to 5, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 5 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:50.767199  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:50.777585  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 78855.4us
I20260709 13:00:50.763789  4130 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:50.785885  4457 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:50.786186  4130 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:50.786616  4457 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.787587  4201 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:50.788177  4201 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:50.788595  4148 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:50.789471  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.789142  4201 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.791543  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.789186  4148 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:50.792786  4148 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.740701  4468 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.796645  1687 raft_consensus.cc:2368] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
I20260709 13:00:50.802243  1685 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:50.805178  1683 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:50.808746  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:50.809365  1690 raft_consensus.cc:2376] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6. Re-sending same reply.
I20260709 13:00:50.812893  1683 pending_rounds.cc:85] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 7
I20260709 13:00:50.813316  1683 pending_rounds.cc:107] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:00:50.813611  1683 raft_consensus.cc:2889] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:50.814795  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:50.816974  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.805445  1437 catalog_manager.cc:5697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 4, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 4 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:50.822597  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.828704  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:50.830122  2233 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:50.838930  2230 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:50.817997  4130 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:50.838379  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:50.837870  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:50.844026  2233 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:50.845793  1437 catalog_manager.cc:5697] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 3 to 6, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 6 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.825320  4468 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:50.853618  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.854141  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:50.867185  1692 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:50.868111  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 43706.5us
I20260709 13:00:50.869905  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.873745  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.877806  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:50.878134  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:50.878947  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.879272  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:50.860249  4338 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:50.885207  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 60171.3us
I20260709 13:00:50.888046  2230 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:50.893290  2229 raft_consensus.cc:2410] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:50.907939  4235 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.898111  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:50.879878  4383 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:50.899191  4371 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:00:50.914311  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.915124  4481 raft_consensus.cc:3055] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:50.915426  2229 raft_consensus.cc:2410] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:00:50.915547  4481 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:50.915956  1689 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:50.916549  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.916679  4481 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:50.918401  4481 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:50.918834  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.919620  2231 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:50.920518  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.900147  2201 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:50.895103  2236 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 3: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:50.910075  1686 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 13:00:50.891819  4468 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:00:50.911744  1692 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:50.893184  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:50.889307  4148 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:50.893127  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:50.922461  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.889317  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:50.903973  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:50.926308  2231 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:50.892861  1437 catalog_manager.cc:5697] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 5 to 6, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 6 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:50.930142  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.930294  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.930891  2231 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:50.931092  2234 raft_consensus.cc:2410] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:00:50.931818  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.932997  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:50.933684  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 46581.6us
I20260709 13:00:50.933974  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:50.934056  4411 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:50.934705  4411 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:50.935075  4411 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:50.931879  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.937014  2234 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:50.937213  1692 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:50.938416  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:50.938776  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:50.939019  1687 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:50.938465  2234 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:50.881752  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.945295  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.947785  2233 raft_consensus.cc:2410] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:00:50.950063  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.961618  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:50.942157  4338 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:50.964538  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.967240  2201 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:50.972200  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 68443.2us
I20260709 13:00:50.985410  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:50.986749  1686 raft_consensus.cc:2410] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:00:50.973958  1437 catalog_manager.cc:5697] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 6, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 6 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.953054  4371 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:51.013710  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:50.971383  4402 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.016940  1437 catalog_manager.cc:5697] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:50.974494  4468 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:51.019214  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 127319us
W20260709 13:00:51.019855  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.016243  1687 raft_consensus.cc:2410] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 13:00:51.044785  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 106066us
W20260709 13:00:51.045940  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:51.046255  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:51.046497  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.012987  4148 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:51.016409  4371 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:51.060185  1509 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:51.032370  4468 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.032698  4103 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:00:51.084033  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 116932us
I20260709 13:00:51.048583  4450 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.055987  4148 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.056188  4457 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:51.059732  4383 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:00:51.107416  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.082818  4371 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.085179  4468 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:51.090054  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 192026us
I20260709 13:00:51.099674  1584 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:51.090762  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.085721  4103 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:51.101859  1437 catalog_manager.cc:5697] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:51.086473  4434 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:51.117529  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:51.110160  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:51.120707  1437 catalog_manager.cc:5697] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 6, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 6 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:51.122361  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:51.099328  4148 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:51.122611  1613 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:51.123224  1616 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 4: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:51.129921  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:51.130157  4383 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:51.095115  4501 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:51.129990  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:51.146102  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:51.150374  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 93790.6us
I20260709 13:00:51.150249  1618 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:51.150198  1437 catalog_manager.cc:5697] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 4, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 4 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:00:51.167953  1616 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:51.171705  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:51.166998  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:51.185880  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:51.187450  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.151664  4501 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:51.113170  4450 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.146541  4507 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:00:51.166838  4506 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:51.156983  4130 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:51.204082  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 145851us
I20260709 13:00:51.202112  1437 catalog_manager.cc:5697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 6, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 6 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:00:51.210515  4103 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:51.213918  4494 log.cc:927] Time spent T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:00:51.215189  4235 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:51.215914  4235 raft_consensus.cc:2764] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:00:51.218742  4450 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:51.219321  4130 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.222208  4130 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:51.222730  4130 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:51.223212  4130 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:51.197319  4501 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:51.211133  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:51.197608  4369 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.225230  4130 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:51.227022  1582 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:51.235905  4421 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.237252  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:51.238281  4421 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:51.227079  4413 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 13:00:51.233824  4369 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:51.229694  1615 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:51.245576  1437 catalog_manager.cc:5697] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:00:51.248468  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 148116us
I20260709 13:00:51.213963  4103 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:51.257328  4421 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:51.258744  4421 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:51.259202  4421 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:51.260167  4421 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:51.261615  4421 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.254231  1584 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:51.269469  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:51.271157  4419 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:51.271775  4419 raft_consensus.cc:697] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:51.272511  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:51.272859  1539 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:51.273373  1534 raft_consensus.cc:2410] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:00:51.273672  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 78581.4us
I20260709 13:00:51.274271  1535 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:51.274557  1509 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:51.276139  1538 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:51.276636  1536 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:51.276989  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:51.277626  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:51.278221  4130 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:51.269570  4513 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:51.272785  4419 consensus_queue.cc:237] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:51.256749  1582 leader_election.cc:400] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
I20260709 13:00:51.225376  4371 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.253646  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:51.246167  4506 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:51.286706  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:51.295229  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.299151  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:51.292743  1437 catalog_manager.cc:5697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 2 to 4, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 4 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
W20260709 13:00:51.308748  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 52202.9us
I20260709 13:00:51.309701  1582 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:00:51.311964  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.310305  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:51.316733  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 129749us
I20260709 13:00:51.294737  4201 raft_consensus.cc:2764] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:00:51.327207  1437 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 6 to 8, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 8 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:00:51.328261  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 51369.7us
I20260709 13:00:51.328454  1615 raft_consensus.cc:2393] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:51.298044  4402 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:51.264479  4148 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.304785  4499 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:51.332187  1437 catalog_manager.cc:5697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:00:51.334290  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:51.339828  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:51.346951  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:51.347808  1611 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:51.348352  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:51.349339  1616 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:51.350564  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:51.355387  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.355290  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:51.356630  1610 raft_consensus.cc:2410] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:00:51.306334  4506 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:51.358246  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:51.359406  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 72796.4us
I20260709 13:00:51.361800  1509 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:51.370285  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:51.361898  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:51.341672  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 191395us
I20260709 13:00:51.327219  4507 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:51.333647  4310 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:51.390777  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:51.392807  1611 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:51.387710  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:51.393904  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.360397  4513 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:51.395290  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:51.359249  4130 raft_consensus.cc:2764] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:00:51.401543  1612 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:00:51.395089  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 68110.5us
I20260709 13:00:51.404136  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:51.364087  4402 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:51.410459  4507 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.414232  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:51.415273  1616 raft_consensus.cc:2393] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:00:51.418737  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 166900us
I20260709 13:00:51.423539  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:51.399896  4383 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:51.400980  4320 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:51.405453  4369 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:51.428664  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:51.369819  4411 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.415810  4432 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.417893  4310 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:51.443632  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:51.450040  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 88314.2us
I20260709 13:00:51.452986  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:51.454371  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:51.454919  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:51.455317  4310 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:51.456202  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:51.449045  4148 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:51.426079  4513 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.423760  4530 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.042s
W20260709 13:00:51.437323  4516 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 13:00:51.468994  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:51.469691  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:51.470085  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:51.445326  4515 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:00:51.436542  4383 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:00:51.504231  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 49965.9us
I20260709 13:00:51.510391  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:51.511647  1541 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
W20260709 13:00:51.445732  4510 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 13:00:51.439198  4512 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.445487  4411 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:51.430168  4468 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:51.519266  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 63148.5us
I20260709 13:00:51.462381  4450 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:51.462693  4371 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:51.462088  4522 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:51.514982  4481 consensus_meta.cc:287] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.596s	user 0.008s	sys 0.003s
W20260709 13:00:51.520649  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 170188us
I20260709 13:00:51.505715  4505 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:51.527248  4148 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:00:51.523792  4450 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:51.459811  4504 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
W20260709 13:00:51.534721  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56888.9us
I20260709 13:00:51.505203  4517 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:51.506392  4503 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:51.536443  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 112075us
I20260709 13:00:51.512107  4103 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 13:00:51.520503  4201 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.538636  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:51.523109  4320 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.539937  2203 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:51.561730  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:51.552706  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.553514  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:51.534667  4505 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:51.571671  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:51.574512  1541 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:51.566805  1582 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:51.539641  4450 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.466179  4506 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:51.575656  1582 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:51.546171  4517 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:51.542732  4522 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:51.602975  1582 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:51.580448  4504 consensus_queue.cc:1059] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
W20260709 13:00:51.595558  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 55699us
W20260709 13:00:51.596262  1507 leader_election.cc:336] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.551s (SENT)
I20260709 13:00:51.535763  4501 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.578253  4512 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:51.599098  4419 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:51.552428  4503 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:51.580169  4310 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:51.629843  4310 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.630385  4310 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:51.614182  4512 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:51.632279  4512 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.633625  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:51.634991  4503 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:51.598737  4421 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:00:51.636716  4502 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.416s	user 0.000s	sys 0.006s
W20260709 13:00:51.603706  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 62958.4us
I20260709 13:00:51.639281  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:51.640559  4514 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:51.640625  1658 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:51.641135  4514 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:51.639216  4310 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:51.641649  4514 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:51.642520  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.642717  4499 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:51.643066  4501 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:51.643263  4499 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:51.643863  4499 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:51.644150  4517 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.643872  4434 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.646564  4434 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:51.648710  4434 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:51.654525  4530 raft_consensus.cc:3055] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:51.655197  4530 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:51.541846  4402 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.577986  4505 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:51.546337  4523 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:00:51.654222  4522 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.611109  1616 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:00:51.565392  4201 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:51.674598  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56771.4us
I20260709 13:00:51.607666  4371 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:51.694262  1541 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
W20260709 13:00:51.697165  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.702875  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:00:51.704141  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.569999  4320 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.709456  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:51.711474  1536 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:00:51.713259  4508 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:51.713328  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:51.713933  4508 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:51.714372  1541 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.715147  4508 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.713613   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:51.716900  4508 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:51.718659  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 106991us
W20260709 13:00:51.725555  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 150187us
W20260709 13:00:51.727784  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.697700  4369 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:51.733261  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:51.737008  1541 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:00:51.738008  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:51.739068  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:51.722777  4148 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:51.745760  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.748288  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:51.752275  1683 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:51.755400  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:51.756243  1616 raft_consensus.cc:2376] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6. Re-sending same reply.
I20260709 13:00:51.613340  4383 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.733669  4413 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:51.656688  4530 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.764503  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:51.767949  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45992.2us
I20260709 13:00:51.770561  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:51.771466  1616 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:51.773856  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:51.786028  1614 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:51.789341  1615 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
W20260709 13:00:51.792268  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.786242  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:51.754196  4148 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:51.774887  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:51.794071  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:51.794629  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 55683.8us
I20260709 13:00:51.793205  1691 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:00:51.769639  4402 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:00:51.762293  4369 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:51.796269  1618 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:51.801698  1612 raft_consensus.cc:2368] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9.
I20260709 13:00:51.742955  4523 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:00:51.764856  4509 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:51.774446  4507 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:51.771248  4468 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:51.785808  4434 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:51.811574  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 82799.7us
I20260709 13:00:51.798934  4148 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:51.786584  4413 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:00:51.822932  1507 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.713s (SENT)
W20260709 13:00:51.824048  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 72412.9us
I20260709 13:00:51.839277  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:51.837157  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:51.854840  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 81062.3us
W20260709 13:00:51.861341  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 73177.4us
I20260709 13:00:51.821743  4501 raft_consensus.cc:2764] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:00:51.843726  4421 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:51.835690  4509 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:51.805521  4533 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.244s
I20260709 13:00:51.854411  4529 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:51.895838  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:51.850450  4507 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:51.833144  4434 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:51.913584  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68826.2us
I20260709 13:00:51.874053  4421 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:00:51.922397  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 167417us
I20260709 13:00:51.919379  4552 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:51.931272  4532 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:51.920676  4369 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:51.854398  4549 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.076s
W20260709 13:00:51.954226  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 156887us
W20260709 13:00:51.967856  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 53979.3us
I20260709 13:00:51.933132  4320 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:51.978237  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:51.980957  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:51.979151  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:51.962327  4201 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:51.969542  4413 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:51.998131  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:51.998919  2228 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:52.000033  4565 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:51.987053  2235 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:51.995255  4555 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:51.994936  4550 consensus_queue.cc:1059] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.104s
I20260709 13:00:51.995966  4434 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:52.010421  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 131546us
I20260709 13:00:51.967818  4552 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:52.013199  4529 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:51.985957  4481 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:00:52.020733  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 197830us
W20260709 13:00:52.025576  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 185199us
W20260709 13:00:52.027032  1507 leader_election.cc:336] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.746s (SENT)
I20260709 13:00:52.027822  1507 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:52.034162  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:52.039912  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:52.041445  4507 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.023736  4509 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.037326  4550 raft_consensus.cc:3055] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:52.052376  4550 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:52.053682  4550 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:52.055286  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:52.055608  4550 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:52.056080  1612 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:52.023551  4421 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.057519  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:52.058164  1688 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:52.058393  4421 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:52.059849  4562 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:00:52.053961  4529 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.041801  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:52.069175  1507 leader_election.cc:336] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.770s (SENT)
I20260709 13:00:52.069804  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:52.013307  4501 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:00:52.082499  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:52.086462  1509 leader_election.cc:400] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9.
I20260709 13:00:52.087082  1509 leader_election.cc:403] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:00:52.044718  4201 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:52.091650  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:51.984555  4369 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.043350  4481 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:52.114440  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:52.115723  1614 raft_consensus.cc:2410] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:00:52.115971  4544 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 13:00:52.013643  4555 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:52.116266  4565 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:52.117125  4551 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:00:52.123633  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:52.125850  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:52.127223  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:52.044330  4434 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.053s
W20260709 13:00:52.131805  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 48816.1us
I20260709 13:00:52.099931  4517 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:52.072017  4532 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:00:52.097630  4201 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:52.122812  1658 leader_election.cc:336] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.798s (SENT)
I20260709 13:00:52.167886  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:52.138934  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.102540  4432 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:52.159469  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:52.174557  2236 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 2. Current term is 3. Ops: []
I20260709 13:00:52.118395  4562 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:52.138070  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:52.180135  1618 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:52.136433  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:52.161819  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:52.171226  1508 leader_election.cc:336] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.635s (SENT)
I20260709 13:00:52.041502  4552 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:52.159452  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:52.212150  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 76077.9us
I20260709 13:00:52.213840  1542 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:52.212001  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:52.176424  4434 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:52.215723  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:52.175613  4201 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.218550  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:52.177862  4130 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.220731  1612 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:52.179076  4562 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:52.178848  4555 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:52.222908  1536 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.511s	user 0.008s	sys 0.004s
I20260709 13:00:52.229537  1688 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:00:52.230808  1542 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:52.231195  1536 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:00:52.233388  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:52.234556  2236 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20260709 13:00:52.235349  4547 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.254s	user 0.001s	sys 0.001s
I20260709 13:00:52.236660  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:52.237828  4566 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:52.238394  2203 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:52.238927  4568 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:52.239332  4566 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:52.239605  4568 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.239825  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:52.240180  4568 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.240832  4566 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:52.230916  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:52.247506  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:52.245863  1657 leader_election.cc:336] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.850s (SENT)
I20260709 13:00:52.241328  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:52.251838  1618 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:00:52.248245  1535 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
W20260709 13:00:52.259400  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.261713  4558 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.262410  4558 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.247779  1543 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:52.263571  4501 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:52.220129  4434 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:52.265295  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:00:52.244410  4371 consensus_meta.cc:287] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.544s	user 0.011s	sys 0.000s
I20260709 13:00:52.270242  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:52.228519  4554 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:52.296465  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:52.297533  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:52.301982  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 60728.3us
W20260709 13:00:52.303118  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 233970us
I20260709 13:00:52.228988  4130 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.244305  4565 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9.
I20260709 13:00:52.293689  4371 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:00:52.317303  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 201710us
I20260709 13:00:52.296464  4434 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:52.322247  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0715606000000548s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3028s for a response
I20260709 13:00:52.293598  4532 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:00:52.322816  1507 leader_election.cc:336] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.951s (SENT)
I20260709 13:00:52.293102  4558 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:52.301944  4419 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:52.334401  2233 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:52.254345  4481 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.335819  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:52.323582  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:52.336741  4577 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:52.373759  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:52.337224  4559 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:52.337052  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:52.337723  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:52.336767  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:52.338711  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:52.336982  4572 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 13:00:52.342239  4574 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:00:52.343148  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:52.343999  2234 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
W20260709 13:00:52.346146  4569 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 13:00:52.347615  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:52.394188  1614 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:52.348846  4529 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:52.351533  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:52.352984  4499 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.708s	user 0.013s	sys 0.000s
I20260709 13:00:52.350543  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:52.326906  4554 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:52.325637  4517 raft_consensus.cc:697] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:52.339530  4501 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:52.360693  4512 consensus_meta.cc:287] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.728s	user 0.007s	sys 0.005s
W20260709 13:00:52.361084  4557 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 13:00:52.361624  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:52.365005  1535 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:00:52.373975  1685 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:00:52.407621  1535 pending_rounds.cc:85] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 3
I20260709 13:00:52.327548  4532 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:52.348289  4508 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.627s	user 0.006s	sys 0.003s
I20260709 13:00:52.385289  1692 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:52.418444  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:52.419766  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:52.420362  1692 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 3.
W20260709 13:00:52.420902  4450 consensus_meta.cc:287] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.812s	user 0.009s	sys 0.000s
I20260709 13:00:52.388177  2234 pending_rounds.cc:85] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 4
I20260709 13:00:52.421602  4584 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:52.421643  2234 pending_rounds.cc:107] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 13:00:52.421871  4586 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:52.422255  2234 raft_consensus.cc:2889] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:52.422574  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 118812us
I20260709 13:00:52.414621  4512 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:52.424710  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.425105  4402 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:52.425902  4529 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:52.427459  4402 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.429517  1692 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:52.430226  4402 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:52.431492  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:52.426051  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 75606us
I20260709 13:00:52.392202  1615 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:52.398187  1689 raft_consensus.cc:2368] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9.
I20260709 13:00:52.331931  4419 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:52.358174  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.407928  1535 pending_rounds.cc:107] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:00:52.442818  1535 raft_consensus.cc:2889] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:52.407850  1685 raft_consensus.cc:606] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:52.443619  1685 raft_consensus.cc:614] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:52.385082  4434 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.383970  4577 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.412171  4558 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.405583  4499 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.464675  4577 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:52.412982  4554 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:52.411393  1612 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:52.326121  4522 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
W20260709 13:00:52.414279  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 62900.9us
I20260709 13:00:52.416517  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:52.553490  2233 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:52.439528  1686 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:52.553742  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:52.457111  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:52.403796  4559 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:52.453527  4419 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:52.421334  4450 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.493451  4421 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:00:52.321758  4562 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:52.512099  2234 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:52.434315  4532 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:52.414990  4501 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:52.579128  1687 raft_consensus.cc:2410] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:00:52.579752  4575 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.306s	user 0.004s	sys 0.000s
I20260709 13:00:52.580199  4310 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:52.501879  4558 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.498484  4517 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:52.511385  4561 log.cc:927] Time spent T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.288s	user 0.000s	sys 0.002s
W20260709 13:00:52.582772  4550 consensus_meta.cc:287] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.518s	user 0.006s	sys 0.004s
I20260709 13:00:52.583544  4130 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:52.592013  4514 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.950s	user 0.009s	sys 0.002s
I20260709 13:00:52.592619  4514 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:52.513562  1657 leader_election.cc:336] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.622s (SENT)
W20260709 13:00:52.467415  4505 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.769s	user 0.010s	sys 0.002s
W20260709 13:00:52.555259  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 123980us
W20260709 13:00:52.589159  4536 log.cc:927] Time spent T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.847s	user 0.000s	sys 0.003s
I20260709 13:00:52.562438  4421 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:52.611147  4421 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:52.606976  4130 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.611073  4505 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:52.611989  4130 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:52.613790  1690 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:52.606101  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:52.614620  4130 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:52.572598  4568 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:52.574160  1685 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:00:52.551291  4513 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:52.593456  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.594152  4565 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:52.594478  4579 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:52.595611  4503 consensus_meta.cc:287] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.960s	user 0.011s	sys 0.002s
W20260709 13:00:52.457588  4148 consensus_meta.cc:287] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.604s	user 0.004s	sys 0.007s
I20260709 13:00:52.593120  4530 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:52.709345  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:52.716320  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.676038  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:52.717594  1685 raft_consensus.cc:606] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Received request to transfer leadership to TS cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:52.718128  1685 raft_consensus.cc:614] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:00:52.718581  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 164906us
I20260709 13:00:52.719434  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:52.714176  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 257173us
I20260709 13:00:52.721678  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:52.559252  4586 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:00:52.703202  4545 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.884s	user 0.005s	sys 0.007s
W20260709 13:00:52.754035  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 240502us
W20260709 13:00:52.758457  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 320678us
I20260709 13:00:52.755883  1507 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:52.757660  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 56226us
W20260709 13:00:52.761200  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.713775  2233 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:52.604951  4457 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:52.703814  4505 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:52.659844  4130 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:52.704041  4581 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:52.615285  4512 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:52.776072  4432 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:52.616506  4550 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:52.777727  4550 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:52.779037  4103 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:00:52.785781  1657 leader_election.cc:336] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.634s (SENT)
I20260709 13:00:52.709568  4562 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:52.774361  4457 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:52.774688  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:52.724309  4148 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.792650  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:52.775573  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:52.775482  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:52.724555  4499 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:52.715803  4419 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:52.702121  4421 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.699278  4573 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:00:52.613250  4411 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.154s
I20260709 13:00:52.793712  1542 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:52.786299  1508 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:52.794951  2234 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:52.808017  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:52.808766  1541 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
W20260709 13:00:52.810032  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 54213.7us
I20260709 13:00:52.814211  2236 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:52.814608  1508 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:52.788297  4576 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:52.786532  1657 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:52.755650  4513 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.716122  4532 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.791900  4383 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:52.818593  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:52.803802  2235 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:52.820891  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2237) took 1070 ms. Trace:
I20260709 13:00:52.794132  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:52.823133  1690 rpcz_store.cc:276] 0709 13:00:51.750029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.751743 (+  1714us) service_pool.cc:224] Handling call
0709 13:00:51.752234 (+   491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:52.614214 (+861980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.820647 (+206433us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b540003eb54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:52.820821 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1980902272}
I20260709 13:00:52.755448  4503 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:52.788674  4130 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:00:52.833360  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47626.5us
I20260709 13:00:52.808436  4432 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:52.705127  4559 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:52.822118  4450 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:52.833797  4509 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.011s	sys 0.018s
I20260709 13:00:52.824846  4565 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:52.852185  1688 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:52.824466  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:52.862197  1535 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:00:52.861243  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:52.835446  4558 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:52.853785  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:52.854773  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2402) took 1279 ms. Trace:
I20260709 13:00:52.869194  1537 rpcz_store.cc:276] 0709 13:00:51.575637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:51.575824 (+   187us) service_pool.cc:224] Handling call
0709 13:00:51.595856 (+ 20032us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:51.605140 (+  9284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:51.605171 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:51.616626 (+ 11455us) log.cc:844] Serialized 26 byte log entry
0709 13:00:51.699018 (+ 82392us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:51.699377 (+   359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:51.699458 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:52.702446 (+1002988us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:00:52.853323 (+150877us) raft_consensus.cc:1685] finished
0709 13:00:52.853350 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:52.854689 (+  1339us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1069,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":14696,"spinlock_wait_cycles":6178176,"thread_start_us":62837,"threads_started":1,"wal-append.queue_time_us":121264}
I20260709 13:00:52.871548  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:52.865597  1538 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:52.852842  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 133453us
I20260709 13:00:52.845870  4499 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:52.854800  4563 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.006s	sys 0.014s
W20260709 13:00:52.857616  4509 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.012s	sys 0.018s
I20260709 13:00:52.851328  4584 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:52.831022  4552 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 13:00:52.876947  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:53.157470  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:53.252184  1611 raft_consensus.cc:2376] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6. Re-sending same reply.
I20260709 13:00:53.253461  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:53.254976  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:53.255772  1611 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:53.256783  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:53.275763  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:53.148617  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:53.105285  4593 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:53.073799  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 196360us
I20260709 13:00:53.278532  4593 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:53.279570  4504 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:00:53.281538  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.283977  1540 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:00:53.285238  4594 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.285795  4594 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:53.281215  4593 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:53.155637  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.277868  4554 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:53.290827  2229 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:53.291265  1614 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:53.135989  4457 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:53.141036  1542 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 13:00:53.292186  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.292488  1614 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:00:53.060751  4513 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:53.112843  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:53.294253  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:53.295133  1687 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
W20260709 13:00:53.295408  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 489959us
I20260709 13:00:53.296205  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:53.157472  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 747) took 1061 ms. Trace:
I20260709 13:00:53.293407  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.298537  2229 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:53.130520  1691 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:00:53.300266  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:53.300660  1691 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:53.301415  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:53.302629  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.303872  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:53.305017  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:53.305765  1536 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:53.146625  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:53.153839  1541 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:53.309474  1687 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:53.311682  1687 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:00:53.313202  2229 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:00:53.314267  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.315603  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.317261  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.160159  1610 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:53.147583  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:53.111802  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:53.318547  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:53.124871  4589 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.402s	user 0.001s	sys 0.001s
I20260709 13:00:53.148541  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:53.320709  1612 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:53.143206  1537 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:00:53.135046  1689 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:53.105286  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:53.147641  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:53.322844  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.323078  1689 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:53.324050  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.323731  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:53.106113  4577 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.001s
I20260709 13:00:53.325634  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.325700  4577 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:53.098282  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.047676  4421 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:52.979076  4587 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:00:53.130012  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:52.882496  4563 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.006s	sys 0.015s
W20260709 13:00:53.110644  4201 consensus_meta.cc:287] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.878s	user 0.012s	sys 0.000s
W20260709 13:00:53.110308  4434 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.617s	user 0.009s	sys 0.000s
W20260709 13:00:52.979883  4590 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
W20260709 13:00:53.107856  4599 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
W20260709 13:00:52.969271  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 157324us
I20260709 13:00:53.046453  4532 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:53.101013  4130 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:52.969326  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:52.930562  4419 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:53.331143  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.331704  2236 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:53.332988  2229 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.333607  4201 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:53.335482  1534 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:00:53.050287  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.338307  4558 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:00:53.042636  4595 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:00:53.151788  4550 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:53.175349  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:53.176499  1685 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:00:53.341403  1685 raft_consensus.cc:606] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:00:53.341789  1685 raft_consensus.cc:614] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:53.344830  1685 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:00:53.345391  1685 raft_consensus.cc:606] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:00:53.345834  1685 raft_consensus.cc:614] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:53.346174  4577 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:53.331216  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.333175  2236 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:00:53.348362  4201 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:53.182528  4509 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:53.349779  4509 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:53.350100  4509 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:53.350443  4501 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:53.351145  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.351230  1691 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:53.350948  4509 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:53.352223  4509 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.353667  1685 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:00:53.354357  1685 raft_consensus.cc:606] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:00:53.355243  1685 raft_consensus.cc:614] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:00:53.183835  1657 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:52.932323  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:52.969066  4573 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.356016  2231 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:53.356083  4573 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:53.356587   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:00:53.357170  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 173422us
I20260709 13:00:53.191325  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:53.202405  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:53.362635  1617 raft_consensus.cc:2410] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:00:52.930554  4514 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:53.235896  4549 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.363476  4549 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:53.364060  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 758) took 1023 ms. Trace:
I20260709 13:00:53.364420  2230 rpcz_store.cc:276] 0709 13:00:52.380487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:52.380610 (+   123us) service_pool.cc:224] Handling call
0709 13:00:52.380981 (+   371us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:52.385878 (+  4897us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:52.385922 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:52.434981 (+ 49059us) log.cc:844] Serialized 52 byte log entry
0709 13:00:52.804048 (+369067us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:52.804089 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:52.804150 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:53.363813 (+559663us) raft_consensus.cc:1685] finished
0709 13:00:53.363834 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:53.363985 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":202,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":17471,"thread_start_us":239978,"threads_started":1,"wal-append.queue_time_us":397448}
I20260709 13:00:53.365630  4549 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:53.368060  2229 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:53.240016  4148 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:52.929812  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.290107  4594 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:53.296933  4533 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:53.374758  2234 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:00:53.297605  2235 rpcz_store.cc:276] 0709 13:00:52.096569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:52.096713 (+   144us) service_pool.cc:224] Handling call
0709 13:00:52.129344 (+ 32631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:52.825142 (+695798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:53.157208 (+332066us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54006586d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:53.157393 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1550614400}
I20260709 13:00:53.106475  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:53.108975  4598 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.127s	user 0.001s	sys 0.001s
I20260709 13:00:53.377684  2228 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 5: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:53.158215  4524 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:53.318707  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.378511  4524 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:53.378865  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.379539  4524 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:53.380115  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:53.380937  4524 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:53.381704  4594 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:53.381978  4419 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:00:53.382258  4594 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:53.382925  4419 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:53.383622  4419 raft_consensus.cc:3055] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:53.384028  4419 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:53.384572  4594 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:53.385085  4419 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:53.386778  4419 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:53.326182  1690 pending_rounds.cc:85] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 3
I20260709 13:00:53.388269  1690 pending_rounds.cc:107] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:00:53.388638  1690 raft_consensus.cc:2889] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:53.177090  1535 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:53.348984  1542 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:00:53.357708  4201 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:53.358385  4573 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:52.982156  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:53.358582   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:00:53.325224  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:53.325745  1616 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:53.104745  1538 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:00:53.389030  1583 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:52.875361  4555 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.240785  4568 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:53.394771  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:53.407186  1687 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:53.396859  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:53.405725  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 423674us
I20260709 13:00:53.408576  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:53.411319  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:53.414687  1584 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:53.416585  1583 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:53.418478  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:53.421291  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 145697us
I20260709 13:00:53.394408  4201 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:53.422331  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:53.415864  4369 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 13:00:53.424034  4201 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.424775  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:53.427594  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:53.430152  1687 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:53.417227  4549 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.414419  4513 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:53.411078  4532 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:53.443897  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:53.414896  4555 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:53.465689  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 49185.1us
I20260709 13:00:53.442972  4505 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:53.430249  4533 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:53.474555  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:53.456640  4513 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.449242  4514 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.457487  4503 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:53.486773  1613 raft_consensus.cc:2376] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6. Re-sending same reply.
W20260709 13:00:53.487901  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 61986.9us
I20260709 13:00:53.490973  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:53.425779  4421 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:00:53.471167  4559 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:53.473767  4549 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:53.471210  4499 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:53.449921  4594 consensus_queue.cc:1059] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 13:00:53.447528  4566 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:00:53.482713  4533 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.489302  4513 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.500752  4554 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:53.499979  4504 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:53.515722  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 62849.2us
I20260709 13:00:53.516901  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:53.500329  4549 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.499919  4593 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:00:53.509877  4533 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:53.539238  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.536082  4555 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:53.540825  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:53.541052  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.551192  2230 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 5: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:53.554023  1690 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:53.557746  1508 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:53.560118  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:53.584229  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:53.557639  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 61474.8us
I20260709 13:00:53.560401  1683 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:00:53.648532  1683 pending_rounds.cc:85] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 6
I20260709 13:00:53.648937  1683 pending_rounds.cc:107] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:00:53.649168  1683 raft_consensus.cc:2889] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:53.630904  1657 leader_election.cc:336] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.525s (SENT)
I20260709 13:00:53.649957  1657 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:53.632175  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.650743  2235 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
W20260709 13:00:53.632941  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 77362.7us
I20260709 13:00:53.633096  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:53.633929  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:53.635617  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 89929.4us
I20260709 13:00:53.629026  1684 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:00:53.559602  4499 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:53.637369  4555 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.520545  4504 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.550725  4549 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.644264  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:53.685727  2231 raft_consensus.cc:2376] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5. Re-sending same reply.
I20260709 13:00:53.513285  4514 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:00:53.638567  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 195408us
I20260709 13:00:53.686713  4514 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:53.656822  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:53.660182  1507 leader_election.cc:336] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.635s (SENT)
W20260709 13:00:53.673606  1657 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.607s (SENT)
I20260709 13:00:53.632782  4562 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:00:53.677886  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 136136us
I20260709 13:00:53.682569  2232 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:53.683252  4555 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:53.683902  4504 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:53.631592  4532 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:00:53.689318  1657 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:53.640355  4559 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:00:53.702564  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:53.704406  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47664.4us
I20260709 13:00:53.628785  4530 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:00:53.628521  4535 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:00:53.653200  4592 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.689222  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:53.699995  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:53.688648  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:53.719288  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 45727.6us
I20260709 13:00:53.727630  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:53.718278  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:53.720671  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68418.3us
I20260709 13:00:53.680378  4499 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:53.722282  1508 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:53.698450  4562 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:53.700361  4554 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:53.703374  4568 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:00:53.731396  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 71235.4us
I20260709 13:00:53.700152  2232 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:53.757102  2232 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:53.758165  4632 consensus_queue.cc:1059] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:53.758778  2232 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:53.739494  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:53.745673  4517 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:00:53.746508  4611 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:00:53.746505  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:53.723074  4509 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:53.751122  4457 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:53.752568  4434 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:53.754885  4628 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:53.755847  4130 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:53.757791  4514 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:53.740137  4562 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.730242  4501 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.736682  4550 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:53.737751  4499 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.769590  1540 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:53.778484  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:53.741716  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 184807us
I20260709 13:00:53.784782  4201 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:53.730064  4592 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:53.788596  4549 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:53.740024  4566 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
W20260709 13:00:53.802603  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 115655us
I20260709 13:00:53.777395  4554 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:53.781009  4130 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:53.840049  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 191827us
I20260709 13:00:53.815407  4535 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:53.843336  4535 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:53.842346  4130 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:53.816423  1534 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:53.832110  2229 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
W20260709 13:00:53.842463  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 43695us
W20260709 13:00:53.849244  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 196722us
I20260709 13:00:53.911202  4592 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:53.851486  4530 raft_consensus.cc:3055] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:53.916996  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:53.937184  4530 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:53.899127  4533 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:53.938036  4535 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:53.894207  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 46809.8us
W20260709 13:00:53.886883  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 114769us
I20260709 13:00:53.900503  4640 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:00:53.783087  4434 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:53.887768  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:53.930917  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 201569us
W20260709 13:00:53.931454  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 81331.2us
I20260709 13:00:53.935657  2232 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:00:53.806816  4509 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:53.944851  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:53.955821  2232 pending_rounds.cc:85] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 7
I20260709 13:00:53.945510  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:53.956251  2232 pending_rounds.cc:107] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:00:53.957008  1538 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:53.957010  2232 raft_consensus.cc:2889] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:53.952201  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:53.958571  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:53.935251  4545 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:00:53.956188  1535 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:53.966874  1537 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:53.966662  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:53.932313  4554 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:53.959983  4514 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:00:53.974116  4505 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:53.974236  4514 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:53.974869  4514 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:53.975011  4201 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:53.975798  4514 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:53.938469  4530 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:53.983342  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 95728.7us
I20260709 13:00:53.964592  4517 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:53.964087  4577 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:53.985994  1507 leader_election.cc:336] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.561s (SENT)
I20260709 13:00:53.987589  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:53.987937  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:53.988386  1612 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:00:53.987272  4535 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 13:00:53.989900  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:53.992856  1537 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:53.994086  1535 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:00:53.995263  1507 leader_election.cc:336] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.571s (SENT)
I20260709 13:00:53.995566  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:53.995882  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:53.996848  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:53.998271  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:53.998240  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:53.998883  4201 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:53.999328  1616 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:53.999771  4201 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:00:54.002734  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.003463  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.003631  1685 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:00:54.008559  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:54.009527  1617 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:53.983548  1584 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:54.011005  1615 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:53.983965  4555 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:53.846089  4559 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:54.021754  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 49663.1us
I20260709 13:00:53.960995  4503 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:00:54.030251  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:54.030889  1611 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.033151  4632 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:54.033695  4632 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:54.029593  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:54.034580  4632 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.036144  1609 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:00:54.030951  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:53.959252  4532 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:00:54.042444  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:54.051293  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.022063  4635 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:00:54.051986  1685 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:53.976722  4513 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.030622  4201 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:54.058784  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:54.061584  1543 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.061930  1684 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:00:54.065785  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:53.979804  4554 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:54.043550  4517 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:54.066125  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.025988  4577 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:54.084980  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.051047  4653 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:54.097316  4659 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:00:54.099026  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.068542  4573 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:54.068991  4514 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:54.068480  4201 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.104930  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.105855  1686 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:54.106989  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:54.107124  1507 leader_election.cc:336] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.683s (SENT)
I20260709 13:00:54.112737  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:54.112993  2230 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:54.107692  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:54.068418  4632 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:54.046824  4505 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:00:54.068084  4533 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:54.114982  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.113802  2232 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:54.089684  4558 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:54.118072  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:54.119670  2233 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:54.121836  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:54.092537  4635 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:54.117794  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:54.104142  4653 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.114413  4201 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:54.110277  4549 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:54.123474  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:54.144469  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:54.123627  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:54.119371  4558 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.164143  4653 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.168393  1543 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
W20260709 13:00:54.172051  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.144622  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:54.111320  4573 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.178777  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:54.218662  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 235171us
W20260709 13:00:54.193816  4656 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.107s	user 0.003s	sys 0.000s
I20260709 13:00:54.231201  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.187227  2232 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:54.222414  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.232002  1692 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:54.232411  1686 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:54.223866  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:54.223624  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 192806us
W20260709 13:00:54.181442  4607 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.682s	user 0.002s	sys 0.003s
W20260709 13:00:54.203086  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 84382us
I20260709 13:00:54.178036  4558 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:54.229839  1690 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
W20260709 13:00:54.227747  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 189899us
I20260709 13:00:54.190909  4434 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:54.220309  4634 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.237690  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:54.242170  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:54.246198  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.250676  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:54.251820  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:54.255789  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:54.256945  2203 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:54.261617  1617 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:00:54.263808  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.264940  1692 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:54.246304  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.223790  4573 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:54.265774  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:54.267129  1615 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:54.165534  4635 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.269176  1613 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:54.271644  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:54.275215  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:54.290230  1543 raft_consensus.cc:2393] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:00:54.269593  4504 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.000s
I20260709 13:00:54.297585  1683 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:00:54.298256  1610 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:54.298448  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:54.297250  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:54.283924  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.297246  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:54.277643  4666 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:00:54.301594  4637 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.369s	user 0.001s	sys 0.001s
I20260709 13:00:54.304531  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:54.202507  4654 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.306428  2228 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:54.276437  4648 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:00:54.331342  1657 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:54.332794  1686 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:00:54.336341  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:54.322115  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:54.329084  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 85001.1us
I20260709 13:00:54.266036  4657 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:54.268160  4434 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:00:54.323874  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:54.275688  4639 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:54.310669  4513 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:54.322690  4649 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
W20260709 13:00:54.320608  4598 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:00:54.209813  4517 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.426820  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:54.571641  1692 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.334450  4631 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.491887  1582 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:54.489988  1683 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:54.573127  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:54.490516  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 218977us
I20260709 13:00:54.309413  4509 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:00:54.533569  1537 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:54.500260  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:54.504042  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 172765us
I20260709 13:00:54.422555  1611 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:00:54.482580  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.438619  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:54.596232  1543 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:00:54.321045  4577 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:54.280844  4532 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.211s
I20260709 13:00:54.320519  4554 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:54.602087  1541 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 4.
I20260709 13:00:54.424324  4528 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.153s
I20260709 13:00:54.347560  4504 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.324450  4411 raft_consensus.cc:3055] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:54.560349  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 325618us
I20260709 13:00:54.618688  4411 raft_consensus.cc:740] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:54.311396  4320 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.619692  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:54.620019  4320 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.621193  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:54.622296  4320 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:54.575780  1583 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:54.622982  4668 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:00:54.623446  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47745.5us
I20260709 13:00:54.624425  1583 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:54.496629  4555 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:54.344215  4640 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:54.629062  4640 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.631352  4640 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:54.536654  4514 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.632611  4670 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:54.633124  4670 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.633507  4670 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.637756  4661 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:54.429477  4654 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:54.639832  4654 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:54.642907  4504 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:54.643663  1612 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:54.645016  4632 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.529s	user 0.008s	sys 0.001s
I20260709 13:00:54.645794  1683 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:54.589023  1691 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:00:54.535495  4639 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:54.281826  4634 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:54.495584  4655 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.373s	user 0.001s	sys 0.000s
I20260709 13:00:54.593825  1686 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:00:54.602705  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:54.593675  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 257449us
I20260709 13:00:54.648376  1543 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:54.326035  4641 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:54.561117  4651 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:54.605670  1616 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 3. Current term is 4. Ops: []
I20260709 13:00:54.424953  4566 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:54.614624  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 122808us
I20260709 13:00:54.615406  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.670019  1685 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:54.609761  4577 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.616940  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:54.618109  4646 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:54.623502  4668 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.648717  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.648317  1583 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:54.559648  4434 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.654345  2229 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
W20260709 13:00:54.600950  4562 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.799s	user 0.010s	sys 0.001s
I20260709 13:00:54.663451  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.677170  1692 raft_consensus.cc:2410] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 13:00:54.660199  1507 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.907s (SENT) [suppressed 64 similar messages]
W20260709 13:00:54.693621  4201 consensus_meta.cc:287] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.000s
I20260709 13:00:54.773833  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:54.747632  1537 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:54.748344  1543 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:00:54.697358  4646 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:54.696230  4651 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.668849  4653 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:54.690194  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:54.785755  4554 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:54.685396  4668 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:54.749369  1614 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:54.793562  4639 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.795157  4639 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:54.796274  4669 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.777046  4654 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.779592  4562 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.798591  4654 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:54.799177  4651 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:54.799619  1538 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:54.800408  4646 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:54.800709  4654 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:54.589880  4631 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:54.754957  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:54.655676  4641 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:00:54.765131  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 144016us
I20260709 13:00:54.682672  4566 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.772383  1541 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:54.679451  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:54.619725  4411 consensus_queue.cc:260] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:54.781036  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 111116us
I20260709 13:00:54.783600  4369 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 13:00:54.784598  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 136381us
I20260709 13:00:54.930186  1583 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:54.933319  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:54.784374  4573 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:54.785145  4320 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:00:54.765169  1584 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.005s	sys 0.015s
W20260709 13:00:54.943733  1584 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.005s	sys 0.015s
I20260709 13:00:54.875569  1683 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:00:54.815760  1684 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:54.948009  1691 raft_consensus.cc:2368] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9.
I20260709 13:00:54.877411  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:54.752315  4657 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:54.878623  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:54.958542  1543 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:54.801239  4562 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:54.962073  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 84793.5us
I20260709 13:00:54.969018  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:54.969779  1543 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:54.969017  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:54.974215  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:54.864238  4201 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.975071  1613 raft_consensus.cc:2393] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:54.781580  4577 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:54.975836  4651 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:54.975730  4641 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.976857  4641 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:54.977216  4577 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:54.977480  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:54.878923  4554 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:54.877498  4499 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.069s	user 0.017s	sys 0.000s
I20260709 13:00:54.965732  4320 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:54.788652  4434 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.981750  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:54.998023  1615 raft_consensus.cc:2368] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9.
I20260709 13:00:54.982860  1537 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:54.928668  1582 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:54.929975  4632 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:54.785878  4653 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.172s
W20260709 13:00:54.985220  1508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.000s	sys 0.016s
W20260709 13:00:54.985948  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 212335us
I20260709 13:00:54.801518  4514 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.181s
I20260709 13:00:54.936987  4669 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:54.988740  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 55529.9us
I20260709 13:00:55.004866  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:55.006002  1508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.001s	sys 0.016s
I20260709 13:00:54.990304  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:54.888456  2232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.003s	sys 0.004s
W20260709 13:00:55.009332  2232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.004s	sys 0.005s
I20260709 13:00:54.777940  4558 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:54.999809  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 71266us
I20260709 13:00:54.965461  4635 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:54.996636  4661 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:55.001577  4662 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:55.002197  4674 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:00:54.969095  4654 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:55.008100  1584 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:55.015190  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:55.025633  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:55.073316  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 68707.4us
I20260709 13:00:55.074550  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:55.002352  4632 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.803828  4670 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:55.098338  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 83208.6us
I20260709 13:00:55.103955  2203 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:55.121371  4457 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:55.121677  4667 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.362s	user 0.001s	sys 0.000s
I20260709 13:00:55.122076  4457 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.122611  4320 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.124281  4457 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:55.109171  4640 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:55.026414  4562 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.125588  4528 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:00:55.126921  4528 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:55.127210  4558 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.127365  4528 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:55.129315  4201 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.126955  4635 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:55.130443  4635 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.130964  4635 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:00:55.132625  4130 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.288s	user 0.013s	sys 0.000s
I20260709 13:00:55.133054  4130 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.136040  4130 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:55.137859  4555 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.512s	user 0.005s	sys 0.006s
I20260709 13:00:55.138345  4555 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:55.128832  4528 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.142199  4677 raft_consensus.cc:2764] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:00:55.142985  4671 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.177s
I20260709 13:00:55.143134  4528 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:55.145138  4675 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:00:55.146286  4532 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:55.146746  4532 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.102301  1658 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:55.147125  4532 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:55.147678  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45502.4us
I20260709 13:00:55.148422  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:55.125204  4640 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.149753  4505 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:55.149621  4654 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:55.150453  4505 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:55.151031  4654 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:55.151299  4505 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:55.151903  4634 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:55.152069  4654 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.150228  4640 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.043066  4662 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.024832  4659 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:54.987294  4554 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.073316  4648 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:54.996920  4499 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:54.984792  4681 consensus_queue.cc:1059] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.141s
I20260709 13:00:55.172344  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:55.024897  4669 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:55.107904  4559 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.000s	sys 0.012s
W20260709 13:00:55.179482  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 518622us
W20260709 13:00:55.180684  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.102058  4673 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:55.164196  4201 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.167702  4514 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:55.179026  4522 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:00:55.203580  1689 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:55.203359  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:55.206199  2237 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:55.206418  2236 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:55.205247  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:55.208067  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:55.208959  2232 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.171921  4662 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:55.125741  4562 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.191304  1658 leader_election.cc:400] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9.
W20260709 13:00:55.195993  4559 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.000s	sys 0.013s
I20260709 13:00:55.219198  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:55.229501  1690 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:55.212720  4514 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.195475  4677 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:55.195775  4524 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.202903  4506 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:00:55.249468  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 44332.8us
I20260709 13:00:55.247500  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:55.254973  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:55.256582  1507 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:55.258879  1618 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:00:55.258981  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:55.261108  4653 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:55.244223  4678 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.262087  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:55.261940  4678 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.262885  4653 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:55.264058  1507 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:55.245651  4594 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:55.257560  4681 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:55.265311  4653 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:55.265969  4594 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.267361  1507 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:55.267627  4670 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.268164  4653 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:55.268425  4594 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.269600  4653 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:55.270413  4632 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:55.271374  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:55.271960  4434 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.245513  4201 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.274796  4201 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:55.275136  4434 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:55.258692  4524 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.278662  4562 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:55.278733  4524 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:55.280827  4514 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.281747  4554 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:55.221331  4628 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.282768  4434 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:55.282624  4628 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.283481  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 931) took 1177 ms. Trace:
I20260709 13:00:55.283864  2229 rpcz_store.cc:276] 0709 13:00:54.106402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.106784 (+   382us) service_pool.cc:224] Handling call
0709 13:00:55.283282 (+1176498us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b54005c0dd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:55.283394 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1228868480}
I20260709 13:00:55.284731  4628 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:55.284993  1507 leader_election.cc:400] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9.
I20260709 13:00:55.276075  4130 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:55.218837  4568 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:55.253866  4506 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:55.285595  1690 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:55.285835  1507 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:55.303938  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:55.305397  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:55.286469  4568 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:55.306140  1688 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:00:55.265321  4669 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:55.302095  4505 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.287385  4573 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:55.307817  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 384561us
I20260709 13:00:55.340080  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:55.340929  1686 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:00:55.343638  1509 connection.cc:256] RPC call timeout handler was delayed by -1.0863702390000185s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1531s for a response
I20260709 13:00:55.264111  4678 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.314894  2232 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
W20260709 13:00:55.315429  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 373248us
I20260709 13:00:55.362210  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:55.363588  1657 leader_election.cc:336] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.642s (SENT)
I20260709 13:00:55.333248  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:55.335688  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:55.285400  4532 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.293030  4668 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.301249  4635 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.376993  1691 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:55.380674  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:55.378748  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:55.392052  1686 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:55.368716  4661 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:55.385775  4685 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:55.334877  4434 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.289462  4506 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.365038  4669 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.399777  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:55.376022  4573 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:55.405961  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:55.388530  4640 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.301574  4646 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:55.400372  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 392363us
I20260709 13:00:55.408535  1507 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:55.408375  1584 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:55.397711  4514 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:55.400270  4685 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.403745  4499 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.425864  2230 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:55.432366  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2449) took 1202 ms. Trace:
I20260709 13:00:55.411116  4573 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:55.408828  4517 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:55.420043  4687 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:55.415746  2228 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:55.418499  4506 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.426671  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:55.419775  4504 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:55.433748  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:55.420431  4673 raft_consensus.cc:697] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:55.425604  4684 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.436488  1617 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:00:55.437049  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:55.408674  4434 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.438094  1686 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:55.420218  4685 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.409554  4669 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.444427  1609 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:00:55.439213  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:55.412739  4568 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.441951  1692 raft_consensus.cc:2393] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:55.446465  4517 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:00:55.441911  4693 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.447252  4673 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.448254  4684 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.455974  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1881) took 1448 ms. Trace:
I20260709 13:00:55.460903  1537 rpcz_store.cc:276] 0709 13:00:54.007269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.889051 (+881782us) service_pool.cc:224] Handling call
0709 13:00:54.958398 (+ 69347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:54.999067 (+ 40669us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.455739 (+456672us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b54001e4454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:55.455911 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55791872}
I20260709 13:00:55.443321  1615 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:55.434653  4573 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:55.458345  1540 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:55.437180  4687 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.462308  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:55.464210  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:55.443442  4504 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:55.478060  4562 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:55.482628  4668 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.481877  4568 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.463838  4693 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.462338  4687 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.444196  4661 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.492628  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:55.493537  1540 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:55.482352  4562 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.483537  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:55.479506  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 575) took 1334 ms. Trace:
I20260709 13:00:55.457541  4514 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.479537  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:55.502825  1537 raft_consensus.cc:2393] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:00:55.498659  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.504501  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:55.497728  1541 rpcz_store.cc:276] 0709 13:00:54.145098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.925870 (+780772us) service_pool.cc:224] Handling call
0709 13:00:54.926390 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:55.172676 (+246286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.172706 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:55.175437 (+  2731us) log.cc:844] Serialized 26 byte log entry
0709 13:00:55.391022 (+215585us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:55.391062 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.391132 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:55.479269 (+ 88137us) raft_consensus.cc:1685] finished
0709 13:00:55.479290 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.479443 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58561,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":9012,"thread_start_us":421137,"threads_started":2,"wal-append.queue_time_us":241217}
I20260709 13:00:55.499811  4668 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:55.500874  4640 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:55.499833  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:55.510195  2237 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:55.513763  2229 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:00:55.514168  2229 pending_rounds.cc:85] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 6
I20260709 13:00:55.514393  2229 pending_rounds.cc:107] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:00:55.514634  2229 raft_consensus.cc:2889] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:55.523623  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:55.508226  4695 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.525028  4635 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:55.524856  4695 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.525621  4554 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:55.526044  4554 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.526407  4554 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.527395  4695 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:55.508386  4668 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:55.528322  4668 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:55.528656  4626 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.769s	user 0.002s	sys 0.016s
I20260709 13:00:55.541766  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:55.542506  1540 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:55.545727  4504 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.547047  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:55.550688  4677 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.551265  2229 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:55.551471  4677 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.553565  4654 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.550748  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:55.554121  4654 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.508891  4645 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.565461  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:55.570142  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2439) took 2167 ms. Trace:
I20260709 13:00:55.570204  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:55.495757  4499 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:00:55.576196  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:55.570602  1542 rpcz_store.cc:276] 0709 13:00:53.402995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.403128 (+   133us) service_pool.cc:224] Handling call
0709 13:00:53.416796 (+ 13668us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:53.417122 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:53.417151 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:53.422430 (+  5279us) log.cc:844] Serialized 26 byte log entry
0709 13:00:53.516948 (+ 94518us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:53.517003 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:53.517094 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:55.449126 (+1932032us) spinlock_profiling.cc:243] Waited 9.36 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:00:55.549911 (+100785us) raft_consensus.cc:1685] finished
0709 13:00:55.549935 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.570073 (+ 20138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3646,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":13413,"spinlock_wait_cycles":21530496,"thread_start_us":69285,"threads_started":1,"wal-append.queue_time_us":306208}
I20260709 13:00:55.579396  1535 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:55.593786  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:55.590150  1508 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:55.589843  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:55.604529  2229 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.599680  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 504) took 1361 ms. Trace:
I20260709 13:00:55.605315  1612 rpcz_store.cc:276] 0709 13:00:54.238254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.238363 (+   109us) service_pool.cc:224] Handling call
0709 13:00:54.251374 (+ 13011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:54.644048 (+392674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.599324 (+955276us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b540008c5d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:55.599583 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":901693184}
I20260709 13:00:55.583259  1507 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:55.582756  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:55.609274  1609 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:00:55.515532  4514 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.600219  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:55.627210  1542 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:55.571484  4532 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:55.621923  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:55.632474  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:55.633116  1539 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.604573  1540 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:55.598450  4654 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:55.601595  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 591116us
I20260709 13:00:55.637269  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:55.634794  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:55.637881  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:55.637976  2237 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:00:55.641496  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:55.640733  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:55.643929  1542 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:55.620783  4653 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:55.621376  4201 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.601220  4434 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:55.648285  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:55.660964  1612 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:55.619916  2229 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:00:55.623124  4681 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:00:55.653739  4677 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.620795  4696 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.670030  4654 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:55.670495  4696 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.670635  4654 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.672969  4696 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:55.658208  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:55.673904  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.662676  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:55.663961  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2450) took 1430 ms. Trace:
I20260709 13:00:55.595136  4517 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:00:55.667421  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:55.669672  4681 raft_consensus.cc:3055] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:00:55.682606  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:55.683516  1540 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:55.677091  1542 rpcz_store.cc:276] 0709 13:00:54.233549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.659960 (+1426411us) service_pool.cc:224] Handling call
0709 13:00:55.660388 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.663369 (+  2981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.663412 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:55.663472 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:55.663648 (+   176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.663729 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.663881 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:55.690153  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:55.678325  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 95187.8us
I20260709 13:00:55.673329  4654 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.569748  4645 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.623087  4640 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:00:55.667994  4130 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:55.674181  4201 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.709846  4640 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:00:55.714003  4699 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:00:55.714620  4528 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:55.733727  1539 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:55.432932  1543 rpcz_store.cc:276] 0709 13:00:54.230172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.970687 (+740515us) service_pool.cc:224] Handling call
0709 13:00:54.970974 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:54.971202 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:55.306669 (+335467us) log.cc:844] Serialized 14 byte log entry
0709 13:00:55.431775 (+125106us) log.cc:844] Serialized 14 byte log entry
0709 13:00:55.432010 (+   235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:55.432053 (+    43us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:55.432114 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.432175 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.432282 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":430384,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":2201,"raft.queue_time_us":147,"spinlock_wait_cycles":1505024,"thread_start_us":435225,"threads_started":2}
I20260709 13:00:55.721076  1508 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:55.682915  4681 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:55.735338  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2452) took 1499 ms. Trace:
I20260709 13:00:55.737629  1539 rpcz_store.cc:276] 0709 13:00:54.236254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.734415 (+1498161us) service_pool.cc:224] Handling call
0709 13:00:55.734727 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.734905 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.734942 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:55.734994 (+    52us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:55.735050 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.735127 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.735264 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:55.666640  4641 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:00:55.746356  1543 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:00:55.747251  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1884) took 1140 ms. Trace:
I20260709 13:00:55.747648  1543 rpcz_store.cc:276] 0709 13:00:54.606606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.745456 (+1138850us) service_pool.cc:224] Handling call
0709 13:00:55.745881 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.746893 (+  1012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.747178 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:55.719311  4654 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:55.741381  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:55.722404  4550 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:55.775782  4535 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.707s	user 0.016s	sys 0.010s
I20260709 13:00:55.731343  4640 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:55.745046  4528 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.787827  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:55.787398  4640 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.789100  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:55.760375  4568 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:55.737208  4661 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.785511  1539 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:00:55.742914  1540 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:00:55.779978  2229 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:55.791744  1692 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:55.793412  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1885) took 1186 ms. Trace:
I20260709 13:00:55.793977  1539 rpcz_store.cc:276] 0709 13:00:54.606972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.743916 (+1136944us) service_pool.cc:224] Handling call
0709 13:00:55.785242 (+ 41326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:55.793042 (+  7800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.793342 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:00:55.789567  1612 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:55.795362  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:55.788602  4528 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:55.804198  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:55.805491  2231 raft_consensus.cc:2410] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:55.806589  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:00:55.809118  4702 log.cc:927] Time spent T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:00:55.814687  4201 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.812162  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:55.820324  4685 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.821246  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:55.731590  4554 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.824139  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:55.786580  4535 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.726s	user 0.017s	sys 0.011s
I20260709 13:00:55.758471  4654 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:55.785465  4514 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.840027  4434 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:55.733085  4662 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.788918  4550 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.852805  1537 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:55.854938  1611 raft_consensus.cc:2368] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6.
I20260709 13:00:55.860766  1685 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:55.862646  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:55.863322  1692 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:55.830305  4681 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.830127  4709 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:55.866978  4709 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.843621  4640 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:00:55.790161  4668 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.883232  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1874) took 1891 ms. Trace:
I20260709 13:00:55.883971  1543 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:00:55.883699  1538 rpcz_store.cc:276] 0709 13:00:53.991775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.994870 (+  3095us) service_pool.cc:224] Handling call
0709 13:00:53.995159 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:54.800142 (+804983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.882969 (+1082827us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b54000ed8d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:55.883167 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1849674496}
I20260709 13:00:55.898874  4669 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.902442  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2451) took 1668 ms. Trace:
I20260709 13:00:55.902959  4554 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:55.904138  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.902987  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:55.906437  2237 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:00:55.904577  4550 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.905344  4411 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:55.906730  1542 rpcz_store.cc:276] 0709 13:00:54.233968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.690556 (+1456588us) service_pool.cc:224] Handling call
0709 13:00:55.706574 (+ 16018us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.706836 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:55.707189 (+   353us) log.cc:844] Serialized 14 byte log entry
0709 13:00:55.877120 (+169931us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:55.877202 (+    82us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:55.877273 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.877359 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:55.902363 (+ 25004us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6824,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":621,"thread_start_us":168885,"threads_started":1}
I20260709 13:00:55.922941  1535 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:55.801206  4661 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:55.832230  4687 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:55.993968  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:55.997967  1583 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:56.000211  1538 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:55.926021  1542 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:55.987434  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:55.987434  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:55.997851  1615 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
W20260709 13:00:56.002162  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.002736  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.003961  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:55.979489  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 580) took 1699 ms. Trace:
W20260709 13:00:55.996678  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 172606us
I20260709 13:00:56.004525  1689 rpcz_store.cc:276] 0709 13:00:54.280044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.280161 (+   117us) service_pool.cc:224] Handling call
0709 13:00:54.309869 (+ 29708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.213226 (+903357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.979160 (+765934us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b54002cd154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:55.979406 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2054868736}
I20260709 13:00:56.005795  1508 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:56.008984  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:56.009819  1615 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:56.009811  1692 raft_consensus.cc:2410] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:00:56.010601  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.012290  1538 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:56.014067  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:56.014158  1540 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:56.014675  1615 raft_consensus.cc:2376] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8. Re-sending same reply.
I20260709 13:00:56.015599  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:56.013526  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:56.016939  1689 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
W20260709 13:00:56.019366  1683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1405) took 1472 ms (client timeout 1842 ms). Trace:
W20260709 13:00:56.019766  1683 rpcz_store.cc:269] 0709 13:00:54.546767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.572866 (+ 26099us) service_pool.cc:224] Handling call
0709 13:00:56.019179 (+1446313us) spinlock_profiling.cc:243] Waited 72 ms on lock 0x7b540003fa54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:56.019284 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165567744}
I20260709 13:00:56.020871  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:55.993717  4554 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.021703  1683 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:56.021734  4554 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.022723  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:56.023015  4685 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.024031  4554 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:56.026751  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.017741  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:56.035380  1689 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.044073  4684 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:55.999102  4709 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:55.925446  4654 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:55.924804  4457 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:56.045172  4704 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.261s	user 0.000s	sys 0.002s
I20260709 13:00:56.065866  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:55.960786  4573 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:56.140152  1683 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:56.012184  4669 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:00:56.145133  1614 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1426) took 1904 ms (client timeout 1682 ms). Trace:
W20260709 13:00:56.186175  1614 rpcz_store.cc:269] 0709 13:00:54.240962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.241823 (+   861us) service_pool.cc:224] Handling call
0709 13:00:56.144869 (+1903046us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400102554. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:56.145042 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1148616064}
I20260709 13:00:56.025789  4504 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:56.184736  1692 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:00:56.095965  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:56.187158  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:56.193905  1615 raft_consensus.cc:2376] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8. Re-sending same reply.
I20260709 13:00:56.194238  1612 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:56.194941  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:56.195519  1615 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.180212  1610 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
W20260709 13:00:56.175853  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 162679us
I20260709 13:00:56.198135  4670 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:56.199163  4661 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:56.200521  4687 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.200716  4670 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:56.201057  4457 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.201824  4549 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:56.202045  4457 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:56.202260  4549 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:56.203161  4549 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.206465  1542 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:00:56.207197  1583 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:56.208570  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:56.209213  1614 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.096263  4662 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.214802  1583 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:56.216864  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:56.217453  1612 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:00:56.219753  2231 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:00:56.220152  2231 pending_rounds.cc:85] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 5
I20260709 13:00:56.220371  2231 pending_rounds.cc:107] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:00:56.220470  1689 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:56.220609  2231 raft_consensus.cc:2889] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:56.210729  4592 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:56.222049  4592 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:56.222462  4592 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:00:56.222785  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.222808  4592 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:56.226399  1612 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:56.227624  1612 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:56.228768  1612 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:56.229653  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:56.233610  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.234934  1615 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:56.235054  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:56.236057  1683 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:56.234727  4640 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.238989  4640 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.240306  4646 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.241039  4640 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.245507  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:56.246425  1683 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:00:56.246678  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:56.247535  1689 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:56.247646  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:56.235481  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.246723  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:56.249657  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:56.252657  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:56.253584  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:56.254745  1615 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:56.256412  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:56.257040  1538 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:56.258724  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.259877  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.259843  1582 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:56.264333  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.264753  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:56.318718  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.319180  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.320369  1614 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:56.252228  4668 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:56.326390  1614 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:00:56.326543  2237 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
W20260709 13:00:56.320668  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 55998us
I20260709 13:00:56.320632  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:56.254308  4646 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:56.334020  2232 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:00:56.333722  1538 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
W20260709 13:00:56.334224  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 70216.7us
I20260709 13:00:56.333964  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:56.266294  4661 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:56.334342  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 90318.7us
I20260709 13:00:56.336508  1538 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:56.284144  4504 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:56.323820  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:56.343673  4657 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:00:56.344583  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 577) took 1013 ms. Trace:
I20260709 13:00:56.348706  4592 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.338927  4481 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:56.320969  4640 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:56.351382  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2456) took 1289 ms. Trace:
I20260709 13:00:56.354048  1509 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:56.355275  1543 rpcz_store.cc:276] 0709 13:00:55.061856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.780335 (+718479us) service_pool.cc:224] Handling call
0709 13:00:55.780649 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.884511 (+103862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.351152 (+466641us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b540030a7d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:56.351324 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236488064}
I20260709 13:00:56.321238  4652 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:56.352581  1539 rpcz_store.cc:276] 0709 13:00:55.330955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.801978 (+471023us) service_pool.cc:224] Handling call
0709 13:00:55.802301 (+   323us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:55.833329 (+ 31028us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.833357 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:55.852611 (+ 19254us) log.cc:844] Serialized 52 byte log entry
0709 13:00:56.174645 (+322034us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:56.174694 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.174774 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:56.344379 (+169605us) raft_consensus.cc:1685] finished
0709 13:00:56.344400 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.344529 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":411306,"raft.queue_time_us":31199,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":395044,"thread_start_us":150724,"threads_started":1,"wal-append.queue_time_us":342155}
I20260709 13:00:56.334880  4646 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.357182  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:56.356528  2228 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:56.361742  1612 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 4. Current term is 5. Ops: []
I20260709 13:00:56.362147  2232 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:56.362310  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.362844  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.363809  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.365182  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:56.365761  2237 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:56.363775  4646 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.353024  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:56.369973  1683 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:56.370045  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.371371  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.374341  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:56.378196  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:56.380934  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:56.359135  4668 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:56.340405  4535 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 13:00:56.340392  4644 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:00:56.391062  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:56.368954  4652 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.394865  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:56.390240  4720 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 13:00:56.388166  4530 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:56.387758  4130 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:56.391811  4653 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:56.369344  4640 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:00:56.401279  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:00:56.402503  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.386432  4573 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.397732  4550 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:56.406453  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:56.407236  4644 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:56.410002  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:56.413764  1683 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:56.416438  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:56.409874  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:56.403035  4130 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:56.425558  4550 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:56.426201  4550 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:56.427065  4550 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:56.418817  4722 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:00:56.429222  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1890) took 1033 ms. Trace:
I20260709 13:00:56.416952  4573 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:56.430074  4573 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:56.429610  1535 rpcz_store.cc:276] 0709 13:00:55.395929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.923915 (+527986us) service_pool.cc:224] Handling call
0709 13:00:55.924239 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:55.969472 (+ 45233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.969518 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:55.975445 (+  5927us) log.cc:844] Serialized 26 byte log entry
0709 13:00:56.067543 (+ 92098us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:00:56.067582 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.067662 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:56.429023 (+361361us) raft_consensus.cc:1685] finished
0709 13:00:56.429042 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.429167 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":258846,"raft.queue_time_us":38107,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":268251,"thread_start_us":90366,"threads_started":1,"wal-append.queue_time_us":241947}
I20260709 13:00:56.431134  4573 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:56.427382  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2458) took 1365 ms. Trace:
W20260709 13:00:56.432593  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 76508us
I20260709 13:00:56.432864  1540 rpcz_store.cc:276] 0709 13:00:55.062457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.794810 (+732353us) service_pool.cc:224] Handling call
0709 13:00:55.834247 (+ 39437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:56.014683 (+180436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.427048 (+412365us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b5400419bd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:56.427300 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":413347072}
I20260709 13:00:56.434497  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:56.398454  4652 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.415359  4644 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:56.436167  1692 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:00:56.435961  1582 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:56.438870  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:56.441542  1692 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:00:56.433744  4130 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:56.415776  4457 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:56.442111  1692 pending_rounds.cc:85] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 5
I20260709 13:00:56.440917  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:56.443899  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:56.444485  1692 pending_rounds.cc:107] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:00:56.444756  1692 raft_consensus.cc:2889] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:56.445279  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.445647  1683 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:00:56.446110  1683 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:56.446732  1692 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 13:00:56.447407  1683 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:56.449855  1683 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:56.449834  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1419) took 1134 ms. Trace:
I20260709 13:00:56.450702  2232 raft_consensus.cc:2376] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9. Re-sending same reply.
I20260709 13:00:56.451371  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:56.450433  1686 rpcz_store.cc:276] 0709 13:00:55.315548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.465373 (+149825us) service_pool.cc:224] Handling call
0709 13:00:55.495604 (+ 30231us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:00:55.495987 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.496024 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:00:55.508598 (+ 12574us) log.cc:844] Serialized 52 byte log entry
0709 13:00:55.907380 (+398782us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b18001f81c0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:00:56.192283 (+284903us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:00:56.192345 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.192449 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:56.449578 (+257129us) raft_consensus.cc:1685] finished
0709 13:00:56.449597 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.449779 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2446,"raft.queue_time_us":11134,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":6150,"spinlock_wait_cycles":8576256,"thread_start_us":284115,"threads_started":1,"wal-append.queue_time_us":694078}
I20260709 13:00:56.451717  4535 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:56.452525  4535 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:56.452730  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:56.453459  4535 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.454006  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:56.455216  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:56.437460  4678 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:56.440182  4670 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:56.459254  4411 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:56.459621  4670 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:00:56.460585  4683 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.061s	user 0.004s	sys 0.000s
I20260709 13:00:56.460649  4670 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:56.461894  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 574) took 2340 ms. Trace:
I20260709 13:00:56.462247  1536 rpcz_store.cc:276] 0709 13:00:54.121734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.786654 (+664920us) service_pool.cc:224] Handling call
0709 13:00:54.786987 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:54.787398 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:54.787424 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:54.787942 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 13:00:55.220756 (+432814us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:55.220797 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:55.220862 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:55.489965 (+269103us) spinlock_profiling.cc:243] Waited 2.14 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:00:56.460274 (+970309us) spinlock_profiling.cc:243] Waited 4.03 ms on lock 0x7b2400175a48. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c8859c4 00007fe08c883acb 00007fe08ca3edc9 00007fe08ca3ea55 00007fe08ca3e09d 00007fe08f67010e 00007fe08f68788c 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221
0709 13:00:56.461682 (+  1408us) raft_consensus.cc:1685] finished
0709 13:00:56.461702 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.461838 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9876,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":103,"spinlock_wait_cycles":14190208,"thread_start_us":432389,"threads_started":1,"wal-append.queue_time_us":553399}
I20260709 13:00:56.463768  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:56.464592  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2462) took 1033 ms. Trace:
I20260709 13:00:56.464998  1542 rpcz_store.cc:276] 0709 13:00:55.431526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.001788 (+570262us) service_pool.cc:224] Handling call
0709 13:00:56.013917 (+ 12129us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:56.206924 (+193007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.464344 (+257420us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54000adf54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:56.464535 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442282368}
I20260709 13:00:56.439294  4642 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:00:56.469645  4678 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:56.470708  4678 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:56.483482  2237 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:56.486776  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:56.493788  2235 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:56.494287  4695 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.494805  4695 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.495790  4644 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.496873  4644 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.498703  4457 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.458403  4558 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
W20260709 13:00:56.502622  4689 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.171s	user 0.003s	sys 0.000s
I20260709 13:00:56.456367  4369 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:00:56.507999  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:56.446290  4669 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 13:00:56.446132  4709 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 13:00:56.491128  4411 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.517014  4411 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:56.518683  4631 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:56.518983  4411 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.521829  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:56.522457  4645 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.523316  4693 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:56.524564  1690 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1412) took 1304 ms (client timeout 1684 ms). Trace:
W20260709 13:00:56.524997  1690 rpcz_store.cc:269] 0709 13:00:55.219733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.247193 (+ 27460us) service_pool.cc:224] Handling call
0709 13:00:56.524384 (+1277191us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b5400374954. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:56.524501 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70748416}
I20260709 13:00:56.526089  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:56.527191  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:56.527904  1690 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:56.531293  4481 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:56.531894  4481 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:56.532406  1507 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:56.533646  1507 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:56.534987  1507 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:56.536095  4653 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:00:56.536715  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1922) took 1100 ms. Trace:
I20260709 13:00:56.536921  4687 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:56.537261  1609 rpcz_store.cc:276] 0709 13:00:55.436243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.479555 (+ 43312us) service_pool.cc:224] Handling call
0709 13:00:56.536498 (+1056943us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400190854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:56.536640 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":235270400}
I20260709 13:00:56.536774  4653 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:56.539176  4653 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:56.539638  4653 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:56.541749  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:56.542923  1688 raft_consensus.cc:2410] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 9 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:00:56.543098  1507 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:56.544585  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:56.549194  1536 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:00:56.548588  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:56.547631  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:56.553589  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:56.554857  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:56.544284  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:00:56.594767  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 50542.5us
I20260709 13:00:56.596081  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:56.614760  2228 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
W20260709 13:00:56.617975  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.493368  4592 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:56.623925  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:56.624379  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 70959.2us
I20260709 13:00:56.625468  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:56.627888  4530 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:00:56.630029  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:56.630681  2237 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:56.631196  4668 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:00:56.533074  4481 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.632673  4481 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:56.633838  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:56.642590  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:56.633208  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 572) took 2681 ms. Trace:
I20260709 13:00:56.637197  4668 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:00:56.623248  4709 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:56.633193  4658 raft_consensus.cc:3055] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:56.664067  4687 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.664623  4687 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:56.645753  1534 rpcz_store.cc:276] 0709 13:00:53.951609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:53.952797 (+  1188us) service_pool.cc:224] Handling call
0709 13:00:53.971161 (+ 18364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.708913 (+1737752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:55.708961 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:55.709020 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:55.709067 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.632967 (+923900us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b5400418cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:56.633015 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.633145 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3995767168}
I20260709 13:00:56.624042  4645 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:56.633675  4648 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:00:56.648144  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.670584  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 794) took 1084 ms. Trace:
I20260709 13:00:56.669234  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 502) took 2431 ms. Trace:
I20260709 13:00:56.672091  1616 rpcz_store.cc:276] 0709 13:00:54.237668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.237812 (+   144us) service_pool.cc:224] Handling call
0709 13:00:54.267160 (+ 29348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:54.656179 (+389019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.668989 (+2012810us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54000c7e54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:56.669173 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":777947392}
I20260709 13:00:56.673858  1614 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:56.670964  2231 rpcz_store.cc:276] 0709 13:00:55.586363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.807775 (+221412us) service_pool.cc:224] Handling call
0709 13:00:55.808191 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:56.221372 (+413181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.670342 (+448970us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b5400690ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:56.670527 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":945637248}
I20260709 13:00:56.682801  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.685793  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.648494  4530 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:56.630682  4646 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:56.633531  4726 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.625393  4554 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:56.700418  4554 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:56.700896  4554 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:56.704101  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:56.705459  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.706637  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.636016  4657 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:56.708252  4657 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.709350  4566 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.710356  4657 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:56.711215  4505 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:56.713140  4695 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.643474  4503 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:56.714120  4503 raft_consensus.cc:697] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:56.687474  1613 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [3.4-6.6]
I20260709 13:00:56.688818  1617 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:00:56.718466  1583 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:56.721285  4693 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:56.721835  4693 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.722227  4693 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:56.723443  4558 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:00:56.727061  4687 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.727505  4652 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.707645  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.732560  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.641182  1683 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 13:00:56.669566  4573 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:56.625905  4669 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:56.662464  4658 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:56.692627  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:56.675410  4645 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:56.695621  1609 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:00:56.667538  4709 raft_consensus.cc:697] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:56.666780  4681 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:56.699429  4726 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.735082  1683 pending_rounds.cc:85] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 5
I20260709 13:00:56.770308  1683 pending_rounds.cc:107] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 13:00:56.770665  1683 raft_consensus.cc:2889] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:00:56.735559  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.737773  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:56.754222  1612 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:00:56.753196  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 127869us
I20260709 13:00:56.772014  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:56.773164  1541 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:56.773258  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:56.774225  2203 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:56.775271  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:56.775893  1542 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:56.776639  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:56.778250  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.778513  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2463) took 1345 ms. Trace:
I20260709 13:00:56.778957  1537 rpcz_store.cc:276] 0709 13:00:55.432869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.569860 (+136991us) service_pool.cc:224] Handling call
0709 13:00:56.778289 (+1208429us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b540012cfd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:56.778415 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":645572480}
I20260709 13:00:56.779867  4687 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.780509  4687 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:56.783097  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.784536  1657 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:56.785877  1611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1918) took 1823 ms (client timeout 2259 ms). Trace:
W20260709 13:00:56.786273  1611 rpcz_store.cc:269] 0709 13:00:54.962144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.995539 (+ 33395us) service_pool.cc:224] Handling call
0709 13:00:56.785693 (+1790154us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b54003c87d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:56.785821 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1865769344}
I20260709 13:00:56.787429  4505 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:00:56.788929  4505 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:56.789387  4505 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:56.790256  4505 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.794178  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:56.794621  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2286) took 2553 ms. Trace:
I20260709 13:00:56.794384  1536 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:00:56.798023  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:56.799064  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 79183.5us
I20260709 13:00:56.802695  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:56.795156  1687 rpcz_store.cc:276] 0709 13:00:54.241142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.242551 (+  1409us) service_pool.cc:224] Handling call
0709 13:00:54.242926 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.561352 (+1318426us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:00:55.746986 (+185634us) log.cc:844] Serialized 14 byte log entry
0709 13:00:55.793156 (+ 46170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:55.793209 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:00:55.793261 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:56.794292 (+1001031us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540028f5d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:56.794357 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:56.794544 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":217440,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":98,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":17475,"spinlock_wait_cycles":3031498112,"thread_start_us":201282,"threads_started":2,"wal-append.queue_time_us":93447}
I20260709 13:00:56.755046  4645 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:56.836010  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:56.839311  2228 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:56.839607  1534 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: []
W20260709 13:00:56.811108  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 256344us
I20260709 13:00:56.817910  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:56.841728  2236 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 4.
I20260709 13:00:56.813506  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:56.817103  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:56.732560  4558 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:00:56.818006  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:56.815446  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:56.850041  1536 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:00:56.850797  1535 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:56.844775  1583 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:56.846977  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 204453us
I20260709 13:00:56.770666  4669 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:56.850703  1537 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:00:56.845500  1541 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:00:56.853283  1507 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:56.855942  1542 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:56.832875  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:56.862718  2229 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:00:56.862743  1539 raft_consensus.cc:2393] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:00:56.865914  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 81443us
I20260709 13:00:56.794708  4530 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:56.818257  4533 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:56.871023  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:56.814249  4657 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.811213  4535 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:56.783155  4687 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.871394  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:56.861650  4670 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:56.887504  4670 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.887935  4670 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:56.879032  2231 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:00:56.862871  4577 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:00:56.867291  4631 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:56.911815  4631 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:56.912940  4631 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.913812  4740 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:00:56.905668  4735 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:56.838799  4130 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.922487  4530 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:56.886488  4535 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:56.886111  4503 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.870868  4421 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:00:56.926496  4741 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:56.926218  4669 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.873175  4658 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.904338  4709 consensus_queue.cc:237] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:56.884835  4533 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:56.930011  4646 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.930855  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:56.934095  4726 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:56.936261  1687 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:56.937119  1688 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:00:56.937716  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:56.915467  4735 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:56.938396  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:56.939034  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:56.948702  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:56.950163  4457 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:00:56.969700  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 127643us
I20260709 13:00:56.971307  1540 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:00:56.895972  4657 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.039198  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:56.967093  4535 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:56.981372  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:57.043859  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.042701  1688 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:57.044631  1685 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:57.048182  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.053545  1683 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:56.959705  4740 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.086s
I20260709 13:00:57.055384  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:56.977809  4533 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:57.058990  1534 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:56.967641  4530 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:57.062325  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 195246us
I20260709 13:00:57.064280  1534 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:57.039585  4457 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:00:56.975759  4566 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
W20260709 13:00:57.055202  4730 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 13:00:57.072149  1684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1392) took 3003 ms (client timeout 1683 ms). Trace:
W20260709 13:00:57.072695  1684 rpcz_store.cc:269] 0709 13:00:54.065383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.065536 (+   153us) service_pool.cc:224] Handling call
0709 13:00:57.068415 (+3002879us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b540041e1d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:57.068544 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1234797952}
I20260709 13:00:57.074316  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:57.071708  4644 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.000s
I20260709 13:00:57.055469  4738 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:57.075650  4505 raft_consensus.cc:2764] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 13:00:57.090865  4530 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:57.091658  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:57.092723  1534 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:00:57.082392  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:56.981109  4741 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:57.128209  4734 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 13:00:57.130914  4654 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:00:57.090906  4703 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.422s	user 0.005s	sys 0.001s
I20260709 13:00:57.107475  4735 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:57.109542  4457 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:57.110296  2203 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:57.113163  4657 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:54.587415   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:00:57.113788  4747 consensus_queue.cc:1059] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:00:57.121636  4628 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:57.122298  4554 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:57.123543  4522 raft_consensus.cc:3055] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:57.174435  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:57.147598  2203 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:57.157771  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:57.163548  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 100460us
I20260709 13:00:57.172823  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:57.175185  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.123718  4130 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:57.186859  1692 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:57.188673  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:57.190442  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:57.193171  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:57.193360  1584 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:57.194011  2229 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:00:57.165764  4733 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.248s	user 0.002s	sys 0.000s
I20260709 13:00:57.195281  1508 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:57.196156  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 48655us
I20260709 13:00:57.194906  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:57.209512  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:57.165988  4653 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.626s	user 0.009s	sys 0.002s
I20260709 13:00:57.165949  4554 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:57.163389  4499 raft_consensus.cc:3055] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:00:57.204356  4743 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 13:00:57.157691  4750 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:57.190364  4509 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:00:57.206831  4645 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.101085  4738 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:57.246078  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 56577.9us
I20260709 13:00:57.209519  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:57.211294  1657 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:00:57.212944  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:57.247998  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.168480  4693 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.217204  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:57.163380  4457 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:57.221920  1538 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:57.157770  4628 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:57.164383  4562 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:57.185284  4522 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:57.249698  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.250034  1508 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:57.242933  4745 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.251467  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:57.214948  4735 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:57.253402  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.261059  1534 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:57.214417  4737 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:57.237715  4369 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:57.254941  1657 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:57.241408  4750 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.217973  4499 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:57.305083  4750 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.305704  4499 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.306910  4499 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:57.257804  1540 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:57.239188  4130 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:57.257241  1691 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:00:57.256035  4657 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:57.261272  1685 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:57.267650  4737 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.311971  4657 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:57.312438  4657 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.313936  4737 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:57.254245  4554 raft_consensus.cc:3055] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:00:57.314574  4554 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:57.315414  4554 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.316587  4554 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:57.237609  4505 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
W20260709 13:00:57.274814  4728 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.406s	user 0.001s	sys 0.000s
W20260709 13:00:57.296685  4739 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
W20260709 13:00:57.339619  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 92544.5us
I20260709 13:00:57.277164  4549 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.253118  4592 consensus_queue.cc:1059] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:00:57.285786  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.286314  4745 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.287258  4738 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:57.287771  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.289660  4693 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:57.289984  4481 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.657s	user 0.008s	sys 0.004s
W20260709 13:00:57.292685  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 252010us
I20260709 13:00:57.262558  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:57.237789  4653 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:57.296195  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 46247us
I20260709 13:00:57.243736  4555 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 13:00:57.276119  4680 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.854s	user 0.001s	sys 0.002s
I20260709 13:00:57.370419  4745 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:57.342113  1584 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:00:57.373061  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.297451  4669 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:57.262753  4509 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:57.264329  4735 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:57.349578  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.361198  4369 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:57.261292  4749 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 13:00:57.300043  4573 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:57.363660  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 108799us
I20260709 13:00:57.345047  4562 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:57.331596  4737 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.366598  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:57.301966  4661 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:00:57.374490  4752 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:57.374604  4738 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:57.240432  4503 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:57.339075  4741 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:57.365211  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 102752us
I20260709 13:00:57.409116  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:57.362118  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 69479.3us
I20260709 13:00:57.383353  1657 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:57.411662  1657 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:57.345235  4668 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:57.299544  4533 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:57.391760  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 49743.4us
I20260709 13:00:57.297151  4670 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.397099  4577 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:57.410609  1540 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.423254  2201 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:57.382953  4751 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:57.387475  4735 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:00:57.472144  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 48990us
I20260709 13:00:57.401525  4421 raft_consensus.cc:3055] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:57.400257  4752 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:57.397020  4709 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:57.414494  4738 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:57.402029  4737 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.473354  4668 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.477872  4659 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:57.508111  1610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1913) took 2547 ms (client timeout 2163 ms). Trace:
I20260709 13:00:57.511760  2201 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:57.401067  4320 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:00:57.537951  1610 rpcz_store.cc:269] 0709 13:00:54.960710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.964961 (+  4251us) service_pool.cc:224] Handling call
0709 13:00:57.480538 (+2515577us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54003499d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:57.508016 (+ 27478us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2748911872}
I20260709 13:00:57.513948  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 941) took 2316 ms. Trace:
W20260709 13:00:57.525218  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 70036.5us
I20260709 13:00:57.455626  4130 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.539294  2230 rpcz_store.cc:276] 0709 13:00:55.192044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:55.192245 (+   201us) service_pool.cc:224] Handling call
0709 13:00:55.192662 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:55.432547 (+239885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.480697 (+2048150us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54007404d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:57.507818 (+ 27121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":523431680}
I20260709 13:00:57.540019  1584 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:57.530717  1540 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:00:57.397020  4562 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:57.514780  4421 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:57.533454  4505 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:00:57.513156  4751 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.556097  4320 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:00:57.556671  4751 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.556845  4320 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:00:57.537633  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.548977  4761 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.557802  4320 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:57.558151  4761 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.521749  4752 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.560808  4761 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:57.556149  4661 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 13:00:57.553740  4505 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.563753  4505 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:57.567022  4562 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:57.568428  4533 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:57.551461  4509 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:57.569248  4509 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:57.569641  4509 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.571089  4657 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.573153  4533 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:00:57.449059  4749 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:00:57.527886  4766 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:00:57.555116  2201 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:57.471005  4750 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.550701  4592 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
W20260709 13:00:57.556005  4764 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:00:57.529484  4654 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
W20260709 13:00:57.577646  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 166052us
I20260709 13:00:57.463248  4530 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 13:00:57.512921  4577 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:57.583372  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:57.588218  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:57.547724  4566 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:00:57.594355  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 43825.3us
I20260709 13:00:57.540926  4745 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
W20260709 13:00:57.594228  4681 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.818s	user 0.011s	sys 0.000s
I20260709 13:00:57.601518  1534 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:57.598573  1657 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:57.608593  2201 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:57.610329  1540 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:57.551849  4130 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.612812  4737 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:57.614042  4653 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:57.614487  4130 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:57.615774  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:57.617550  4773 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:00:57.618322  4773 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:57.617542  4481 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:57.619374  4644 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:00:57.620895  2235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 938) took 3507 ms (client timeout 1659 ms). Trace:
W20260709 13:00:57.621333  2235 rpcz_store.cc:269] 0709 13:00:54.113167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:54.115222 (+  2055us) service_pool.cc:224] Handling call
0709 13:00:57.620675 (+3505453us) spinlock_profiling.cc:243] Waited 2.31 s on lock 0x7b54002e95d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:57.620792 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5307083136}
I20260709 13:00:57.621830  4657 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:57.623003  2235 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:00:57.624198  4768 raft_consensus.cc:3055] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:57.624624  4768 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:57.596328  4749 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:57.626890  4658 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.590382  4767 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:57.619282  4773 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.629606  2201 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:57.523784  4369 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.646087  2201 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:57.632968  4566 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:57.633875  4499 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:00:57.634567  4670 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:57.635236  1584 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:57.595084  4532 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:00:57.637041  1534 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
W20260709 13:00:57.640655  4535 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.585s	user 0.007s	sys 0.007s
I20260709 13:00:57.640455  4751 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.641726  4767 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.644248  4421 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:57.660533  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 801) took 1398 ms. Trace:
I20260709 13:00:57.664494  2201 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:57.654313  4745 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.626909  4750 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:57.669301  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:00:57.680732  2234 rpcz_store.cc:276] 0709 13:00:56.262171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.262290 (+   119us) service_pool.cc:224] Handling call
0709 13:00:57.167503 (+905213us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:00:57.167697 (+   194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:57.168072 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:57.168100 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:57.168514 (+   414us) log.cc:844] Serialized 26 byte log entry
0709 13:00:57.365004 (+196490us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:57.365048 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.365117 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:57.627450 (+262333us) raft_consensus.cc:1685] finished
0709 13:00:57.627471 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:57.660426 (+ 32955us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":578,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"spinlock_wait_cycles":6423552,"thread_start_us":196107,"threads_started":1,"wal-append.queue_time_us":345633}
I20260709 13:00:57.684154  2201 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:57.669895  4411 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:57.661733  4749 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:57.694339  2237 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:57.668219  4709 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.698545  4738 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:00:57.668077  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 290803us
I20260709 13:00:57.704218  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:57.708335  1534 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:00:57.680480  4532 consensus_queue.cc:1243] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 8.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:00:57.686694  4566 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:57.709074  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:57.696311  4657 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.694541  4411 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:57.673563  4666 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.009s	sys 0.008s
I20260709 13:00:57.726653  1612 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: [8.4-8.4]
W20260709 13:00:57.726850  4666 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.010s	sys 0.008s
I20260709 13:00:57.728646  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:57.633028  4768 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.729727  4505 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.730216  4768 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.732206  4577 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:57.696087  4670 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:00:57.678135  4535 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.716949  2234 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:57.716044  4774 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:57.717782  4592 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
W20260709 13:00:57.756335  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.694365  4745 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:57.708474  4750 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.717741  4566 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:57.722460  4751 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:57.707480  4709 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.717095  4421 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:57.725659  4411 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.806900  2229 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:00:57.715080  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:57.732965  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.732941  1612 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:00:57.808621  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:57.809122  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0775214490000167s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2259s for a response
W20260709 13:00:57.809566  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0485502640000277s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1521s for a response
W20260709 13:00:57.810277  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:57.743184  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:57.735324  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:57.742677  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:57.876487  1690 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:57.778813  4505 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:57.670699  4752 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.781136  1534 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:57.876487  1692 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:00:57.804261  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:00:57.779645  4457 consensus_meta.cc:287] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.000s
I20260709 13:00:57.897027  4554 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:57.899276  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.902732  1684 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:00:57.904838  1688 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:00:57.905217  1688 pending_rounds.cc:85] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 3
I20260709 13:00:57.905469  1688 pending_rounds.cc:107] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:00:57.906080  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.906682  1689 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:00:57.907503  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:57.907756  1684 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:00:57.908375  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.909073  1687 raft_consensus.cc:2393] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:57.909804  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.911082  1687 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:00:57.905742  1688 raft_consensus.cc:2889] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:57.913246  1683 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:00:57.913501  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.785621  4566 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:57.914527  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.908134  1684 pending_rounds.cc:85] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 3
I20260709 13:00:57.920049  1684 pending_rounds.cc:107] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:00:57.914162  1691 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 4.
I20260709 13:00:57.922000  1684 raft_consensus.cc:2889] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:00:57.926118  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:57.932484  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:57.932981  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:57.802937  4640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.008s	sys 0.051s
I20260709 13:00:57.934142  1687 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:57.895727  4681 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.937701  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:57.914552  4670 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:57.940399  4670 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.943897  4457 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:57.944453  4457 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:57.944588  4509 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.919855  4749 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.945848  4749 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.951241  4745 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:57.916211  4751 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.786132  4750 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.952374  4751 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.915072  4554 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:57.953843  4735 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.955439  4554 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:57.916361  4505 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.958328  4505 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:57.960484  4505 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:57.935015  4577 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.961712  4577 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.963991  4767 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:57.899086  4499 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:57.914593  4555 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:57.905725  4741 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:00:57.934135  4640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.008s	sys 0.052s
I20260709 13:00:57.937004  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:57.964644  4774 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:00:57.993453  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 54399.5us
I20260709 13:00:57.993342  4657 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:57.998108  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:57.995815  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:00:57.995141  4740 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.000s	sys 0.011s
I20260709 13:00:58.029163  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:57.995982  4505 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:00:57.980115  4681 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:57.982980  4499 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:57.988416  4555 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:57.986054  4767 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
W20260709 13:00:58.030649  4740 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.952s	user 0.000s	sys 0.011s
I20260709 13:00:58.035491  1687 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:57.989986  4741 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.081194  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:58.082710  4778 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:58.083351  4778 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:58.032173  4532 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.030566  4745 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:58.035418  4654 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:58.094233  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:58.085328  4411 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.099215  4741 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:58.100430  4788 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:00:58.101394  2236 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:00:58.102839  4509 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.103397  4651 raft_consensus.cc:3055] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:58.077811  4669 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:58.104393  4669 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.079741  4505 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:58.105552  4555 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:58.107157  4681 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.086570  4503 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.108952  4535 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:58.103842  4554 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:58.104871  4499 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:58.078729  4771 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:00:58.091508  4767 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:58.091401  4654 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:58.099215  4411 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.103860  4651 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:00:58.123292  4782 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.121s	user 0.000s	sys 0.002s
W20260709 13:00:58.120764  4785 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.155s	user 0.002s	sys 0.000s
I20260709 13:00:58.121030  4750 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:58.140061  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:58.095431  4532 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.141755  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 807) took 1006 ms. Trace:
I20260709 13:00:58.142107  4505 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.131069  4768 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:58.127944  4767 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:58.129374  4778 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:58.132280  4654 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:58.130724  4681 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.143044  4554 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:00:58.143752  4566 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.146575  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:58.122270  4503 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.227766  4503 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.161860  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:58.229234  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:58.230109  2229 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:58.230958  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:58.231593  2229 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:58.232992  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:58.167192  2234 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:00:58.137295  4752 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.234160  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:00:58.176776  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:58.235230  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:58.235874  2234 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
W20260709 13:00:58.235914  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 59377.6us
I20260709 13:00:58.145886  2232 rpcz_store.cc:276] 0709 13:00:57.135215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.135738 (+   523us) service_pool.cc:224] Handling call
0709 13:00:57.374792 (+239054us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:00:57.374986 (+   194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:57.375444 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:57.375472 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:57.737827 (+362355us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b2c000cc068. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:00:57.738383 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 13:00:57.992912 (+254529us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:57.992949 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.993034 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:58.141500 (+148466us) raft_consensus.cc:1685] finished
0709 13:00:58.141528 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:58.141694 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9052,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":9865,"spinlock_wait_cycles":7684224,"thread_start_us":254184,"threads_started":1,"wal-append.queue_time_us":340041}
I20260709 13:00:58.236822  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:58.237490  4735 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:58.238308  4735 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:58.238435  4768 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:00:58.238806  4735 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.184581  4566 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:58.239989  4573 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.240484  4573 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.240996  4532 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:58.241816  4767 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.242626  4573 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:58.185689  4759 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.794s	user 0.002s	sys 0.007s
I20260709 13:00:58.236845  1658 leader_election.cc:400] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 5. Current term is 6.
I20260709 13:00:58.188086  4645 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:58.192839  4522 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:58.195554  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.195850  4659 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:00:58.196552  4628 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.198240  4709 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.200017  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:58.264652  2237 raft_consensus.cc:2410] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:00:58.166880  4457 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:58.205526  4681 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:58.206797  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:58.225284  4651 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.239168  4768 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:00:58.184957  4771 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:00:58.243233  4767 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:58.186743  4786 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 13:00:58.249666  2234 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:00:58.182164  1690 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:58.250334  1582 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:58.250310  2229 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:58.256702  1684 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:00:58.257243  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 806) took 1175 ms. Trace:
I20260709 13:00:58.261116  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:58.267149  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:58.269306  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:58.295482  1683 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:58.288599  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:58.288390  1687 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:00:58.296204  1688 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
W20260709 13:00:58.297597  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.259383  4659 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:58.266806  4628 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.292618  2231 rpcz_store.cc:276] 0709 13:00:57.082164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.082331 (+   167us) service_pool.cc:224] Handling call
0709 13:00:57.087792 (+  5461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:57.103934 (+ 16142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:57.103982 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:57.158293 (+ 54311us) log.cc:844] Serialized 26 byte log entry
0709 13:00:57.223306 (+ 65013us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:57.223347 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.223418 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:58.185062 (+961644us) spinlock_profiling.cc:243] Waited 4.31 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:00:58.246377 (+ 61315us) raft_consensus.cc:1685] finished
0709 13:00:58.246400 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:58.257158 (+ 10758us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15652,"raft.queue_time_us":20643,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"spinlock_wait_cycles":9907968,"thread_start_us":18324,"threads_started":1,"wal-append.queue_time_us":158986}
I20260709 13:00:58.302557  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:00:58.305787  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55621.8us
I20260709 13:00:58.306452  1684 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:58.271148  4522 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:58.309067  1687 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:00:58.287823  4744 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:58.306656  4651 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:00:58.323844  4789 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.235s	user 0.001s	sys 0.001s
I20260709 13:00:58.335889  2229 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
W20260709 13:00:58.322928  4780 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.382s	user 0.003s	sys 0.000s
I20260709 13:00:58.313843  4558 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:00:58.341447  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:58.321476  4669 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.322408  4369 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:00:58.370569  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:58.340642  4659 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.452018  4659 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:58.323374  4631 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
W20260709 13:00:58.453751  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 83300.9us
I20260709 13:00:58.462589  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:58.470216  4558 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:00:58.519539  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 57044.6us
I20260709 13:00:58.355597  4668 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.462002  4661 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:00:58.331071  4744 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:58.546623  4780 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:00:58.348273  4670 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.200s
I20260709 13:00:58.453156  4771 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:58.517181  4760 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.550005  4693 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:58.581705  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 199559us
I20260709 13:00:58.471683  4504 consensus_queue.cc:1059] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 13:00:58.471977  4692 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.110s
W20260709 13:00:58.591265  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 70759.6us
I20260709 13:00:58.595090  1688 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:58.578711  4558 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:00:58.596098  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.585896  4735 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:58.575822  4654 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.607977  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2482) took 1012 ms. Trace:
I20260709 13:00:58.608426  1539 rpcz_store.cc:276] 0709 13:00:57.595285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.595482 (+   197us) service_pool.cc:224] Handling call
0709 13:00:57.595956 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:57.600227 (+  4271us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:00:57.618470 (+ 18243us) log.cc:844] Serialized 14 byte log entry
0709 13:00:57.749684 (+131214us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:57.780722 (+ 31038us) log.cc:844] Serialized 26 byte log entry
0709 13:00:57.780950 (+   228us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:00:57.803652 (+ 22702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:57.803736 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:58.209934 (+406198us) spinlock_profiling.cc:243] Waited 2.73 ms on lock 0x7b480030c858. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:00:58.270203 (+ 60269us) spinlock_profiling.cc:243] Waited 44.6 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:00:58.607716 (+337513us) raft_consensus.cc:1685] finished
0709 13:00:58.607738 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:58.607908 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4104,"prepare.queue_time_us":45483,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":2151,"raft.queue_time_us":60396,"raft.run_cpu_time_us":3076,"raft.run_wall_time_us":613980,"spinlock_wait_cycles":108953728,"thread_start_us":97014,"threads_started":1,"wal-append.queue_time_us":282812}
I20260709 13:00:58.597106  4681 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:58.597771  4558 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:58.586421  4771 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:58.599900  4592 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:00:58.620600  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.622771  1689 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:00:58.590413  4693 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.588096  4760 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.615334  4631 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:58.620642  4558 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:58.624069  4646 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.622375  4592 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:58.616153  4661 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.026s
W20260709 13:00:58.661934  4781 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.704s	user 0.002s	sys 0.000s
I20260709 13:00:58.665045  4751 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:58.674082  1535 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:00:58.675822  1538 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:00:58.676576  1535 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:00:58.677724  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:58.678232  1534 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:58.680166  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:58.680694  1537 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:00:58.681808  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:58.682978  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.684816  1541 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
I20260709 13:00:58.648146  4646 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:00:58.647264  4631 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.695935  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:58.720252  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:58.721055  1618 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:58.696617  1534 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:00:58.697201  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:00:58.707293  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:58.717722  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:58.670396  4411 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:58.718564  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:58.724332  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:58.725098  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:58.725728  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:58.732909  1617 raft_consensus.cc:2393] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:58.749743  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:58.750504  1611 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
W20260709 13:00:58.758309  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 57513.6us
I20260709 13:00:58.771587  1618 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:58.783674  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:58.687034  4747 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:00:58.835770  1616 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 13:00:58.837670  1609 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:00:58.723703  4744 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:00:58.853981  4801 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.218s	user 0.000s	sys 0.000s
I20260709 13:00:58.843086  4505 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.855192  4505 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.857729  4654 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:58.845504  4628 consensus_meta.cc:287] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.534s	user 0.000s	sys 0.010s
I20260709 13:00:58.872605  4807 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.692929  4592 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.876662  2228 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:58.783864  4411 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.873224  4807 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:58.910137  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 149869us
I20260709 13:00:58.791993  4803 consensus_queue.cc:1059] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.124s
I20260709 13:00:58.883311  4524 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:58.918823  4681 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:58.919096  4668 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:58.931246  4772 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:00:58.937134  4592 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:00:58.935678  4760 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.954433  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 962) took 2111 ms. Trace:
I20260709 13:00:58.954854  2236 rpcz_store.cc:276] 0709 13:00:56.843038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:56.864229 (+ 21191us) service_pool.cc:224] Handling call
0709 13:00:56.864768 (+   539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:58.101848 (+1237080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:58.954207 (+852359us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54007102d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:58.954375 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2842927872}
I20260709 13:00:58.939023  4693 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.939229  4771 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:58.952462  4681 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.963830  4681 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.964686  4577 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.979235  4749 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:58.982035  4505 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.986822  2228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 964) took 1372 ms. Trace:
I20260709 13:00:58.987255  2228 rpcz_store.cc:276] 0709 13:00:57.614431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.615473 (+  1042us) service_pool.cc:224] Handling call
0709 13:00:58.986652 (+1371179us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540036a6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:58.986756 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2879693568}
I20260709 13:00:58.992241  1610 raft_consensus.cc:2410] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:00:58.952270  4668 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:58.995290  4668 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:58.951447  4772 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:58.999194  4411 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:59.000221  4646 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:59.001180  4766 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:58.945513  4524 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:59.010057  4767 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.711s	user 0.011s	sys 0.000s
I20260709 13:00:59.010993  4577 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:59.012316  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.020692  1541 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:59.014606  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:59.021967  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.022176  1534 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:59.023749  1535 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:00:59.017350  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:59.024855  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.017928  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.025434  1535 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:59.016896  1538 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:00:59.020707  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.026667  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.023674  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.027367  1535 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:59.027642  1540 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:00:59.028735  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:58.966495  4807 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:00:59.014043  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.029932  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.030520  1540 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:00:59.031764  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:00:59.035902  4421 consensus_meta.cc:287] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.140s	user 0.012s	sys 0.000s
W20260709 13:00:59.039393  4659 consensus_meta.cc:287] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.587s	user 0.003s	sys 0.009s
I20260709 13:00:59.040257  2235 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:00:59.040477  4670 raft_consensus.cc:3055] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:00:59.041842  1613 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:00:59.042419  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 636) took 1316 ms. Trace:
W20260709 13:00:59.044036  4522 consensus_meta.cc:287] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.730s	user 0.010s	sys 0.000s
I20260709 13:00:59.046507  4631 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:59.049695  4532 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.050271  4532 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.052896  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1434) took 1155 ms. Trace:
I20260709 13:00:59.053258  1685 rpcz_store.cc:276] 0709 13:00:57.898003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.899416 (+  1413us) service_pool.cc:224] Handling call
0709 13:00:57.899818 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:58.090934 (+191116us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:00:58.109065 (+ 18131us) log.cc:844] Serialized 14 byte log entry
0709 13:00:58.286707 (+177642us) log.cc:844] Serialized 14 byte log entry
0709 13:00:58.287314 (+   607us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:00:58.287380 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:00:58.288682 (+  1302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.052152 (+763470us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b18003ce380. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9cd2a6 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419
0709 13:00:59.052667 (+   515us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b5400374e54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:59.052712 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.052841 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21493,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":37,"raft.queue_time_us":521,"raft.run_wall_time_us":10419,"spinlock_wait_cycles":490661248,"thread_start_us":176345,"threads_started":1,"wal-append.queue_time_us":239405}
I20260709 13:00:59.055653  4666 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:00:59.055905  4651 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.717s	user 0.009s	sys 0.000s
I20260709 13:00:59.015754  4772 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:59.019297  4681 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.058588  4681 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:59.058933  4524 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:59.060870  4681 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:59.065927  4558 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:59.016105  4505 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:59.067668  4505 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.068707  4558 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:59.042805  2235 rpcz_store.cc:276] 0709 13:00:57.726231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.242072 (+515841us) service_pool.cc:224] Handling call
0709 13:00:58.242459 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.040625 (+798166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.042166 (+  1541us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b54002a3fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:00:59.042356 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1833844096}
I20260709 13:00:59.001730  4766 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.073778  4666 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:59.074812  4666 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.075608  4766 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:59.076381  4772 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:00:59.077889  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.080821  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:59.081517  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:59.082082  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:59.082708  1538 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:00:59.083319  1536 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:59.083477  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:59.086302  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:59.087543  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:59.088721  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:59.089525  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:59.089622  1685 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:00:59.090894  1536 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:59.091166  1535 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:00:59.091893  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:00:59.092953  1540 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:00:59.093137  1534 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:59.085095  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.096109  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:59.098156  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:59.098856  1690 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:59.101143  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:59.102329  1685 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:00:59.103542  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:59.107182  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:59.041121  4670 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:00:59.111186  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:59.115569  2201 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:59.116315  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:59.112147  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:00:59.112833  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.118387  1534 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:00:59.108345  1690 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:00:59.120898  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.121176  4421 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.121748  1610 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:59.121690  4421 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.122864  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.123688  1618 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:00:59.124135  1612 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:00:59.118770  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:59.128072  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:59.193723  4646 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:00:59.144794  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.281715  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.280372  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:59.282615  1609 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:00:59.129385  4654 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.282979  1611 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:00:59.177783  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:59.279951  1617 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:00:59.197530  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.280655  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:59.285249  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 168198us
I20260709 13:00:59.286779  2203 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:00:59.290043  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:59.291639  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:59.293608  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 382386us
I20260709 13:00:59.129074  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.176494  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.297804  1616 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:59.292423  1612 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 5.
I20260709 13:00:59.297175  1618 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:00:59.309459  1610 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:00:59.129158  1535 raft_consensus.cc:1217] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Deduplicated request from leader. Original: 6.8->[7.9-7.9]   Dedup: 7.9->[]
I20260709 13:00:59.365213  1535 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 13:00:59.195626  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 67604.2us
I20260709 13:00:59.368479  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:59.369352  2235 raft_consensus.cc:2410] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:00:59.369053  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:59.370165  2236 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:59.339763  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:59.194545  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 63131.1us
I20260709 13:00:59.373004  1538 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:00:59.106491  4659 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.376833  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.393720  1612 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:00:59.377617  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.379801  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:00:59.382467  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 59305.4us
I20260709 13:00:59.385520  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:59.379542  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:59.393853  1617 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:00:59.393949  1614 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:00:59.400454  2233 raft_consensus.cc:2410] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:00:59.401369  2230 raft_consensus.cc:2410] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:00:59.401691  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:59.401998  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:59.402338  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:59.402789  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:59.404256  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:59.406526  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.406689  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:59.193985  4744 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:59.321647  4769 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.365s	user 0.010s	sys 0.019s
I20260709 13:00:59.298755  4695 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:59.128984  4771 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:59.114316  4554 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:59.113952  4766 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.296s
I20260709 13:00:59.435467  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:59.440670  1611 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:00:59.298435  4755 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:00:59.444419  2229 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:00:59.129845  4767 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:59.445683  1616 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:00:59.424646  4659 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:59.446521  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 44636us
I20260709 13:00:59.447026  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:59.298650  4522 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:00:59.435418  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:59.445569  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 73585.6us
I20260709 13:00:59.451185  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:00:59.468853  2231 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:59.392894  4805 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:00:59.126624  4628 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.127019  4735 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:59.459511  1618 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:00:59.196532  4751 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:00:59.130446  4668 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:59.383220  4505 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:59.463990  2228 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:00:59.462809  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:59.445067  4769 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.489s	user 0.010s	sys 0.019s
I20260709 13:00:59.288139  4524 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:00:59.471216  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.105878  4577 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:59.510795  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.518028  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2331) took 1249 ms. Trace:
I20260709 13:00:59.130060  4651 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.333s
I20260709 13:00:59.323855  4654 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:59.528987  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.460160  4554 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:00:59.531569  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 65179.9us
W20260709 13:00:59.532166  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 63944us
W20260709 13:00:59.548244  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 261555us
I20260709 13:00:59.548712  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:59.536217  1507 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:59.537730  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:00:59.461092  4681 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:59.542035  1689 rpcz_store.cc:276] 0709 13:00:58.268773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.269053 (+   280us) service_pool.cc:224] Handling call
0709 13:00:59.517562 (+1248509us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54000749d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:00:59.517949 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752554240}
I20260709 13:00:59.461258  4810 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:00:59.465045  4767 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:00:59.466064  4532 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:59.499861  4522 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:00:59.561290  2203 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:00:59.457572  4744 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.533143  4750 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:00:59.566304  1611 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:00:59.579468  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.537712  4505 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:00:59.586557  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.556115  4554 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:00:59.613556  4505 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.616482  4505 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:59.566426  4767 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:00:59.584641  4532 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:59.530974  4628 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:00:59.623986  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 75339.5us
W20260709 13:00:59.626240  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 460073us
W20260709 13:00:59.622095  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 73884.6us
I20260709 13:00:59.628405  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.541786  4647 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 13:00:59.537837  4735 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.571125  4522 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:00:59.637023  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.537302  4668 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.653565  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.559170  4681 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:59.667150  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 131031us
I20260709 13:00:59.670615  1507 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:59.638597  4411 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:00:59.639329  4421 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:59.675046  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:00:59.679217  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:00:59.681648  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.641224  4695 raft_consensus.cc:697] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:59.629491  4807 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:00:59.687223  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.687994  1611 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:00:59.690917  4747 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:00:59.691516  4815 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:00:59.691597  4747 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.675107  4766 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.695788  4766 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.697899  4766 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:59.698235  4744 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:59.699152  4654 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:59.671485  4651 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:59.673710  4829 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.700206  4651 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.700567  4829 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.702400  4659 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:59.702643  4829 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:59.702989  4651 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:59.703646  4695 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.705245  4668 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:59.705860  4668 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.706342  4668 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.709038  4668 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:59.671451  4783 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:00:59.635381  4532 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.709864  4783 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:00:59.710139  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.710843  4783 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.711556  4532 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:59.704445  4681 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:59.714748  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:00:59.542163  4524 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.699651  4628 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:59.715682  1683 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:00:59.666234  4771 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:00:59.716861  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:59.717967  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:59.718469  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:59.716812  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:59.717748  1611 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:00:59.723870  1683 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:00:59.761816  2235 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
W20260709 13:00:59.765827  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 95283.9us
I20260709 13:00:59.768042  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:59.688056  4827 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:59.780445  4767 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:00:59.784658  1657 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.000s	sys 0.007s
W20260709 13:00:59.785303  1657 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.000s	sys 0.007s
W20260709 13:00:59.786104  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 224892us
W20260709 13:00:59.786835  1508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.000s	sys 0.007s
I20260709 13:00:59.791486  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.721545  4532 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:00:59.877295  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2483) took 2279 ms. Trace:
I20260709 13:00:59.877942  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:59.881822  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:00:59.792443  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.878206  1542 rpcz_store.cc:276] 0709 13:00:57.597884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:57.598023 (+   139us) service_pool.cc:224] Handling call
0709 13:00:57.598336 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:57.601306 (+  2970us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:57.601333 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:57.605516 (+  4183us) log.cc:844] Serialized 26 byte log entry
0709 13:00:59.518540 (+1913024us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b1800310ba0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:00:59.787354 (+268814us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:00:59.787416 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.787594 (+   178us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:00:59.876980 (+ 89386us) raft_consensus.cc:1685] finished
0709 13:00:59.877001 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:00:59.877229 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2594,"raft.queue_time_us":6468,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":4911,"spinlock_wait_cycles":2714496,"thread_start_us":268574,"threads_started":1,"wal-append.queue_time_us":2269659}
W20260709 13:00:59.850271  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 66429.6us
I20260709 13:00:59.895869  1542 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:00:59.890516  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:00:59.850347  1509 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.710s (SENT) [suppressed 19 similar messages]
I20260709 13:00:59.899708  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.900451  1541 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
W20260709 13:00:59.900369  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50077.4us
W20260709 13:00:59.902858  1508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.005s	sys 0.008s
I20260709 13:00:59.898663  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:59.893857  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.934343  1534 raft_consensus.cc:2410] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:00:59.792888  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.935030  1535 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:00:59.893255  1543 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:00:59.782066  4821 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:59.783842  4834 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:00:59.791785  4668 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:00:59.918676  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.942584  1542 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:00:59.922732  4838 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.875224  4771 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:59.937042  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:59.792089  4827 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:00:59.953835  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.954773  1542 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
W20260709 13:00:59.944382  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 837658us
I20260709 13:00:59.945787  1691 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:00:59.956247  4835 raft_consensus.cc:2764] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:00:59.956089  4522 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.946314  4747 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:00:59.946790  4659 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:00:59.952275  4821 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.957806  4659 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.959142  1508 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:00:59.946806  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 48214.3us
I20260709 13:00:59.947736  4577 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:59.948707  4841 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:00:59.960942  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:59.926486  4843 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:59.894626  4532 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.961961  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:00:59.966777  1691 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:00:59.967510  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:59.976697  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:00:59.919564  4681 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:00:59.986181  1507 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:00:59.988327  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.947162  4735 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:00:59.942360  4834 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:00.000720  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:00.033571  1534 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
W20260709 13:00:59.986542  4772 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.891s	user 0.013s	sys 0.000s
I20260709 13:01:00.035876  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:00:59.985700  4835 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:00:59.987411  4747 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:00:59.989912  4651 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:00.012535  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:00:59.990198  4829 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:00.047199  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:00:59.990880  4666 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.048938  1610 raft_consensus.cc:2410] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:00:59.992039  4750 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.042433  4819 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:00.036589  4834 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:00.061928  4819 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:00:59.963698  4838 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:00.059684  4830 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:00.053220  4651 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.053509  4829 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.062211  4766 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:01:00.038211  4832 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:00.076324  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:00:59.987226  4505 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 13:01:00.043927  4837 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.047506  4835 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:00:59.990866  4554 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.050339  4735 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:00.051698  4747 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:00.038285  4577 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 13:01:00.036190  4654 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:01:00.068499  4827 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:00:59.985656  4843 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:00.066533  4845 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:01:00.059991  4751 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:01:00.109956  4505 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:00.114259  4505 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:00.106181  4695 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:00.060489  4666 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.115262  4695 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.073563  4829 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.042313  4847 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 13:01:00.116824  4847 consensus_queue.cc:1243] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:01:00.103704  4670 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:00.066982  4532 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:00.115396  4505 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.061239  4750 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.123365  4554 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:00.094333  4783 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:00.051011  4846 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:01:00.107827  4747 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:00.115679  4843 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:00.076054  4767 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:00.082698  4828 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:00.070614  4838 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:00.071574  4830 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:00.068136  4819 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:00.106933  4835 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.104290  4837 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.129561  4828 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:00.135788  4828 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:00.136667  4837 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:00.083537  4832 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:00.133872  4830 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:00.147331  4829 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.149718  4829 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:00.151109  4830 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.152735  4830 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:00.141919  4834 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:00.155156  4834 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:00.156693  4832 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:00.196023  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 209701us
W20260709 13:01:00.205830  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 215860us
W20260709 13:01:00.209386  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.210584  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:00.213742  1688 raft_consensus.cc:2368] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8.
I20260709 13:01:00.210654  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:00.222743  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 969) took 2503 ms. Trace:
I20260709 13:01:00.222954  1540 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:00.223431  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:00.226555  1685 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:00.226342  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:00.239178  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:00.240510  1683 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:00.242359  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:00.239184  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:00.243692  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 610436us
I20260709 13:01:00.244117  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:00.246412  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:00.246737  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:00.247665  1688 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:01:00.247745  1684 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:00.225376  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:00.068310  4369 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:00.071797  4651 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:00.117529  4695 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:00.123818  4750 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:00.120198  4654 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:00.338228  4654 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:00.340116  4654 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.124749  4670 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:00.325822  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 116410us
I20260709 13:01:00.117403  4666 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:00.335469  1689 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
W20260709 13:01:00.336148  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 373275us
I20260709 13:01:00.223312  2232 rpcz_store.cc:276] 0709 13:00:57.718916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:58.240428 (+521512us) service_pool.cc:224] Handling call
0709 13:00:58.241035 (+   607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:58.249260 (+  8225us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:00:59.466488 (+1217228us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x7b1800453580. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fc455 00007fe0934e43c7 00007fe08faf69a7 00007fe08fb060bf 00007fe08fb06038
0709 13:00:59.792346 (+325858us) log.cc:844] Serialized 14 byte log entry
0709 13:00:59.933731 (+141385us) log.cc:844] Serialized 14 byte log entry
0709 13:00:59.937622 (+  3891us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:59.938094 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 13:00:59.938235 (+   141us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:00:59.938275 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:00:59.938366 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:00.069330 (+130964us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b1800453580. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff522 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:01:00.221974 (+152644us) raft_consensus.cc:1685] finished
0709 13:01:00.222025 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:00.222404 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85090,"prepare.queue_time_us":1423473,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":84,"raft.queue_time_us":20585,"raft.run_cpu_time_us":649,"raft.run_wall_time_us":29632,"spinlock_wait_cycles":7148672,"thread_start_us":429686,"threads_started":2,"wal-append.queue_time_us":149330}
I20260709 13:01:00.347710  4320 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:00.345837  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:00.343308  4832 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:00.344180  4751 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.360813  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:00.365271  1689 pending_rounds.cc:85] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 4
I20260709 13:01:00.369494  4849 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:00.107951  4735 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.371518  1509 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:00.373346  4821 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:00.374106  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 339513us
I20260709 13:01:00.374600  1508 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:00.375072  4841 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:00.375926  4369 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:00.377514  4532 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.378448  4827 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:00.379328  4681 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:00.381906  4832 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:00.384152  4654 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:00.386432  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 339193us
I20260709 13:01:00.390735  2235 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:00.391935  4505 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:00.394759  4668 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:00.362339  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:00.413661  1689 pending_rounds.cc:107] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:01:00.443032  1689 raft_consensus.cc:2889] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:00.439908  1685 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:00.445039  1542 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
W20260709 13:01:00.432232  4755 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.011s	sys 0.021s
W20260709 13:01:00.432192  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 799051us
I20260709 13:01:00.443264  4532 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:00.462633  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 91363.3us
I20260709 13:01:00.448004  4668 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:00.453984  4755 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.011s	sys 0.022s
I20260709 13:01:00.476981  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:00.127755  4783 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:00.723387  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:00.686585  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 112738us
I20260709 13:01:00.559037  2232 raft_consensus.cc:2410] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:00.565287  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 524) took 1442 ms. Trace:
W20260709 13:01:00.576484  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 201994us
I20260709 13:01:00.581037  2234 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:00.550899  1685 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:00.491712  1687 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 13:01:00.586897  4866 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:01:00.491001  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 413287us
I20260709 13:01:00.569826  2203 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:00.520982  4807 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:01:00.564838  1691 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.576s	user 0.014s	sys 0.002s
I20260709 13:01:00.550792  4861 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:00.718406  4849 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:00.592983  4832 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:00.486505  4522 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:00.549571  4592 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.481052  4554 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.505465  4830 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.517980  4524 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:00.803845  1508 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:00.801156  1691 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:01:00.501943  4411 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:00.798843  4807 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:00.575851  4836 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:00.509745  4854 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.364s
W20260709 13:01:00.812970  4835 consensus_meta.cc:287] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.655s	user 0.012s	sys 0.000s
I20260709 13:01:00.815810  4735 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:00.576128  4369 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:00.517768  4847 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.378s
I20260709 13:01:00.815569  4873 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:01:00.480355  4755 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:00.484056  4850 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:00.815725  4828 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.679s	user 0.006s	sys 0.012s
I20260709 13:01:00.550319  4549 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.259s
I20260709 13:01:00.870743  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2356) took 1076 ms. Trace:
I20260709 13:01:00.805270  4532 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:00.716637  4869 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.106s
I20260709 13:01:00.523154  4751 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.511670  4827 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:00.564479  4805 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.250s
I20260709 13:01:00.802497  4861 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:00.824513  1582 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:00.810952  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:00.813371  4747 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.684s	user 0.007s	sys 0.002s
I20260709 13:01:00.831156  1613 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:01:00.842577  4651 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:00.853507  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 47527.3us
W20260709 13:01:00.852747  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 55415.5us
I20260709 13:01:00.773221  1615 rpcz_store.cc:276] 0709 13:00:59.122269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.127589 (+  5320us) service_pool.cc:224] Handling call
0709 13:00:59.127898 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:00:59.128098 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:00:59.325864 (+197766us) log.cc:844] Serialized 14 byte log entry
0709 13:01:00.243057 (+917193us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b18003f3820. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c979c0a 00007fe0960b9219 00007fe08ca01a40 00007fe08c9fc4e4 00007fe08c9fc455 00007fe08fae4060 00007fe08faf6716 00007fe08fb07287 00007fe08fb07208
0709 13:01:00.483860 (+240803us) log.cc:844] Serialized 14 byte log entry
0709 13:01:00.564529 (+ 80669us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:00.564618 (+    89us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:00.564701 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.564819 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:00.565033 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":336855,"prepare.queue_time_us":197562,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":25700,"raft.queue_time_us":883492,"raft.run_cpu_time_us":399,"raft.run_wall_time_us":398,"spinlock_wait_cycles":11631360,"thread_start_us":244811,"threads_started":2,"wal-append.queue_time_us":299832}
I20260709 13:01:00.873309  1691 rpcz_store.cc:276] 0709 13:00:59.793993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.956577 (+162584us) service_pool.cc:224] Handling call
0709 13:01:00.870612 (+914035us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:00.873322  4554 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:00.900046  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44366us
I20260709 13:01:00.854899  4873 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:00.911913  1686 raft_consensus.cc:2368] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8.
I20260709 13:01:00.899461  4861 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:00.874715  4761 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:01:00.891588  4849 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.906450  4549 consensus_queue.cc:1243] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:00.928808  4642 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:00.923301  4670 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.904671  4828 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.911039  4873 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:01:00.937903  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2340) took 1220 ms. Trace:
I20260709 13:01:00.908051  4747 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.947424  1440 catalog_manager.cc:5697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 5 to 6, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:01:00.948382  1437 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55374 (request call id 321) took 19276 ms (client timeout 15000 ms). Trace:
I20260709 13:01:00.902009  4835 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.949276  4668 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:00.916772  4830 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.950148  4760 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:01:00.950505  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 380854us
W20260709 13:01:00.957990  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 155192us
I20260709 13:01:00.958298  4670 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:00.978807  4828 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:00.979437  4670 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:00.980142  4554 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:00.973227  4760 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:00.981784  4760 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:00.982285  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:00.983376  4760 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:00.959734  4835 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:00.930739  4876 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:00.961156  1613 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1463) took 1442 ms (client timeout 1637 ms). Trace:
I20260709 13:01:00.961709  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:00.961483  1440 catalog_manager.cc:5697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 4 to 5, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:00.962414  4651 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:00.962442  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:00.963131  4411 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:00.964041  4631 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
W20260709 13:01:00.967275  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:00.967242  4854 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:00.912963  4592 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:01.000801  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.002488  4747 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:01.003104  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:01.004292  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:01.005025  1543 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.008806  1692 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:01.009999  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.010303  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 623) took 1061 ms. Trace:
I20260709 13:01:01.010782  1535 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:01:01.010643  1692 rpcz_store.cc:276] 0709 13:00:59.948947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.966676 (+ 17729us) service_pool.cc:224] Handling call
0709 13:00:59.967029 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:01.009292 (+1042263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:01.010066 (+   774us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400374954. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:01.010251 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2354093312}
I20260709 13:01:00.984120  4320 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:00.984808  1613 rpcz_store.cc:269] 0709 13:00:59.519325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.519519 (+   194us) service_pool.cc:224] Handling call
0709 13:01:00.960912 (+1441393us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54002bd254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:00.961074 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2958128512}
I20260709 13:01:00.971664  4850 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:00.967809  4522 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:00.993644  4505 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.540s	user 0.012s	sys 0.000s
I20260709 13:01:00.955163  1690 rpcz_store.cc:276] 0709 13:00:59.711443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.714633 (+  3190us) service_pool.cc:224] Handling call
0709 13:00:59.714927 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.722763 (+  7836us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:00:59.722792 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:59.724704 (+  1912us) log.cc:844] Serialized 26 byte log entry
0709 13:01:00.230478 (+505774us) spinlock_profiling.cc:243] Waited 1.61 ms on lock 0x7b18001e6f40. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:01:00.310522 (+ 80044us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:00.310642 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.310772 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:00.920894 (+610122us) raft_consensus.cc:1685] finished
0709 13:01:00.920913 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:00.932206 (+ 11293us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7532,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":34614,"spinlock_wait_cycles":3703168,"thread_start_us":76827,"threads_started":1,"wal-append.queue_time_us":779919}
I20260709 13:01:01.030462  2235 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:01.031919  4854 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.034560  1658 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:01.036350  4861 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:00.983880  4646 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:01:00.989691  4755 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.002540  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:00.988914  4876 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:01.040970  4659 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:01.025527  4320 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.055217  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:01.056077  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 60721.4us
I20260709 13:01:01.057058  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.071591  1543 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:01.057581  4760 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.053647  4849 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:01.066463  4320 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:01.061273  4876 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.090951  1440 catalog_manager.cc:5697] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 4 to 7, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 7 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:01:01.096642  4865 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.526s	user 0.003s	sys 0.010s
I20260709 13:01:01.097582  1542 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:01.098242  4850 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.104035  1535 raft_consensus.cc:2410] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:01.103317  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1901) took 2092 ms. Trace:
I20260709 13:01:01.078336  4651 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.107419  1539 rpcz_store.cc:276] 0709 13:00:59.011299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.014935 (+  3636us) service_pool.cc:224] Handling call
0709 13:00:59.015250 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.019798 (+  4548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.019832 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:59.057833 (+ 38001us) log.cc:844] Serialized 26 byte log entry
0709 13:01:00.209383 (+1151550us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b1800155200. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:01:00.447831 (+238448us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:00.461727 (+ 13896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.469968 (+  8241us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:01.096087 (+626119us) spinlock_profiling.cc:243] Waited 46.2 ms on lock 0x7b2400175ad8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c8859c4 00007fe08c883acb 00007fe08ca3edc9 00007fe08ca3ea55 00007fe08ca3e09d 00007fe08f67010e 00007fe08f68788c 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221
0709 13:01:01.099915 (+  3828us) raft_consensus.cc:1685] finished
0709 13:01:01.099939 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:01.103118 (+  3179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4349,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":27444,"spinlock_wait_cycles":163539584,"thread_start_us":97111,"threads_started":1,"wal-append.queue_time_us":1420061}
I20260709 13:01:01.110317  4659 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.116452  1536 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:01.116026  4659 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.116781  4747 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:01.117205  4747 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.117762  1537 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:01:01.077687  4642 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:01.122473  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 51414.4us
I20260709 13:01:01.117611  4747 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.096750  1440 catalog_manager.cc:5697] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 3 to 5, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:01.127039  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:01.075680  4522 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:00.957199  1437 rpcz_store.cc:269] 0709 13:00:41.671451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:41.671776 (+   325us) service_pool.cc:224] Handling call
0709 13:01:00.948092 (+19276316us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:00:52.000321 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:00:52.001062 (+   741us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:00:52.001134 (+    72us) write_op.cc:435] Acquired schema lock
0709 13:00:52.001168 (+    34us) tablet.cc:662] Decoding operations
0709 13:00:52.107289 (+106121us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:00:52.107410 (+   121us) write_op.cc:641] Partition lock acquired for write op
0709 13:00:52.115768 (+  8358us) tablet.cc:685] Acquiring locks for 22 operations
0709 13:00:52.117616 (+  1848us) tablet.cc:701] Row locks acquired
0709 13:00:52.117672 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:00:52.117857 (+   185us) write_op.cc:270] Start()
0709 13:00:52.118031 (+   174us) write_op.cc:276] Timestamp: P: 1783602052117792 usec, L: 0
0709 13:00:52.120488 (+  2457us) op_driver.cc:348] REPLICATION: starting
0709 13:00:52.233656 (+113168us) log.cc:844] Serialized 8621 byte log entry
0709 13:00:52.598713 (+365057us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08c9e20ac 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b
0709 13:00:53.505939 (+907226us) op_driver.cc:464] REPLICATION: finished
0709 13:00:53.768700 (+262761us) write_op.cc:301] APPLY: starting
0709 13:00:53.777733 (+  9033us) tablet.cc:1367] starting BulkCheckPresence
0709 13:00:53.915933 (+138200us) tablet.cc:1370] finished BulkCheckPresence
0709 13:00:53.915977 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:00:54.164631 (+248654us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:00:54.165112 (+   481us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 13:00:54.165157 (+    45us) write_op.cc:312] APPLY: finished
0709 13:00:54.229839 (+ 64682us) log.cc:844] Serialized 152 byte log entry
0709 13:00:54.257776 (+ 27937us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:54.297957 (+ 40181us) write_op.cc:454] Released schema lock
0709 13:00:54.425777 (+127820us) write_op.cc:341] FINISH: Updating metrics
0709 13:00:54.602330 (+176553us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:00:54.613264 (+ 10934us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17371,"reported_tablets":43,"spinlock_wait_cycles":18688,"child_traces":[["op",{"apply.queue_time_us":242310,"apply.run_cpu_time_us":11753,"apply.run_wall_time_us":822208,"num_ops":22,"prepare.queue_time_us":144077,"prepare.run_cpu_time_us":38293,"prepare.run_wall_time_us":1342913,"raft.queue_time_us":92673,"raft.run_cpu_time_us":17937,"raft.run_wall_time_us":202151,"replication_time_us":1364771,"spinlock_wait_cycles":24986368,"thread_start_us":1294315,"threads_started":4,"wal-append.queue_time_us":643466,"wal-append.run_cpu_time_us":12267,"wal-append.run_wall_time_us":2657338}]]}
I20260709 13:01:01.129927  4659 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:01.130571  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:01.130745  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:01.086428  4783 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:01.137763  1582 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:01.139372  4668 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:01.140381  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:01.137751  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 52084.2us
W20260709 13:01:01.150100  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 93607.9us
I20260709 13:01:01.151119  1540 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:01.131995  4760 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.158617  1613 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:01:01.171648  4522 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:01.170848  1440 catalog_manager.cc:5697] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 3 to 4, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:01.171938  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:01.172288  1541 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.172667  4761 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:01.173681  4631 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 13:01:01.190284  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2359) took 1156 ms. Trace:
W20260709 13:01:01.166571  4654 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.721s	user 0.008s	sys 0.007s
I20260709 13:01:01.190716  1685 rpcz_store.cc:276] 0709 13:01:00.033415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.452751 (+419336us) service_pool.cc:224] Handling call
0709 13:01:00.473243 (+ 20492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:00.782089 (+308846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.872839 (+ 90750us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b54000dc120. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa63dc0 00007fe08faefa02 00007fe08faebdd6 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:01:01.190045 (+317206us) spinlock_profiling.cc:243] Waited 77.3 ms on lock 0x7b54000f6754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:01.190213 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183606912}
I20260709 13:01:01.194244  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:01.192054  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.196064  1688 raft_consensus.cc:2410] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:01.201608  1615 raft_consensus.cc:2368] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9.
I20260709 13:01:01.205132  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:01.204583  1440 catalog_manager.cc:5697] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 6 to 7, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:01.211750  1610 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:01.214118  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:01.208485  1440 catalog_manager.cc:5697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 5 to 6, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 6 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:01:01.222158  2203 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:01.182159  4651 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:01.184595  4549 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 13:01:01.254971  4760 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:01.268936  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 267994us
W20260709 13:01:01.233619  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 160766us
I20260709 13:01:01.235948  1537 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:01:01.236116  4761 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:01.238204  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:01.239595  2203 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:01.240254  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:01.241088  4522 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:01.241739  1440 catalog_manager.cc:5697] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 2 to 5, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 5 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:01:01.223191  4783 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:01.236434  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.247779  4854 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:01.248608  4830 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:01.201256  4642 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.202777  4827 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:01.216796  4751 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:01.306017  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.311648  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:01.313207  1537 raft_consensus.cc:2410] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:01.315958  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:01.281665  4522 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.320979  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:01.289938  4505 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:01.327206  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:01.330844  1611 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:01.331205  2234 raft_consensus.cc:2410] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:01.317276  1440 catalog_manager.cc:5697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 5 to 7, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:01.334908  1615 raft_consensus.cc:2393] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:01.299083  4654 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:01.336580  1613 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
W20260709 13:01:01.347906  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 108433us
I20260709 13:01:01.349040  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:01.327445  4522 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.348726  1440 catalog_manager.cc:5697] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 1 to 4, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:01:01.354564  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.320322  4761 raft_consensus.cc:697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:01.343146  4751 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:01.361559  4761 consensus_queue.cc:237] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:01.362095  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:01.363276  4554 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:01.363782  4554 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.364300  4554 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.349021  4870 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:01.365669  4861 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:01.367046  1537 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:01.367444  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:01.368248  1685 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:01:01.370174  4766 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:01.370954  4830 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:01.371601  4830 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:01.357300  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:01.373373  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.358922   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:01.359210  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:01.359469  4533 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:01.376452  4533 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:01.360044  4505 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.338231  4592 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:01.377285  4751 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:01.348659  4642 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:01.364841  4870 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:01.385613  4522 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.382318  4533 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.390637  4870 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.391795  4533 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:01.392118  4522 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:01.392874  4522 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:01.385361  4830 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:01.394253  4628 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:01.394647  4628 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:01.395232  4651 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:01.395737  4628 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.381852  4783 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:01.382715  4832 consensus_queue.cc:1059] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
W20260709 13:01:01.398729  4879 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.108s	user 0.002s	sys 0.000s
I20260709 13:01:01.398949  4832 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:01.399364  4832 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:01.399510  4751 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.400414  4832 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.338542  4850 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:01.403370  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:01.404656  4862 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.714s	user 0.001s	sys 0.001s
I20260709 13:01:01.407105  4832 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.407415  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1918) took 1616 ms. Trace:
I20260709 13:01:01.397235  4628 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.387462  4823 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.411110  4823 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.412081  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:01.415102  4661 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.416070  4661 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.417994  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:01.418090  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:01.392904  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:01.403451  1509 leader_election.cc:336] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.526s (SENT)
I20260709 13:01:01.421089  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:01.421707  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:01.422710  4651 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:01.423132  4651 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.423446  4651 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:01.423677  4850 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:01:01.379491  4654 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:01.375707  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:01.425935  4369 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.426924  4850 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:01.431510  4807 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.415232  4549 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:01.434363  4549 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.437412  4549 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:01.412983  1540 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:01.450284  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 278484us
I20260709 13:01:01.439327  4870 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:01.407997  1538 rpcz_store.cc:276] 0709 13:00:59.790956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.793044 (+  2088us) service_pool.cc:224] Handling call
0709 13:00:59.793447 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:00:59.932542 (+139095us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:00:59.932593 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:00:59.942539 (+  9946us) log.cc:844] Serialized 26 byte log entry
0709 13:01:00.511433 (+568894us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:00.570034 (+ 58601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:00.570182 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:01.403775 (+833593us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:01.406935 (+  3160us) raft_consensus.cc:1685] finished
0709 13:01:01.406956 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:01.407221 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71566,"raft.queue_time_us":127797,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":14040,"spinlock_wait_cycles":9889024,"thread_start_us":489474,"threads_started":2,"wal-append.queue_time_us":694422}
W20260709 13:01:01.441502  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 65947.7us
I20260709 13:01:01.443346  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:01.445029  1536 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
W20260709 13:01:01.454468  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 105595us
I20260709 13:01:01.433281  4807 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:01.468267  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.468950  1541 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
W20260709 13:01:01.471580  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 68165.6us
I20260709 13:01:01.475893  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:01.475034  1507 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:01.480911  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:01.485898  2234 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:01.488611  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:01.491775  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:01.499706  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:01.500222  1690 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:01.529592  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1451) took 1513 ms. Trace:
I20260709 13:01:01.499395  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:01.530166  1688 rpcz_store.cc:276] 0709 13:00:59.984523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:00.304303 (+319780us) service_pool.cc:224] Handling call
0709 13:01:01.497980 (+1193677us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b5400374454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:01.498173 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1799238016}
I20260709 13:01:01.535513  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:01.541776  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.545835  4854 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:01.546728  4854 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.542972  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:01.556432  1509 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.564s (SENT)
I20260709 13:01:01.490063  4654 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:01.561890  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:01.564245  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:01.565032  2228 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:01:01.567694  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:01.569411  1692 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:01.571794  1509 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:01.552564  4411 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.071s	sys 0.169s
I20260709 13:01:01.580322  2228 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:01.583398  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 113407us
I20260709 13:01:01.493637  4747 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.590200  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:01.591375  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.591856  1686 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:01.580801  4411 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.071s	sys 0.169s
I20260709 13:01:01.536742  4861 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:01.551632  4592 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.586586  1508 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:01.605108  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:01.569406  4661 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:01.624560  1689 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:01.621455  4846 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:01.627609  2233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 824) took 2241 ms (client timeout 2804 ms). Trace:
I20260709 13:01:01.627467  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:01.628647  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:01.632141  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 102689us
I20260709 13:01:01.633116  4855 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:01.635954  2233 rpcz_store.cc:269] 0709 13:00:59.386529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:59.402566 (+ 16037us) service_pool.cc:224] Handling call
0709 13:01:01.622182 (+2219616us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b540065bb54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:01.627415 (+  5233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5032232320}
I20260709 13:01:01.582837  4854 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:01.597719  4681 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:01.650852  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 149234us
I20260709 13:01:01.618142  4836 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:01:01.655586  4668 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.603593  4823 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:01.656268  1509 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.584s (SENT)
I20260709 13:01:01.611274  4889 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.650820  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:01.661285  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:01.634683  4873 consensus_meta.cc:287] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.641s	user 0.012s	sys 0.000s
I20260709 13:01:01.633774  4783 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.636023  4654 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:01.635898  4846 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.651290  4855 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:01.668957  4836 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:01.668977  4668 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.669711  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:01.670611  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:01.710119  1615 raft_consensus.cc:2410] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:01.667959  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:01.667703  4876 consensus_meta.cc:287] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.000s
I20260709 13:01:01.670490  4873 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:01.677340  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.677587  4783 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.677760  1507 leader_election.cc:400] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8.
I20260709 13:01:01.713929  1507 leader_election.cc:403] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:01.714491  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.714577  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:01.714819  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 50230.2us
I20260709 13:01:01.680403  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:01.682144  4830 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:01.689824  4577 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:01.682546  1509 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.637s (SENT)
I20260709 13:01:01.681732  4651 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:01.659883  4505 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:01.661055  4592 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:01.707011  4846 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:01.708958  1688 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
W20260709 13:01:01.672917  4320 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.576s	user 0.012s	sys 0.000s
I20260709 13:01:01.652384  4554 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:01.726799  1541 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:01.704785  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 628) took 1145 ms. Trace:
I20260709 13:01:01.741044  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:01.728849  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:01.738567  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:01.739506  1686 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:01.738718  2233 raft_consensus.cc:2410] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:01.741364  1689 rpcz_store.cc:276] 0709 13:01:00.559842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.358858 (+799016us) service_pool.cc:224] Handling call
0709 13:01:01.704558 (+345700us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b5400058f54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:01.704695 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":572502912}
I20260709 13:01:01.666626  4854 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:01:01.711186  4855 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:01.747613  2229 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:01.747398  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.746493  2237 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
I20260709 13:01:01.708186  4836 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:01.750953  1536 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:01:01.756912  4783 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:01.747468  4320 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:01.757315  1538 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:01.718317  4876 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.758934  1534 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.759516  4320 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:01.718067  4681 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8.
I20260709 13:01:01.762012  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.762183  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:01.763020  1541 raft_consensus.cc:2410] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:01:01.763252  1690 raft_consensus.cc:2410] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:01.765182  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:01.763087  4554 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:01.773494  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.773918  4854 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.774710  4854 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.775950  4836 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.783026  4505 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:01.783304  4827 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:01.756577  1692 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:01.783612  4505 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:01.784077  4505 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:01.720563  4830 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:01.789636  4884 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:01:01.786450  4668 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:01.789889  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.803308  1611 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:01.818483  4747 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:01.744771  4892 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.821491  4892 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.824782  4836 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:01.834461  1540 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:01:01.840596  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:01.712023  4889 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:01.830049  4854 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:01.829615  4823 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:01.792611  4830 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:01.886777  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:01.884186  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:01.874934  4320 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:01.901336  1613 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:01.904400  1684 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:01.904919  1534 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
W20260709 13:01:01.905640  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.893286  4823 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.913002  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.915697  1541 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:01.916554  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.941074  1539 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:01.917532  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.923288  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:01.924515  4896 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 13:01:01.904589  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:01.925735  1689 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
W20260709 13:01:01.904402  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 278113us
I20260709 13:01:01.945602  1536 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:01.948105  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:01.953044  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.904968  4320 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:01.903333  4654 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:01.937605  4628 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.525s	user 0.006s	sys 0.006s
I20260709 13:01:01.903729  4783 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:01.991103  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 86587.6us
I20260709 13:01:01.994410  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:01.995458  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:01.951057  4823 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:01.987576  4320 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:02.027027  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.029160  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:01.987043  4827 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:01.991945  4642 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:01.997586  4783 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:02.000790  4854 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:02.028735  4830 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:02.043257  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:02.036147  4668 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:02.085690  4631 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:02.086211  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 137928us
I20260709 13:01:02.087013  4906 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:02.088016  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:02.088865  4897 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
I20260709 13:01:02.089771  4892 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:02.090091  1684 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:02.090658  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:02.104633  4904 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.105319  4904 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:02.106986  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.108317  4904 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:02.091276  4889 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:02.090114  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:02.104835  1685 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 5: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:02.042479  4807 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:02.110122  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 427602us
I20260709 13:01:02.099689  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:02.080602  4783 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:02.112005  1540 raft_consensus.cc:2393] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:02.116465  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.117827  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.120035  1690 raft_consensus.cc:2410] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:01:02.123176  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.119308  1689 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:02.045986  4411 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 13:01:02.079995  4873 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:02.092733  4651 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:02.129997  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.099107  4631 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:02.133836  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.111430  4861 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.136755  1684 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 5.
W20260709 13:01:02.137378  1509 leader_election.cc:336] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.846s (SENT)
I20260709 13:01:02.135854  1613 raft_consensus.cc:2376] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9. Re-sending same reply.
W20260709 13:01:02.141353  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 98203.5us
I20260709 13:01:02.142829  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:02.124989  4592 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:02.122344  4805 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.122560  4906 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:02.153522  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.139509  4631 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:02.157773  1684 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:02.110950  4827 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.156327  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.159595  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.173687  1584 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:02.177583  1611 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:01:02.179369  1613 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:02.141654  4807 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:02.187048  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:02.135496  4668 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:02.188935  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 60557.5us
I20260709 13:01:02.138303  4854 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.193934  1615 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:02.132263  4654 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:02.185834  4628 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:01:02.170266  4832 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
W20260709 13:01:02.197387  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:02.197564  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 448017us
I20260709 13:01:02.197690  4892 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:02.198189  4369 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:02.198618  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.201076  4668 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:02.206245  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:02.227733  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.228447  4747 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:02.224622  1509 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:02.229060  4747 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:02.230345  4668 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:02.230034  4747 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.228338  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.231561  1685 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:01:02.231782  4827 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:02.198083  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.153821  4861 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:02.235633  1613 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:02.185307  4807 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:02.247153  1685 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
W20260709 13:01:02.206651  4901 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.212s	user 0.000s	sys 0.000s
W20260709 13:01:02.251526  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 64609.3us
I20260709 13:01:02.253883  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.254850  1684 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
W20260709 13:01:02.257517  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 94198.3us
W20260709 13:01:02.252867  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 59692.6us
I20260709 13:01:02.259855  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.259675  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.260779  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:02.257838  4870 consensus_meta.cc:287] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.790s	user 0.009s	sys 0.004s
I20260709 13:01:02.238699  4577 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:02.172606  4805 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.298237  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:02.299024  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:02.302343  1509 leader_election.cc:336] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.710s (SENT)
I20260709 13:01:02.302855  1509 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:02.304755  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:02.309428  1685 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9. Ops: [5.3-8.4]
I20260709 13:01:02.311509  4861 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:02.313747  4854 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:02.315969  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.312716  4823 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.321882  4870 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:02.301646  4917 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:02.326985  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:02.331077  1685 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:02.345117  4805 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:02.349319  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.201959  4906 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.319856  4577 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:02.364295  1615 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
W20260709 13:01:02.368515  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 66179us
W20260709 13:01:02.373210  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 72346.8us
I20260709 13:01:02.329298  4917 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.373323  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:02.381729  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:02.382318  1615 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:02.383430  2203 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:02.407598  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.418537  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.423094  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:02.425292  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.425774  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:02.425971  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.426662  1686 raft_consensus.cc:2368] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8.
I20260709 13:01:02.427598  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.427913  4892 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:02.428404  1686 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:02.429808  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:02.433234  1684 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:01:02.431838  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.381031  4876 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:02.434558  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.438987  4755 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:02.440426  4823 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:02.429276  4925 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 13:01:02.443145  4836 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.618s	user 0.010s	sys 0.001s
I20260709 13:01:02.445716  1684 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:02.462365  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.462428  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.463277  1691 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:02.449389  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:02.385646  4917 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.428417  4892 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:02.454660  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:02.454543  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:02.474710  1536 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:01:02.464916  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:02.476111  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.477494  1685 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:01:02.477762  1543 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:02.474934  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:02.476758  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:02.476730  1686 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:01:02.480693  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:02.483055  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.480836  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:02.485579  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1930) took 1477 ms. Trace:
I20260709 13:01:02.486037  1534 rpcz_store.cc:276] 0709 13:01:00.999262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.002681 (+  3419us) service_pool.cc:224] Handling call
0709 13:01:02.476009 (+1473328us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b5400418cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:02.476116 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1736704384}
I20260709 13:01:02.486611  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.486897  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.487638  1613 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:01:02.487835  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.491734  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.492131  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:02.492733  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.493006  1684 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:02.420284  4320 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.493470  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:02.494239  1691 raft_consensus.cc:2368] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8.
W20260709 13:01:02.489694  4505 consensus_meta.cc:287] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.705s	user 0.014s	sys 0.000s
I20260709 13:01:02.495440  4320 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:02.495183  4505 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:02.420311  4870 consensus_queue.cc:1059] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 13:01:02.496909  4870 raft_consensus.cc:3055] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:02.450393  4807 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:01:02.497629  1687 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:02.498278  4807 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:02.498723  4807 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:02.499642  4807 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.500586  4505 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:02.500986  4807 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:02.503338  1540 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:02.492350  1686 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:01:02.506696  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:02.507493  1536 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:02.508718  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:02.509398  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.509698  4783 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.511003  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:02.511716  1618 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.512178  4783 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:02.512647  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.514014  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.515033  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.515239  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:02.515841  2233 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:02.515904  1616 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.516175  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:02.516912  1543 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:02.518085  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:02.518832  4891 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.732s	user 0.003s	sys 0.004s
I20260709 13:01:02.519469  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:02.520421  4909 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.362s	user 0.003s	sys 0.003s
I20260709 13:01:02.520812  1535 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.520735  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.522575  1542 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:02.523557  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:02.527568  4855 consensus_meta.cc:287] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.699s	user 0.003s	sys 0.008s
I20260709 13:01:02.527647  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.528136  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.528558  1611 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:01:02.529567  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.529790  2231 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:02.530468  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 159637us
I20260709 13:01:02.497318  4870 raft_consensus.cc:740] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:02.532724  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.532907  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:02.484416  1688 raft_consensus.cc:2376] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6. Re-sending same reply.
I20260709 13:01:02.536157  2229 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:02.531946  1614 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:02.538213  1615 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:01:02.536005  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:02.513293  2237 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:01:02.542589  1535 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:02.542156  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:02.546960  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:02.547165  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:02.548229  1615 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:02.548472  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:02.549157  1614 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:02.549750  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.567184  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:02.568560  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:02.569008  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.572551  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:02.577538  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 102686us
I20260709 13:01:02.579947  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.545272  4836 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:02.586158  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:02.587162  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.587494  2237 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:02.586408  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:02.588354  1536 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:01:02.586472  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:02.552491  4805 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:02.528169  4855 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.610705  1507 leader_election.cc:400] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8.
I20260709 13:01:02.618022  1615 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:02.620703  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:02.586023  4411 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.625277  2229 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:02.625680  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:02.668900  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.674247  1611 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:02.669039  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:02.623997  1508 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:02.601823  4823 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:02.597329  4876 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:02.602202  4805 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:02.705911  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:02.706974  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 86404.2us
I20260709 13:01:02.706532  1692 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:02.691509  1688 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:01:02.706099  2203 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:02.708299  1688 pending_rounds.cc:85] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 9
W20260709 13:01:02.710528  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 86586us
I20260709 13:01:02.719717  1688 pending_rounds.cc:107] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:01:02.720192  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:02.720240  1688 raft_consensus.cc:2889] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:02.721472  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:02.668633  4906 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:02.604852  4577 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.084s
I20260709 13:01:02.722983  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:02.709708  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:02.738813  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:02.722437  1537 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:02.739075  1686 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:02.709375  4850 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:02.709697  4505 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:02.753319  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 81930.8us
I20260709 13:01:02.610638  4861 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.116s
I20260709 13:01:02.619793  4661 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.113s
I20260709 13:01:02.718321  4827 raft_consensus.cc:2764] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 13:01:02.711927  4805 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:02.710654  4823 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:02.770646  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:02.777838  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 96748.5us
I20260709 13:01:02.670499  4871 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 13:01:02.704785  4876 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.749209  4873 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:02.672987  4870 consensus_queue.cc:260] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:02.676522  4411 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.787099  1615 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:02.799448  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.782060  4823 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:02.742587  4906 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.812014  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.814529  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:02.858898  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 187909us
I20260709 13:01:02.798588  4850 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:02.863922  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:02.856232  4631 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.006s
I20260709 13:01:02.867676  4631 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.870488  4631 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:02.798483  4654 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 13:01:02.874981  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:01:02.881862  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 95499.2us
I20260709 13:01:02.846572  4661 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:02.863698  4932 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.158s	user 0.001s	sys 0.001s
I20260709 13:01:02.874675  1616 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:02.853920  4761 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:02.857534  4827 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:02.896891  2203 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:02.810864  4628 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:01:02.902305  1618 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:02.899540  4661 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:02.905789  2203 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:02.910359  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:02.912377  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.915287  1617 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:01:02.886615  4870 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:02.909477  4661 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:02.921662  4936 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:01:02.922348  4661 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:02.924594  4935 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 13:01:02.928169  4931 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.489s	user 0.004s	sys 0.000s
W20260709 13:01:02.929910  4921 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:01:02.914539  4873 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:02.933010  4846 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.146s	user 0.010s	sys 0.000s
I20260709 13:01:02.933998  4755 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.935578  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:02.936889  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.937341  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:02.937959  2229 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:02.938346  2230 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:01:02.939389  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:02.940634  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.941610  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.944361  4906 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:02.945894  4876 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:02.947139  4855 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:02.905711  4747 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:02.948745  4747 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:02.949168  4747 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:02.954538  4411 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:02.956924  4933 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
W20260709 13:01:02.958418  4830 consensus_meta.cc:287] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.857s	user 0.009s	sys 0.009s
I20260709 13:01:02.960924  1690 raft_consensus.cc:2410] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:02.873967  4850 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:02.963784  4850 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:02.906471  4320 raft_consensus.cc:2764] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:01:02.964984  2234 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:02.919023  4761 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.965935  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.972288  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.973989  1686 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
W20260709 13:01:02.969130  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 105859us
W20260709 13:01:02.976092  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 70606.6us
I20260709 13:01:02.977413  1617 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:02.978574  2229 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:01:02.907758  4827 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:02.982061  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.982946  2228 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:02.984951  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.985029  1613 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:01:02.985893  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:02.986650  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:02.987572  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 267450us
I20260709 13:01:02.984107  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.988601  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.990775  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.893241  4892 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:02.991108  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:02.991969  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:02.992177  2230 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:02.992223  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:02.996744  1616 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:02.990108  2229 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:02.991590  2233 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:02.998131  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:02.936707  4917 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:02.972988  4855 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:03.040807  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:03.008710  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:03.012168  1509 leader_election.cc:336] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.221s (SENT)
I20260709 13:01:02.988236  4827 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.016732  1609 raft_consensus.cc:2410] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:01:03.017911  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.019114  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:02.997714  4766 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:02.973445  4846 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:03.009541  4830 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:03.030249  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:03.031361  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.037142  2228 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:03.041735  1535 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.043131  2230 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:03.058748  2237 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:01:03.045468  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.045892  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:03.060135  1534 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:03.046855  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.061578  1610 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:01:03.050096  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.050776  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.043870  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:03.069842  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 73858.4us
I20260709 13:01:03.027204  4943 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:01:03.078264  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.078684  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.049789  4766 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.040904  4861 consensus_queue.cc:1243] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:01:03.080063  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.089717  1613 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:03.090672  1610 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:03.092888  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.095122  4861 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:03.096000  4861 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:03.079948  2237 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:03.098920  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:03.080013  1538 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:03.113124  1613 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.050933  4855 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:03.116400  4855 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:03.106362  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.117406  1616 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:03.119072  1616 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9. Ops: [5.3-8.4]
I20260709 13:01:03.120713  1613 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:03.122013  2232 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:03.112349  1535 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
W20260709 13:01:03.124958  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 94766.3us
I20260709 13:01:03.073297  4906 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:03.138053  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.121279  1616 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:01:03.133641  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.140280  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.126066  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:03.160804  1610 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:03.099604  4861 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:03.088948  4766 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:03.159941  4897 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:01:03.184669  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.185282  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.218729  1692 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:03.220919  2237 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
W20260709 13:01:03.222901  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.222433  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.227376  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.228070  1616 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:03.201982  4830 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:01:03.235930  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2364) took 1360 ms. Trace:
I20260709 13:01:03.237797  1687 rpcz_store.cc:276] 0709 13:01:01.875962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.882820 (+  6858us) service_pool.cc:224] Handling call
0709 13:01:01.897860 (+ 15040us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:02.498094 (+600234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.235684 (+737590us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b540028e6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:03.235870 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1378884992}
W20260709 13:01:03.239315  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 113337us
I20260709 13:01:03.256083  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:03.256867  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.230455  4761 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:03.230839  4892 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:03.270264  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.278957  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:03.281003  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:03.285866  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.287822  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:01:03.292248  4961 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 13:01:03.298583  4954 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:01:03.302613  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.303531  2231 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:03.304325  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:03.305550  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.266847  4906 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.307204  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.307967  2231 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:03.308507  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:03.312806  2228 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:03.315047  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:03.315791  1508 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:03.242843  4947 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 13:01:03.276974  4951 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:03.291095  4654 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:03.322084  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.279196  4855 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:01:03.306828  4917 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:03.340368  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.318713  4505 raft_consensus.cc:2764] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:01:03.317597  4369 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:01:03.350490  4934 log.cc:927] Time spent T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.757s	user 0.004s	sys 0.000s
I20260709 13:01:03.352389  4873 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:03.353549  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.354856  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.355700  2232 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:03.356470  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.357546  4946 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:03.358045  4946 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:03.358901  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2527) took 1529 ms. Trace:
I20260709 13:01:03.359287  1538 rpcz_store.cc:276] 0709 13:01:01.829180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:01.829414 (+   234us) service_pool.cc:224] Handling call
0709 13:01:01.843395 (+ 13981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:03.113309 (+1269914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.358661 (+245352us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400343fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:03.358842 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2843486080}
W20260709 13:01:03.361228  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.362234  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.327097  4846 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:03.329854  4654 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.358409  4946 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:03.345781  4805 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.000s
I20260709 13:01:03.366422  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.320356  4577 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.026s
W20260709 13:01:03.363230  4823 consensus_meta.cc:287] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.001s
I20260709 13:01:03.320423  4906 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:03.376884  2236 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 9. Ops: []
I20260709 13:01:03.377743  4577 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:03.367381  4369 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:03.382551  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.428417  2232 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:01:03.432080  4836 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.836s	user 0.007s	sys 0.004s
I20260709 13:01:03.432552  4836 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:03.433238  1536 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
W20260709 13:01:03.382335  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 99318.3us
W20260709 13:01:03.408351  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.397169  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.371290  4654 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:03.442248  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.442819  2237 raft_consensus.cc:2376] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8. Re-sending same reply.
I20260709 13:01:03.443986  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:03.443889  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.444772  2237 raft_consensus.cc:2376] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6. Re-sending same reply.
W20260709 13:01:03.418738  4960 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 13:01:03.445731  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.448328  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.448737  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:03.382023  4577 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:03.451027  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.451828  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:03.453075  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.453933  1687 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:03.453258  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.454617  2234 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:03.376468  4805 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:03.450301  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:03.455123  2237 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:03.457072  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.379058  4747 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:03.447916  4830 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:03.381701  4823 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:03.462028  1692 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:01:03.460573  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.469637  1507 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:03.460526  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.473598  2237 raft_consensus.cc:2376] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8. Re-sending same reply.
W20260709 13:01:03.475432  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.480527  4747 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:03.480741  4836 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:03.481571  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:03.481647  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:03.483652  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:03.451839  4505 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:03.486941  1507 leader_election.cc:400] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8.
I20260709 13:01:03.489998  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:03.488130  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:03.491994  4861 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:03.492281  4369 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:03.492975  4846 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:03.494253  1687 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:03.495002  1507 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:03.496044  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:03.459405  4938 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.865s	user 0.004s	sys 0.000s
I20260709 13:01:03.497167  1614 raft_consensus.cc:2368] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11.
I20260709 13:01:03.498320  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.532909  2230 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:03.497874  4755 consensus_meta.cc:287] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.563s	user 0.011s	sys 0.001s
I20260709 13:01:03.533341  1617 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:01:03.532994  1610 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:03.533556  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.536439  4850 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:03.537045  4850 raft_consensus.cc:697] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:03.537608  4577 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:03.538053  4850 consensus_queue.cc:237] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:03.541002  4823 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:03.493916  4836 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:03.541771  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:03.542419  1685 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:03.542678  4836 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:03.543325  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:03.544977  1537 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:03.545966  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:03.546190  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:03.541843  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 51960.7us
I20260709 13:01:03.546967  1692 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:03.548202  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:01:03.611591  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 63463.8us
I20260709 13:01:03.469569  4830 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:03.618980  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.619263  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:03.498392  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:03.498679  4861 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:03.620489  1538 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:03.623037  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:03.619841  1691 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:03.494632  4892 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:03.621940  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:03.622319  2228 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:03.619029  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:03.624178  1688 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:03.613075  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:03.522750  4505 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:03.619773  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:03.712684  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 92987.7us
I20260709 13:01:03.713877  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:03.653085  4953 raft_consensus.cc:2764] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:03.653824  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:03.724568  1537 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:03.610948  4668 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.612812  4755 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:03.612120  4577 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.657768  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:03.620680  4830 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.619772  4592 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:03.670910  4971 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:03.734757  1616 raft_consensus.cc:2368] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9.
W20260709 13:01:03.719247  2203 connection.cc:256] RPC call timeout handler was delayed by -1.9363767940000116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20230000000000004s for a response
W20260709 13:01:03.671712  4967 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:01:03.740450  2235 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:03.677834  4906 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:03.679306  1691 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
W20260709 13:01:03.684916  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.684504  1538 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:01:03.762811  1538 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:03.766415  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:03.623054  4861 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:03.767468  4861 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.702307  4805 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:03.705406  4870 consensus_meta.cc:287] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.740s	user 0.008s	sys 0.001s
I20260709 13:01:03.705839  4747 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:03.611718  4811 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.123s
W20260709 13:01:03.706334  4827 consensus_meta.cc:287] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.655s	user 0.009s	sys 0.001s
I20260709 13:01:03.707948  4876 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:01:03.710125  1509 leader_election.cc:336] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.767s (SENT)
I20260709 13:01:03.783674  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:03.722437  2232 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:01:03.785176  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.786720  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.787777  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.788511  2232 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:01:03.722607  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 109645us
I20260709 13:01:03.789817  1582 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:03.506217  4951 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:03.791838  4951 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:03.783474  4876 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:03.794762  4876 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:03.795885  4876 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.728783  2234 raft_consensus.cc:2376] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8. Re-sending same reply.
W20260709 13:01:03.670913  4968 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:01:03.728279  4533 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:03.771210  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 43859.8us
I20260709 13:01:03.800073  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:03.770409  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:03.731699  4592 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:03.776122  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:03.803511  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.802261  4592 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:03.803265  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:03.801376  1657 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:03.726801  4651 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:01:03.773713  4906 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:03.758345  4971 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:03.783424  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:03.848338  1540 raft_consensus.cc:2410] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:01:03.839597  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 49875.3us
I20260709 13:01:03.854238  1615 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:01:03.846935  4953 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:03.855707  4953 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:03.862389  4956 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.622s	user 0.007s	sys 0.004s
W20260709 13:01:03.863397  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 153371us
I20260709 13:01:03.864192  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2375) took 1708 ms. Trace:
I20260709 13:01:03.864609  4854 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:03.758458  4668 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:03.870548  4944 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:03.871060  4944 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:03.871589  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 1979) took 1338 ms. Trace:
I20260709 13:01:03.871893  4668 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:03.871999  4944 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:03.872262  4505 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:03.872005  1614 rpcz_store.cc:276] 0709 13:01:02.532995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.551122 (+ 18127us) service_pool.cc:224] Handling call
0709 13:01:03.871384 (+1320262us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b5400001c54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:03.871527 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2129975680}
I20260709 13:01:03.875934  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 542) took 1616 ms. Trace:
I20260709 13:01:03.876993  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:03.864579  1690 rpcz_store.cc:276] 0709 13:01:02.155813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.156003 (+   190us) service_pool.cc:224] Handling call
0709 13:01:03.863972 (+1707969us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b54000744d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:03.864132 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1832556928}
I20260709 13:01:03.873162  4944 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:03.900662  1540 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:03.902037  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 188202us
I20260709 13:01:03.864795  4654 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:03.847612  4320 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:03.847401  4906 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:03.781394  4747 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:03.876343  1612 rpcz_store.cc:276] 0709 13:01:02.259641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.266361 (+  6720us) service_pool.cc:224] Handling call
0709 13:01:02.266770 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:02.325486 (+ 58716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:02.325527 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:02.332842 (+  7315us) log.cc:844] Serialized 26 byte log entry
0709 13:01:02.797563 (+464721us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b1800309100. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:01:03.046033 (+248470us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:03.122652 (+ 76619us) log.cc:844] Serialized 14 byte log entry
0709 13:01:03.150605 (+ 27953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.861594 (+710989us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:03.875203 (+ 13609us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b1800309100. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9cd2a6 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419
0709 13:01:03.875623 (+   420us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b54003219d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:03.875665 (+    42us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:03.875716 (+    51us) raft_consensus.cc:1685] finished
0709 13:01:03.875732 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:03.875874 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14707,"prepare.queue_time_us":146551,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":886,"raft.queue_time_us":43307,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"spinlock_wait_cycles":112414720,"thread_start_us":300530,"threads_started":2,"wal-append.queue_time_us":817389}
W20260709 13:01:03.924527  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:03.830677  4827 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:03.892827  4946 consensus_meta.cc:287] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.002s
I20260709 13:01:03.939082  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:03.942914  1542 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:03.941753  1540 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:03.923661  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:03.932996  4906 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:03.900992  4592 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:03.981438  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 180101us
W20260709 13:01:03.984452  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 212712us
W20260709 13:01:03.985118  2203 connection.cc:256] RPC call timeout handler was delayed by -1.7896767940000018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21860000000000002s for a response
W20260709 13:01:03.986236  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 136320us
W20260709 13:01:03.988806  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 124751us
W20260709 13:01:03.985535  4766 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.793s	user 0.011s	sys 0.001s
I20260709 13:01:03.978415  4953 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:04.009639  1542 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:03.945977  4892 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:04.014648  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:03.949470  4946 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:04.026789  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:04.018738  1509 leader_election.cc:336] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.396s (SENT)
W20260709 13:01:04.041811  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.083390  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:04.099931  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.039312  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.026274  1657 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:04.123618  1537 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:04.161566  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:04.131850  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:04.090018  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:04.061005  1536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 1945) took 1460 ms (client timeout 1600 ms). Trace:
I20260709 13:01:03.988029  4651 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:04.252480  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 328880us
I20260709 13:01:04.096269  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:04.253767  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:04.260062  1539 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:04.260690  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:04.261327  2233 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:04.131171  4981 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:01:04.114871  4654 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:04.257928  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 257198us
I20260709 13:01:04.279258  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:04.038209  4975 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:01:04.104856  4983 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:04.028100  4980 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:04.256778  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 242204us
W20260709 13:01:04.129707  4939 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.191s	user 0.002s	sys 0.002s
W20260709 13:01:04.262529  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.257718  4986 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:04.130445  4974 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 13:01:04.059010  4805 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:04.268854  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:04.273790  1612 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: [6.7-7.8]
I20260709 13:01:04.275424  1535 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:04.257937  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:04.276948  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:04.316133  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.316226  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:04.276378  1536 rpcz_store.cc:269] 0709 13:01:02.600388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.609868 (+  9480us) service_pool.cc:224] Handling call
0709 13:01:04.060840 (+1450972us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b54004182d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:04.060946 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1743225216}
W20260709 13:01:04.317528  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 313400us
I20260709 13:01:04.277984  1611 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:01:04.138403  4827 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:04.278057  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:04.145861  4971 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:04.141048  4592 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:04.251055  2230 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:04.282179  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 975) took 1346 ms. Trace:
I20260709 13:01:04.284322  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 862) took 1264 ms. Trace:
I20260709 13:01:04.325138  2236 rpcz_store.cc:276] 0709 13:01:02.935788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.940207 (+  4419us) service_pool.cc:224] Handling call
0709 13:01:02.940496 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:03.389033 (+448537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:04.281853 (+892820us) spinlock_profiling.cc:243] Waited 420 ms on lock 0x7b54002f49d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:04.282097 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":966761728}
I20260709 13:01:04.325624  2229 rpcz_store.cc:276] 0709 13:01:03.020351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.069925 (+ 49574us) service_pool.cc:224] Handling call
0709 13:01:03.360559 (+290634us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:03.360774 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:03.366490 (+  5716us) log.cc:844] Serialized 14 byte log entry
0709 13:01:03.456384 (+ 89894us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:03.456439 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:03.456490 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:03.456557 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:04.284122 (+827565us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b5400741150. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:04.284249 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4253,"prepare.queue_time_us":7985,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":1725,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":8790,"spinlock_wait_cycles":703810176,"thread_start_us":89567,"threads_started":1,"wal-append.queue_time_us":179685}
I20260709 13:01:04.013181  4979 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.302109  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:04.258589  4871 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:04.303737  2235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 979) took 1346 ms (client timeout 1780 ms). Trace:
I20260709 13:01:04.054143  4755 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.222s
I20260709 13:01:04.304489  1582 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:04.307711  4953 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:01:04.017653  4870 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.262s
I20260709 13:01:04.004200  4861 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.137354  4830 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:04.028024  4977 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.320s
I20260709 13:01:04.306281  4651 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:04.278520  4876 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:01:04.318177  1509 leader_election.cc:336] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.646s (SENT)
I20260709 13:01:04.314836  1616 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:04.018519  4766 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.317955  1657 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:04.338476  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.309485  4983 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:04.312636  4805 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:04.322563  4943 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:04.343082  1582 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.305864  2232 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:01:04.340081  2235 rpcz_store.cc:269] 0709 13:01:02.937001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.940370 (+  3369us) service_pool.cc:224] Handling call
0709 13:01:04.283574 (+1343204us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b54004731d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:04.283743 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1837869312}
I20260709 13:01:04.322860  4811 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:01:04.347589  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45567.8us
I20260709 13:01:04.376236  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:04.348917  4916 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.362752  1543 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:04.371754  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1947) took 1649 ms. Trace:
I20260709 13:01:04.377782  1582 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.352581  4990 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:04.408102  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 90245.3us
I20260709 13:01:04.411679  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:04.412431  1657 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:04.413511  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 374245us
I20260709 13:01:04.413769  4983 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.414853  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.374711  4943 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.417898  4943 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:04.337342  4892 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.421762  4505 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:04.422235  4505 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.422621  4505 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:04.434854  4944 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.537s	user 0.006s	sys 0.003s
I20260709 13:01:04.403908  4654 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:04.410828  4651 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:04.438516  1615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1488) took 1633 ms (client timeout 1877 ms). Trace:
I20260709 13:01:04.398968  1541 rpcz_store.cc:276] 0709 13:01:02.722491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.722769 (+   278us) service_pool.cc:224] Handling call
0709 13:01:02.742253 (+ 19484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:02.845636 (+103383us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:02.893736 (+ 48100us) log.cc:844] Serialized 14 byte log entry
0709 13:01:03.023053 (+129317us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:03.023125 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:03.023194 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:04.105071 (+1081877us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b180044d0a0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9cd2a6 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419
0709 13:01:04.371404 (+266333us) spinlock_profiling.cc:243] Waited 71.8 ms on lock 0x7b54000ad054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:04.371477 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:04.371670 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4529,"prepare.queue_time_us":27880,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":4437,"raft.queue_time_us":101,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":3536,"spinlock_wait_cycles":170639488,"thread_start_us":128372,"threads_started":1,"wal-append.queue_time_us":207597,"wal-append.run_cpu_time_us":2137,"wal-append.run_wall_time_us":1174112}
W20260709 13:01:04.441669  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 123518us
I20260709 13:01:04.417008  4766 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:04.442358  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.447252  1509 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.399894  4906 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.447992  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:04.448086  1615 rpcz_store.cc:269] 0709 13:01:02.804694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.814195 (+  9501us) service_pool.cc:224] Handling call
0709 13:01:04.438336 (+1624141us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b54000884d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:04.438448 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2145833856}
I20260709 13:01:04.438963  4651 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:04.449779  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 127778us
I20260709 13:01:04.407639  4918 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.458433  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.438755  4861 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:04.400290  4949 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.466363  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:04.465198  1509 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:04.446851  4823 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:04.471993  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1464) took 2199 ms. Trace:
I20260709 13:01:04.485680  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:04.446616  4991 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.490190  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:04.487412  1509 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.453047  4990 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.458081  4654 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:04.460187  4944 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:04.444903  4971 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:01:04.459843  4811 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:04.481168  1689 rpcz_store.cc:276] 0709 13:01:02.272817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:02.297570 (+ 24753us) service_pool.cc:224] Handling call
0709 13:01:02.308442 (+ 10872us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:02.322136 (+ 13694us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:01:02.322587 (+   451us) log.cc:844] Serialized 14 byte log entry
0709 13:01:02.681802 (+359215us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:02.724483 (+ 42681us) log.cc:844] Serialized 26 byte log entry
0709 13:01:02.724680 (+   197us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:02.724726 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:02.724818 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:03.853583 (+1128765us) spinlock_profiling.cc:243] Waited 83.9 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:04.465958 (+612375us) raft_consensus.cc:1685] finished
0709 13:01:04.471756 (+  5798us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:04.471916 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2678,"prepare.queue_time_us":225,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":41058,"raft.run_cpu_time_us":373,"raft.run_wall_time_us":33645,"spinlock_wait_cycles":192939776,"thread_start_us":348233,"threads_started":1,"wal-append.queue_time_us":459482}
I20260709 13:01:04.511580  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:04.475000  4953 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.472843  4755 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.497593  4827 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:04.489078  4668 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:04.491632  4946 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:04.532974  4823 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:04.533929  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 43841.3us
I20260709 13:01:04.534852  4998 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:04.537537  1507 leader_election.cc:336] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.568s (SENT)
I20260709 13:01:04.501936  4861 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:04.548466  1539 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:04.549862  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.549836  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:04.553545  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 141976us
I20260709 13:01:04.510552  4944 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.480494  4949 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.555255  2233 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
W20260709 13:01:04.549243  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 101359us
I20260709 13:01:04.519052  4654 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:04.546238  4827 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.469906  4805 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.531726  4991 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:04.576812  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 89519.1us
I20260709 13:01:04.569505  1582 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:04.521301  4953 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:04.578994  5007 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:04.580024  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:04.569954  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:04.579865  5007 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:04.552175  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 139797us
I20260709 13:01:04.581660  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.582065  5006 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.582360  1657 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:04.582861  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:04.551460  4876 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.583760  4876 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.584973  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:04.553589  4533 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:04.590176  1688 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:04.572410  4755 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:04.549664  4668 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:04.563529  4944 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.594008  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:04.580232  4953 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:04.580691  1689 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:01:04.583235  4980 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:01:04.496994  4871 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:01:04.555111  4505 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.584124  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:04.585891  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:04.563825  4998 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.575156  4827 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:04.591715  4823 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.584065  5007 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:04.595229  2203 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.573175  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:04.593243  4755 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.554419  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:04.653087  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 98754.1us
I20260709 13:01:04.633342  1509 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:04.636487  1690 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:04.575220  4946 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:04.638710  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 54647us
I20260709 13:01:04.591408  4995 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.594367  4668 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:04.592947  4805 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.632439  4861 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:04.662766  4861 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.663192  4861 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.663527  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:04.660270  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:04.666057  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 92960.3us
I20260709 13:01:04.660789  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:04.655586  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:04.667883  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:04.654776  4755 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.666146  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:04.674461  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:04.675567  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:04.676532  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:04.591173  4996 raft_consensus.cc:2764] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:01:04.689056  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.690747  1684 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:04.597393  4906 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:04.691529  1687 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9. Ops: [5.3-8.4]
I20260709 13:01:04.715163  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:04.716276  1618 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:04.655202  4892 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.693408  1683 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
W20260709 13:01:04.693737  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 60507.5us
I20260709 13:01:04.740871  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.687017  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:04.696889  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:04.741724  1688 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:01:04.741835  1615 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:04.699462  1684 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:04.700055  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:04.743026  1614 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.699898  1689 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:04.743423  1687 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:04.701529  1686 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:04.653226  4952 raft_consensus.cc:3055] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:04.690871  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:04.632151  4654 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:04.646749  4980 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:04.693025  1685 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 13:01:04.755586  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:04.721143  4505 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:04.722347  4998 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:04.723425  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:04.723790  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1023) took 1030 ms. Trace:
W20260709 13:01:04.763013  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 61830.4us
I20260709 13:01:04.763370  2232 rpcz_store.cc:276] 0709 13:01:03.693544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.789005 (+ 95461us) service_pool.cc:224] Handling call
0709 13:01:04.283743 (+494738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:04.375004 (+ 91261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:04.723576 (+348572us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b54004731d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:04.723721 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1117631872}
I20260709 13:01:04.728027  1537 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:04.743821  4983 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:04.732941  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:04.734463  4876 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:04.737111  4830 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:04.694291  4971 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:04.636821  4811 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.737798  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:04.747490  5011 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.698012  4823 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:04.786437  1614 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:04.791900  4971 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:04.697600  4871 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:04.749358  1539 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:04.792876  4871 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:04.711167  2233 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:04.794740  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:04.753726  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:04.753983  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 90511.9us
I20260709 13:01:04.798388  1657 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:04.756455  2229 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
W20260709 13:01:04.802023  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 48383.1us
W20260709 13:01:04.802893  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.808759  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:04.756945  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 170110us
I20260709 13:01:04.759425  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:04.732620  4943 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:04.765971  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.763401  1536 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:04.815311  2235 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:04.768357  1616 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.748387  4952 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:04.738920  4949 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:04.800858  4986 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.759002  4999 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:01:04.781531  4998 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:01:04.815917  1507 leader_election.cc:336] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.191s (SENT)
I20260709 13:01:04.816439  2203 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.832813  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:04.822170  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:04.821899  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:04.821911  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:04.779477  4980 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.792989  4971 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:04.825325  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 84530.3us
I20260709 13:01:04.804817  4823 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.807395  4892 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:04.830269  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:04.834493  1612 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:04.800491  4827 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.847095  1616 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:04.836995  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.835243  1611 raft_consensus.cc:2376] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6. Re-sending same reply.
I20260709 13:01:04.838152  2203 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.822804  4906 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:04.764237  4755 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.859009  4946 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.860662  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.861086  4755 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:04.841724  1690 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:04.861840  4876 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:04.843181  1658 leader_election.cc:400] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11.
I20260709 13:01:04.862264  5022 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:04.835863  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:04.834502  4949 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.829656  4830 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:04.845989  4916 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:04.856875  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 58589.3us
I20260709 13:01:04.864888  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:04.865531  1609 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.862411  1658 leader_election.cc:403] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:04.866950  2203 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.852176  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:04.868202  1658 leader_election.cc:400] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9.
I20260709 13:01:04.868494  1658 leader_election.cc:403] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 13:01:04.872699  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:04.874951  1687 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:04.875147  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:04.876364  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:04.853526  4998 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:04.877563  4998 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.877749  4823 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:04.878116  4998 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:04.878414  4823 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:04.879630  4823 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.881611  4651 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9.
I20260709 13:01:04.882081  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:04.882448  4651 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:04.882840  4827 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:04.883333  4651 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:04.884546  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:04.877604  4986 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11.
I20260709 13:01:04.887480  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:04.889012  4996 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:04.813159  4944 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.889699  4654 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:04.889770  4996 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:04.890269  4654 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.890738  4654 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.891810  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:04.884703  4651 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:04.898694  1657 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.600s (SENT) [suppressed 49 similar messages]
I20260709 13:01:04.876395  1691 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:04.900104  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.846359  5017 raft_consensus.cc:2764] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:01:04.908488  1684 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:04.919054  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:04.920228  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:04.920445  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:04.923456  1610 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:01:04.923732  1616 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:04.923864  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:04.909567  1507 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:04.851120  4661 raft_consensus.cc:2764] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:01:04.924470  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:04.911865  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:04.874092  4946 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.871522  4949 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:04.928602  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:04.927764  1536 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:04.891185  4996 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.934585  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:04.940279  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:04.944061  1617 raft_consensus.cc:2376] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10. Re-sending same reply.
I20260709 13:01:04.904776  4651 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:04.950042  1689 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:04.958894  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:04.963801  1616 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:04.964704  1614 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:04.915994  4986 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:04.965941  2203 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:04.973414  1536 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:04.907226  5019 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:04.925624  4755 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:04.921944  4953 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:04.993491  1609 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:04.938532  4533 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:05.076968  4979 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:05.077636  4979 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:05.078158  5019 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:04.965263  5009 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:01:05.080485  5019 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:05.078784  4979 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.081699  4369 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:05.081462  4998 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.082093  4369 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:05.082823  1613 raft_consensus.cc:2368] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10.
I20260709 13:01:04.951383  4999 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.083695  4998 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.083773  4999 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:04.966156  4651 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:04.965221  4916 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:04.846365  4871 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:04.970746  4827 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:05.020572  1657 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.548s (SENT)
I20260709 13:01:05.095999  1616 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:05.096244  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 75710.8us
I20260709 13:01:04.943641  4668 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.949990  4946 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.024670  1508 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:05.080435  5009 consensus_queue.cc:1243] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:01:05.103551  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 78970.2us
I20260709 13:01:04.988740  5013 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:01:04.965050  4949 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:04.982723  4861 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:05.028944  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:05.006258  4876 consensus_queue.cc:1059] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:01:05.032279  4943 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:05.033356  5012 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:01:05.033954  4983 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.034098  1686 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:01:05.034469  1692 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:05.100762  4827 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.030557  4944 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.038930  1689 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:05.053869  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 128604us
W20260709 13:01:05.065976  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 100121us
I20260709 13:01:05.072460  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:05.123577  1614 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.069103  4628 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:01:05.075426  4972 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.409s	user 0.005s	sys 0.000s
I20260709 13:01:05.075812  4986 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:05.125049  5017 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:05.125438  4986 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:05.128110  4811 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.076265  4892 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:05.076340  4755 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:05.116088  4869 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:05.133702  4755 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.134145  4869 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.130434  4983 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.013135  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:05.137667  1617 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:05.089712  1609 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:05.130650  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:05.092621  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:05.089174  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:05.098623  1657 leader_election.cc:336] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.844s (SENT)
I20260709 13:01:05.082221  4979 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:05.144614  1687 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:05.106186  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:05.107841  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:05.108580  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:05.109081  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:05.117065  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 88197.2us
I20260709 13:01:05.114718  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:05.128759  4811 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.130365  5033 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:05.107406  5019 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.141705  4973 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:01:05.142194  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 43621.7us
I20260709 13:01:05.147696  1508 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:05.174618  1616 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:01:05.177206  1508 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:05.153471  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:05.148617  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:05.153148  2236 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:05.179688  1609 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:05.170607  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.150533  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:05.184973  1617 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 13:01:05.186286  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:05.188155  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:05.188789  1689 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:05.188778  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:05.189636  1609 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:01:05.190979  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:05.131101  4827 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:05.195621  1614 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:05.196574  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:05.172289  4876 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:05.191745  4766 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:05.172233  4980 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:05.178203  4628 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:05.131985  4823 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:01:05.171144  4951 consensus_queue.cc:1059] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 13:01:05.149870  4944 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 13:01:05.182592  4999 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:05.270632  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 93539.4us
I20260709 13:01:05.183032  4807 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:01:05.286952  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:05.191745  4811 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.189218  4892 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:01:05.297538  5010 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.153s
I20260709 13:01:05.298264  5031 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.100s
I20260709 13:01:05.300107  4668 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.331543  4973 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:05.334694  5041 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:05.335390  5041 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.336498  5031 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:05.301537  4861 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:05.337085  5031 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.339794  5031 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.341928  4946 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.342285  4949 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.344929  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:05.345770  1609 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:05.346663  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:05.347224  1609 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:05.255028  4998 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:05.256676  4766 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.306039  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:05.200305  5019 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.213585  5033 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:05.306190  1508 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:05.323458  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:05.324857  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.370481  1609 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:05.326337  4876 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:05.192401  4830 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 13:01:05.351850  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:05.352941  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:05.359802  1509 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.650s (SENT)
W20260709 13:01:05.409497  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 103404us
W20260709 13:01:05.409921  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50170.6us
I20260709 13:01:05.271296  4980 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:05.287487  5011 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:05.360610  4949 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.364667  4946 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:05.329216  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:05.388742  5033 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:05.410315  1508 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:01:05.411187  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 260716us
I20260709 13:01:05.372263  5031 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.366999  4766 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:05.465740  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 55504.5us
I20260709 13:01:05.475193  1508 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:01:05.364727  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 166605us
I20260709 13:01:05.475445  1657 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:05.341252  4983 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.469799  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:05.479427  1508 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:05.483526  1616 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:05.489290  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:05.490617  2237 raft_consensus.cc:2410] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:01:05.493678  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:05.466516  4949 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.496799  4949 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.456869  5019 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.503485  1609 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:05.445191  4971 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.507092  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:05.511605  1616 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:05.514858  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:05.469141  5011 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:05.364284  4654 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.469746  4807 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:05.514447  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:05.513173  1616 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:05.497380  4949 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:05.502942  5007 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.008s	sys 0.008s
I20260709 13:01:05.526191  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:05.454532  4977 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:01:05.473515  4946 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:05.526522  4951 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:05.562557  5007 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.011s	sys 0.015s
I20260709 13:01:05.558161  4951 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.521306  4906 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:05.506891  4944 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:05.545976  4807 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:05.555828  4949 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.509052  4990 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:01:05.510761  5019 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:05.481643  5031 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.591602  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:05.603670  1536 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:05.560770  4861 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:05.574469  5011 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:05.609833  5011 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.617380  1536 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
W20260709 13:01:05.618641  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 126104us
I20260709 13:01:05.621557  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:05.621740  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:05.622380  1541 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:05.544551  4998 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:01:05.581666  4944 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:05.522321  4971 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.591950  4949 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.634371  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:05.586356  4755 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:05.598100  5042 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.638319  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:05.629985  5052 consensus_queue.cc:1059] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:01:05.651034  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:05.651736  1541 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:05.636130  4944 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:05.584477  4651 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.661556  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:05.566861  4980 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:05.666946  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 633) took 1907 ms. Trace:
I20260709 13:01:05.644150  5051 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.668641  1609 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:05.667433  1543 rpcz_store.cc:276] 0709 13:01:03.759818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:03.765541 (+  5723us) service_pool.cc:224] Handling call
0709 13:01:03.765919 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:04.398719 (+632800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:05.666720 (+1268001us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b54001fc7d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:05.666888 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1372139264}
I20260709 13:01:05.640286  4755 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.675458  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:05.652395  5011 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.680459  4654 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.681239  4747 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:05.690304  5031 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.691743  5059 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:05.692380  5059 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.699018  4651 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:05.704941  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.705832  5055 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:05.706314  5055 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:05.707095  5013 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.708098  4951 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:05.707794  5013 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.708703  4951 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:05.709241  4951 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:05.670627  5053 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:05.723079  5013 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.710453  4951 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.707147  5055 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.726270  4755 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:05.727999  4973 raft_consensus.cc:3055] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:05.729817  5059 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:05.741394  4747 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.649693  5042 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:05.742424  5040 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:05.744690  4906 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:05.678975  5051 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.752518  4971 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.753890  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:05.756448  4869 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.762534  4369 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:05.765717  4827 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.506s	user 0.011s	sys 0.000s
I20260709 13:01:05.777391  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:05.829998  1692 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:05.741959  1609 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:05.807042  1508 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:05.830456  1609 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:05.822849  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:05.700620  4807 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.825264  1613 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:05.832333  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:05.833047  1539 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:05.833743  5048 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.195s	user 0.001s	sys 0.002s
W20260709 13:01:05.834936  5046 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.183s	user 0.006s	sys 0.000s
I20260709 13:01:05.811867  4661 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.836509  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:05.836366  4661 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.837841  5011 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:05.838442  5011 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:05.838831  4661 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.807157  4973 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:05.839969  4973 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.840497  5011 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:05.841041  5011 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:05.841194  4973 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:05.841889  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:05.842146  1534 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:05.845046  1541 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:05.845263  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:05.847647  4952 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:05.849920  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:05.850872  1542 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:05.833534  1540 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 4. (term mismatch)
I20260709 13:01:05.852087  1540 pending_rounds.cc:85] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 3
I20260709 13:01:05.852421  1540 pending_rounds.cc:107] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 13:01:05.852717  1540 raft_consensus.cc:2889] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:05.850906  1613 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:05.854275  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:05.854738  1539 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:05.854923  1613 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:05.809897  5041 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.849159  4827 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:05.865906  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:05.828505  5040 consensus_queue.cc:1243] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:05.866519  5019 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:05.867218  4369 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:05.871409  5043 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:05.872182  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:05.872485  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:05.874583  4995 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:05.875288  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:05.876199  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:05.876340  1686 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:01:05.821974  4906 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:05.876966  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:05.878897  5042 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:05.879526  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:05.822203  4747 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:05.882051  5029 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.684s	user 0.003s	sys 0.008s
W20260709 13:01:05.882302  4916 consensus_meta.cc:287] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.781s	user 0.011s	sys 0.000s
I20260709 13:01:05.883371  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:05.887293  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:05.888425  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 65701.7us
I20260709 13:01:05.891188  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:05.891831  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 230407us
I20260709 13:01:05.891683  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:05.849443  4983 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:05.896507  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:05.957633  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 482250us
I20260709 13:01:06.021622  1542 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:06.022096  1541 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:06.022138  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 134907us
I20260709 13:01:06.021656  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:06.024554  2237 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:06.025600  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:05.897886  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:06.026207  2237 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:05.908882  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:05.906870  1613 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:06.027215  1539 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:06.020871  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:06.028522  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:06.027168  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:06.029235  1617 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:06.029353  2237 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:01:06.030390  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:06.031514  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:06.031622  1508 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:06.023617  2236 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:06.032582  1583 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:06.034698  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:06.035447  1613 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:06.036912  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:06.037689  1617 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:06.037370  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:06.040113  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:05.892967  5013 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:06.043576  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:06.045746  1690 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:06.046882  1508 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:06.047503  2203 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:06.048069  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:06.050654  1613 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:06.051632  5039 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:06.052328  5039 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:06.053263  5039 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.023648  5043 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:06.055068  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:06.024820  4995 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:06.033746  4916 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:06.037412  5019 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:06.061775  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:06.063328  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:06.068854  1692 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:06.066097  1534 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:06.077050  1541 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:06.087574  2229 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:06.089208  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:06.059643  4668 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:06.054701  4953 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:06.117460  4990 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:06.118599  1613 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:06.157127  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:06.068349  5019 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.165292  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2399) took 1299 ms. Trace:
I20260709 13:01:06.119977  5044 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:06.120345  5065 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:06.122195  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64231.9us
I20260709 13:01:06.054646  4870 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:01:06.079164  5047 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.183s	user 0.003s	sys 0.002s
W20260709 13:01:06.089272  5060 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.109s	user 0.004s	sys 0.001s
I20260709 13:01:06.073894  4871 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:06.119463  1534 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:06.157899  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:06.088949  4980 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.102s
I20260709 13:01:06.162469  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:06.077749  5051 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:06.198684  5035 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:01:06.237308  1613 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:01:06.157194  4953 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.165895  1685 rpcz_store.cc:276] 0709 13:01:04.866099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.866642 (+   543us) service_pool.cc:224] Handling call
0709 13:01:04.866986 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:04.908502 (+ 41516us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:04.928437 (+ 19935us) log.cc:844] Serialized 14 byte log entry
0709 13:01:05.052119 (+123682us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:05.052174 (+    55us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:05.052220 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.164893 (+1112673us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b54002ce054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:06.164961 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:06.165202 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4743,"prepare.queue_time_us":25157,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":4848,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"spinlock_wait_cycles":94856448,"thread_start_us":118271,"threads_started":1,"wal-append.queue_time_us":138662,"wal-append.run_cpu_time_us":1489,"wal-append.run_wall_time_us":1007144}
I20260709 13:01:06.042349  5059 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
W20260709 13:01:06.160581  4979 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.018s	user 0.010s	sys 0.004s
I20260709 13:01:06.125557  5011 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:06.155207  4668 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:06.210562  1687 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:06.213518  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:06.214972  1539 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:06.215240  1541 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:06.246137  1534 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:06.247016  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:06.079056  5013 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:06.215866  1538 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:06.179426  5019 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:06.154834  5040 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 13:01:06.085803  5033 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:06.232668  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.195747  5044 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:06.160527  5043 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:06.244779  1683 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:01:06.250556  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:06.266988  1683 pending_rounds.cc:85] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 7
I20260709 13:01:06.267293  1683 pending_rounds.cc:107] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:01:06.221074  4871 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:06.249204  4668 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:06.121394  4906 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.255944  1541 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
W20260709 13:01:06.244727  4766 consensus_meta.cc:287] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.742s	user 0.008s	sys 0.004s
I20260709 13:01:06.241529  4953 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:06.285857  5044 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.288331  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:06.248492  4949 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.603s	user 0.005s	sys 0.005s
I20260709 13:01:06.267546  1683 raft_consensus.cc:2889] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:06.289803  4949 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:06.291198  5068 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:01:06.252513  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:06.284086  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:06.182876  4990 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:06.292608  1543 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:06.292474  2231 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:06.285580  4766 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:06.249573  4827 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:06.249702  5011 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:06.292541  1541 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:06.519383  4949 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:06.520365  1541 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:06.263727  4747 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:06.331875  1540 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:06.536242  1540 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:06.314476  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.565809  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 658) took 1120 ms. Trace:
I20260709 13:01:06.583020  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:06.370383  1687 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:06.611573  1689 rpcz_store.cc:276] 0709 13:01:05.445320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.445552 (+   232us) service_pool.cc:224] Handling call
0709 13:01:05.445962 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:05.465763 (+ 19801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:05.465800 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:05.474738 (+  8938us) log.cc:844] Serialized 26 byte log entry
0709 13:01:05.635932 (+161194us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:05.635990 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:05.636123 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:06.565545 (+929422us) raft_consensus.cc:1685] finished
0709 13:01:06.565574 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:06.565751 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15617,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":11504,"thread_start_us":71770,"threads_started":1,"wal-append.queue_time_us":167596}
I20260709 13:01:06.330022  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:06.614858  1613 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:06.297245  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:06.616395  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 319233us
I20260709 13:01:06.361020  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:06.617730  1538 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:06.255914  5034 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:06.328871  1534 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:01:06.508741  5067 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.263s	user 0.000s	sys 0.000s
I20260709 13:01:06.634545  1684 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:01:06.509351  5072 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.319s	user 0.000s	sys 0.000s
I20260709 13:01:06.628185  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:06.527355  4990 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:06.630695  1541 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 13:01:06.424600  4876 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.956s	user 0.012s	sys 0.019s
W20260709 13:01:06.514946  5073 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.186s	user 0.000s	sys 0.000s
W20260709 13:01:06.508105  5066 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
W20260709 13:01:06.346424  5070 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:01:06.281908  4871 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:06.622978  4944 consensus_meta.cc:287] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.954s	user 0.011s	sys 0.001s
I20260709 13:01:06.497511  5051 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.346794  4906 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:06.631165  5034 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.627246  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:06.646982  4876 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.191s	user 0.014s	sys 0.025s
I20260709 13:01:06.626056  5065 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:06.497210  5017 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:06.651237  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 565) took 1174 ms. Trace:
I20260709 13:01:06.572662  4949 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.281694  5056 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.369s
I20260709 13:01:06.655334  4951 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:06.618927  4823 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:06.626672  4755 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:06.626533  5063 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:06.733845  5063 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.736649  5063 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:06.736753  1613 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:06.619234  4918 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.737738  4918 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:06.740404  4918 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:06.659222  5049 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:01:06.415843  4980 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:06.671001  2229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 878) took 1578 ms (client timeout 1751 ms). Trace:
W20260709 13:01:06.745586  2229 rpcz_store.cc:269] 0709 13:01:05.092237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.092364 (+   127us) service_pool.cc:224] Handling call
0709 13:01:06.670796 (+1578432us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b5400730fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:06.670937 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2181425536}
W20260709 13:01:06.668195  4973 consensus_meta.cc:287] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.827s	user 0.009s	sys 0.001s
I20260709 13:01:06.707916  5044 consensus_queue.cc:1059] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:06.703963  1616 rpcz_store.cc:276] 0709 13:01:05.477312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.546779 (+ 69467us) service_pool.cc:224] Handling call
0709 13:01:05.547228 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:05.549348 (+  2120us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:05.549395 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:05.715577 (+166182us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b2c0005b3b8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:01:05.716187 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 13:01:06.053875 (+337688us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:06.053920 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.053997 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:06.650956 (+596959us) raft_consensus.cc:1685] finished
0709 13:01:06.650982 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:06.651175 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6284,"raft.queue_time_us":43127,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":17301,"spinlock_wait_cycles":9954688,"thread_start_us":337101,"threads_started":1,"wal-append.queue_time_us":373282}
I20260709 13:01:06.697628  4876 raft_consensus.cc:697] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:06.726651  4668 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.278488  5013 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:06.690675  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 63526.6us
I20260709 13:01:06.766213  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:06.720708  4369 consensus_meta.cc:287] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.717s	user 0.000s	sys 0.008s
I20260709 13:01:06.641635  4827 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 13:01:06.704478  5041 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:06.712931  5076 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.442s	user 0.000s	sys 0.008s
I20260709 13:01:06.752889  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:06.757566  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 72355us
I20260709 13:01:06.700950  5034 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:06.772785  2229 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:06.775018  1614 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:01:06.679540  5080 consensus_queue.cc:1059] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.099s
I20260709 13:01:06.699743  4944 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.700287  5051 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:06.784945  4827 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:06.772485  5041 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:06.786330  5041 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:06.772595  5065 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:01:06.781522  4951 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.790362  4951 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.791857  5034 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.768952  5063 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:06.794657  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2010) took 1054 ms. Trace:
I20260709 13:01:06.810029  2229 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:06.810349  1617 rpcz_store.cc:276] 0709 13:01:05.740480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.038361 (+297881us) service_pool.cc:224] Handling call
0709 13:01:06.038674 (+   313us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:06.058823 (+ 20149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:06.058858 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:06.073773 (+ 14915us) log.cc:844] Serialized 52 byte log entry
0709 13:01:06.216607 (+142834us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:01:06.216687 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.216835 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:06.794341 (+577506us) raft_consensus.cc:1685] finished
0709 13:01:06.794370 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:06.794577 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35863,"raft.queue_time_us":27062,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":47374,"thread_start_us":102856,"threads_started":1,"wal-append.queue_time_us":157408}
I20260709 13:01:06.711828  4949 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.796285  4668 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:06.824898  5080 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.796902  5063 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:06.794158  5022 raft_consensus.cc:3055] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:06.826406  5022 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:06.826275  5063 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.827334  5022 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:06.828652  5022 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:06.829936  4766 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:06.833642  1535 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:06.838384  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:06.711601  5017 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.750001  4823 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.793388  4951 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:06.755553  4980 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.855456  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2000) took 1952 ms. Trace:
I20260709 13:01:06.856114  1612 rpcz_store.cc:276] 0709 13:01:04.903450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.903681 (+   231us) service_pool.cc:224] Handling call
0709 13:01:04.920011 (+ 16330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:04.935775 (+ 15764us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:04.982091 (+ 46316us) log.cc:844] Serialized 14 byte log entry
0709 13:01:06.156743 (+1174652us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b1800087be0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a 00007fe08faf6a2a
0709 13:01:06.416359 (+259616us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:06.421235 (+  4876us) log.cc:844] Serialized 52 byte log entry
0709 13:01:06.421395 (+   160us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:06.709704 (+288309us) log.cc:844] Serialized 14 byte log entry
0709 13:01:06.851934 (+142230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.852060 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:06.855167 (+  3107us) raft_consensus.cc:1685] finished
0709 13:01:06.855190 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:06.855382 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4244,"prepare.queue_time_us":410326,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":1189,"raft.queue_time_us":17333,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":27775,"spinlock_wait_cycles":6766336,"thread_start_us":685141,"threads_started":3,"wal-append.queue_time_us":1649898}
I20260709 13:01:06.853480  4369 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.862001  4369 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.863987  4369 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:06.861274  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:06.865293  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:06.832492  5080 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.866412  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:06.867338  5080 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:06.867935  2233 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:06.868736  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:06.870378  2233 raft_consensus.cc:2410] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:01:06.870590  1507 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:06.872834  5080 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:06.873235  5080 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:06.873615  5080 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:06.877310  5011 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:06.877948  1609 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:06.878893  4953 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.592s	user 0.011s	sys 0.000s
I20260709 13:01:06.879354  4953 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:06.879985  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:06.880952  1610 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:06.883224  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:06.883535  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:06.878922  1609 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:06.884917  2231 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:06.886535  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:06.887204  2231 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:06.796285  5034 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:06.830793  4876 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.894378  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:06.890267  4805 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.852993  5042 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:06.902041  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:06.908972  1609 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:06.909624  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2008) took 1170 ms. Trace:
W20260709 13:01:06.903381  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 256349us
I20260709 13:01:06.897682  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:06.890115  5053 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.910303  1609 rpcz_store.cc:276] 0709 13:01:05.739689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.741456 (+  1767us) service_pool.cc:224] Handling call
0709 13:01:05.741788 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:06.877561 (+1135773us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b540009dba0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa6a06f 00007fe08fae2f60 00007fe08fae47de 00007fe08faeeb49 00007fe08faef416 00007fe08faebd8f 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62
0709 13:01:06.909352 (+ 31791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.909572 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144238976}
I20260709 13:01:06.897281  2231 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:06.899626  4766 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:06.929356  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:06.930058  2233 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:06.898249  5019 consensus_meta.cc:287] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.632s	user 0.009s	sys 0.000s
W20260709 13:01:06.931025  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 164907us
I20260709 13:01:06.931826  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:06.910562  4807 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:06.899394  5049 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:01:06.891572  5011 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:01:06.907500  4651 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:06.909976  4951 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.938787  1509 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:06.936060  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:06.947101  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:06.950340  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:06.907307  4805 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:06.960963  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:06.973130  1609 raft_consensus.cc:2376] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8. Re-sending same reply.
I20260709 13:01:06.966792  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:06.971154  1534 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:06.963963  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:06.913772  5042 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:06.943770  5065 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:07.025197  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:06.928377  5053 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.948235  4827 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:07.054926  1612 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:07.024655  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:07.039669  2233 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:06.939118  4766 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:07.060428  1538 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:07.085001  5104 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.939006  5019 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:07.053984  5009 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
W20260709 13:01:07.090817  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 141466us
I20260709 13:01:07.091495  5051 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.085590  4953 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.092972  4823 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.094452  4805 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.095757  4980 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:07.060063  5082 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 13:01:06.956923  4973 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.137s
I20260709 13:01:07.106891  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:06.972721  5088 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.118s
I20260709 13:01:07.118439  1542 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:07.027725  5057 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.082263  5091 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.114663  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:07.085258  5098 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:06.956491  4651 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:06.971197  4951 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.105218  4949 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.090390  5094 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:07.104537  5101 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:07.135730  1657 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:07.135941  5100 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:07.121618  5057 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:07.121565  5093 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:01:07.141378  5094 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.134316  5098 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.126873  5091 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:07.105561  5104 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.142035  5017 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.148306  5095 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:07.152568  5098 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.154170  5019 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.159170  5104 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.173877  5095 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:07.148895  5100 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:07.175748  5100 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.176559  5095 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:07.151919  5057 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.154736  5102 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:07.150278  5094 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:07.180186  2235 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
W20260709 13:01:07.180249  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.181177  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1028) took 1128 ms. Trace:
I20260709 13:01:07.181805  2235 rpcz_store.cc:276] 0709 13:01:06.052838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.869941 (+817103us) service_pool.cc:224] Handling call
0709 13:01:07.181098 (+311157us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:07.157771  5081 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.408s	user 0.003s	sys 0.000s
W20260709 13:01:07.183732  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.183987  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:07.187438  1690 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:07.187815  1683 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:07.185824  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:07.147719  4980 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:01:07.187350  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.149453  4766 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:07.147461  4651 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.191397  2235 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:07.147133  4951 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.157017  5091 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.212124  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:07.196493  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:07.214290  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 109771us
I20260709 13:01:07.214687  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:07.195762  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1025) took 1143 ms. Trace:
I20260709 13:01:07.196589  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:07.196964  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:07.218674  2235 rpcz_store.cc:276] 0709 13:01:06.052004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.188996 (+1136992us) service_pool.cc:224] Handling call
0709 13:01:07.189289 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.195418 (+  6129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.195698 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:07.197642  4946 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.199536  4755 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:07.200165  4980 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:07.200873  4871 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.201946  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:07.208581  5080 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:07.227813  1537 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:07.215098  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:07.195250  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 182752us
I20260709 13:01:07.219287  1691 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 5.
I20260709 13:01:07.232292  2203 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:07.225064  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:07.223208  1683 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:07.227212  1686 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:07.219287  1690 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:01:07.236282  2235 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:01:07.188817  5065 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:07.237226  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1026) took 1185 ms. Trace:
I20260709 13:01:07.237959  2235 rpcz_store.cc:276] 0709 13:01:06.052329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.219665 (+1167336us) service_pool.cc:224] Handling call
0709 13:01:07.235992 (+ 16327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:07.236882 (+   890us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.237151 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:07.242718  1509 leader_election.cc:400] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10.
I20260709 13:01:07.244402  1509 leader_election.cc:403] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:01:07.229748  5080 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:07.247566  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:07.248394  2203 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:07.250340  5106 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10.
I20260709 13:01:07.250401  5088 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:07.230110  4766 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:07.251778  2235 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:07.253494  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1029) took 1200 ms. Trace:
I20260709 13:01:07.252193  5088 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:07.254406  5085 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:07.255892  5085 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:07.243165  4651 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:07.260020  2235 rpcz_store.cc:276] 0709 13:01:06.053091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.246124 (+1193033us) service_pool.cc:224] Handling call
0709 13:01:07.251471 (+  5347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:07.252713 (+  1242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.253234 (+   521us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:07.245993  5102 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:07.246220  5057 consensus_queue.cc:1059] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:01:07.261816  4980 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.265046  5095 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:07.267936  2229 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:07.270889  2236 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:07.271543  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1033) took 1217 ms. Trace:
I20260709 13:01:07.247618  5017 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:01:07.271834  2236 rpcz_store.cc:276] 0709 13:01:06.054075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.270266 (+1216191us) service_pool.cc:224] Handling call
0709 13:01:07.270573 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.271305 (+   732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.271491 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:07.235718  4951 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:07.273823  4951 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.274745  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1967) took 1381 ms. Trace:
I20260709 13:01:07.275152  1543 rpcz_store.cc:276] 0709 13:01:05.893329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.893480 (+   151us) service_pool.cc:224] Handling call
0709 13:01:06.030157 (+136677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:06.566437 (+536280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.274501 (+708064us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54004187d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:07.274686 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603129088}
I20260709 13:01:07.271843  4766 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:07.276798  4951 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.273347  2236 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 13:01:07.279183  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1038) took 1075 ms. Trace:
I20260709 13:01:07.279559  2236 rpcz_store.cc:276] 0709 13:01:06.203541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.272757 (+1069216us) service_pool.cc:224] Handling call
0709 13:01:07.273056 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.278905 (+  5849us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.279128 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:07.261843  2235 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:07.280951  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1032) took 1227 ms. Trace:
I20260709 13:01:07.281333  2235 rpcz_store.cc:276] 0709 13:01:06.053839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.261059 (+1207220us) service_pool.cc:224] Handling call
0709 13:01:07.261379 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.280689 (+ 19310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.280898 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:07.282608  2235 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:07.283790  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 48809.4us
I20260709 13:01:07.285290  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:07.283237  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1042) took 1055 ms. Trace:
I20260709 13:01:07.249872  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:07.287118  2235 rpcz_store.cc:276] 0709 13:01:06.228304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.282165 (+1053861us) service_pool.cc:224] Handling call
0709 13:01:07.282447 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.283023 (+   576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.283187 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:07.243193  4946 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:07.201321  5041 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:07.294359  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 44606.8us
I20260709 13:01:07.300465  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:07.247449  5059 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:01:07.277156  4766 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.302541  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:07.245585  4876 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:01:07.295307  5080 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:07.296273  4651 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:07.306932  2236 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:07.315474  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.304366  5098 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:07.302807  5104 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:01:07.308501  4823 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:07.321065  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:07.322584  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1039) took 1118 ms. Trace:
W20260709 13:01:07.322738  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.301692  4906 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:01:07.326704  2236 rpcz_store.cc:276] 0709 13:01:06.203793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.280374 (+1076581us) service_pool.cc:224] Handling call
0709 13:01:07.306759 (+ 26385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.322210 (+ 15451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.322508 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:07.331765  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:07.331848  4876 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:07.335098  5107 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.297492  5065 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.338034  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:07.338076  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:07.335805  5107 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:07.339355  1616 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
W20260709 13:01:07.340442  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 90660.5us
I20260709 13:01:07.320577  5080 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.344022  1617 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:07.323424  5100 raft_consensus.cc:2764] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:01:07.341928  1612 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:01:07.341226  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:07.326516  5104 consensus_queue.cc:1243] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:01:07.347440  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:07.348034  1611 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:07.348304  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:07.350584  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:07.352931  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:07.353111  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:07.347123  5059 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:01:07.350522  4946 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:07.356777  4946 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.357185  4946 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:07.359248  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 887) took 1341 ms. Trace:
I20260709 13:01:07.359673  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:07.359673  2231 rpcz_store.cc:276] 0709 13:01:06.018349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.995916 (+977567us) service_pool.cc:224] Handling call
0709 13:01:06.996452 (+   536us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:07.111765 (+115313us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:07.111799 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:07.140677 (+ 28878us) log.cc:844] Serialized 26 byte log entry
0709 13:01:07.307018 (+166341us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:07.307057 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.307128 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:07.358973 (+ 51845us) raft_consensus.cc:1685] finished
0709 13:01:07.358996 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:07.359173 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27610,"raft.queue_time_us":115601,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":3274,"thread_start_us":192226,"threads_started":2,"wal-append.queue_time_us":169890}
I20260709 13:01:07.352088  5039 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.360301  5022 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:07.377830  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:07.378129  5022 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:07.378728  5104 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:07.345719  5080 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:07.379129  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:07.380589  1543 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:01:07.380860  1538 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:07.340057  4876 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:07.328855  4823 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.389508  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:07.390812  1612 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:07.376684  5039 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:07.385532  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.403420  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:07.405354  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:07.352447  4980 consensus_queue.cc:1059] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:01:07.352195  4651 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 13:01:07.415544  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:07.382720  4871 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.395252  4755 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:07.419596  2231 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:07.417410  5059 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:01:07.415658  5094 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.421653  4951 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:01:07.428442  1534 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:07.382565  5100 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 13:01:07.366716  5098 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:01:07.424154  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:07.430439  2231 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 5.
I20260709 13:01:07.431619  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:07.420254  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:07.382311  4949 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:01:07.435752  2231 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:07.407760  5065 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.439431  1612 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:07.418673  5088 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:07.444345  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:07.429522  5091 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.422307  5041 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.455845  2235 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:07.452765  1611 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:07.419689  5119 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:07.417093  5112 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:07.461292  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 49769.6us
I20260709 13:01:07.467010  1657 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:07.468044  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:07.468312  1657 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:07.468928  2231 raft_consensus.cc:2376] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7. Re-sending same reply.
I20260709 13:01:07.471798  4980 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:07.436475  4755 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.458119  5106 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:07.477726  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 90467us
I20260709 13:01:07.482676  5098 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:07.482473  5119 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:07.483423  5098 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.484007  5119 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:07.484532  5119 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:07.485066  5119 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:07.457994  4876 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:07.458571  5091 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:07.485558  5094 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.452548  5065 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:07.487195  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.486109  5100 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:01:07.512246  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 44035.6us
I20260709 13:01:07.566033  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:07.488890  4946 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:07.475087  4906 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:01:07.587662  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:07.507129  4651 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:07.591336  1543 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:07.591490  4651 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:07.589948  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:07.509503  5125 raft_consensus.cc:3055] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:07.595733  5115 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:01:07.595180  4651 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:07.596513  5113 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:01:07.613685  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:07.614475  1690 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:07.617975  5042 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.618734  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1969) took 1610 ms. Trace:
I20260709 13:01:07.619203  1542 rpcz_store.cc:276] 0709 13:01:06.008424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.028464 (+ 20040us) service_pool.cc:224] Handling call
0709 13:01:06.045486 (+ 17022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.118960 (+1073474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.618437 (+499477us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400418cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:07.618662 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2467078400}
I20260709 13:01:07.624199  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:07.628006  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:07.592458  5107 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.598347  5125 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:07.510373  5106 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:07.626371  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:07.583911  5124 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:01:07.641541  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:07.644752  5065 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.629582  5094 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:01:07.642155  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:07.643901  1538 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:01:07.647698  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:07.630903  4980 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.652928  5065 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:07.636439  4651 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:07.648267  1690 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:07.634191  5123 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.634366  5100 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.637954  5039 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.695065  1691 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:01:07.666697  1507 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:07.667299  1686 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:01:07.703445  1686 raft_consensus.cc:606] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:07.672232  2229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1035) took 1469 ms (client timeout 1701 ms). Trace:
W20260709 13:01:07.704316  2229 rpcz_store.cc:269] 0709 13:01:06.202584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.838036 (+635452us) service_pool.cc:224] Handling call
0709 13:01:07.672055 (+834019us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b5400599cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:07.672170 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":985350400}
I20260709 13:01:07.674717  5043 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:07.675809  1538 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:07.679838  5106 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.652406  4766 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:07.681283  5091 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.683224  4946 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.683364  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1953) took 1852 ms. Trace:
I20260709 13:01:07.684942  5125 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.687263  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:07.642920  5107 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:07.690356  4823 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:07.658125  5127 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.649389  5094 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:07.729887  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:07.727725  1535 rpcz_store.cc:276] 0709 13:01:05.830776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.830913 (+   137us) service_pool.cc:224] Handling call
0709 13:01:05.845055 (+ 14142us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:06.834262 (+989207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:07.683125 (+848863us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b54003e1f54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:07.683303 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2272869760}
I20260709 13:01:07.732156  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:07.723134  5033 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:07.705034  5098 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.695055  5088 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:07.730608  5107 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:07.663038  4980 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.758492  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:07.705482  5100 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:07.746380  5094 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:07.730782  5059 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:01:07.770493  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:07.771745  1691 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:01:07.754072  5125 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:07.748322  4755 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:07.695791  5123 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.724614  4766 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.783490  1692 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:01:07.762243  5107 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:07.765549  5033 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:07.806818  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:07.807915  1686 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:01:07.811740  4980 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.815938  5123 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:07.818401  5128 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:01:07.760206  5088 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.827214  5100 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:07.842890  2201 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:07.847236  4766 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.747045  5127 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.809242  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:07.862454  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:07.865120  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:07.865772  2231 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:07.866268  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:07.878346  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:07.879737  2229 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:07.868947  5059 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:07.894798  5059 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:07.868031  5080 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:07.920796  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:07.944662  5127 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.873873  5119 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.911927  5059 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:07.922996  2231 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:07.924510  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:07.959951  1543 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: [4.3-9.4]
I20260709 13:01:07.958849  1686 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:07.962056  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:07.920688  2229 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:07.965050  1686 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 6: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:07.973941  1583 connection.cc:256] RPC call timeout handler was delayed by -1.1970007159999909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 13:01:07.976223  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 94464.7us
I20260709 13:01:07.976696  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:07.978693  1686 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:07.986506  2232 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:07.999629  1583 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:08.005486  1686 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:01:08.006441  5088 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:08.004200  5139 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.009979  5139 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:08.012260  5139 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:08.012462  2201 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:08.014019  5080 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:08.014067  5039 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:08.014802  5039 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.015164  5039 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:08.015710  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1043) took 1766 ms. Trace:
I20260709 13:01:08.016249  2235 rpcz_store.cc:276] 0709 13:01:06.249030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.289800 (+1040770us) service_pool.cc:224] Handling call
0709 13:01:07.290212 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.456504 (+166292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.015355 (+558851us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54007102d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:08.015625 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380346624}
I20260709 13:01:07.978863  4871 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.004998  4661 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:08.024163  5063 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:08.025931  1685 raft_consensus.cc:2410] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:08.020888  5107 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:07.980185  5119 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:07.909170  5033 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:01:08.034768  5133 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:01:07.977239  5091 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:08.035307  5078 log.cc:927] Time spent T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.274s	user 0.003s	sys 0.003s
I20260709 13:01:08.036528  1534 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:08.037546  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 570) took 1787 ms. Trace:
I20260709 13:01:08.040993  2232 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:01:08.043115  1686 raft_consensus.cc:606] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:08.066234  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1506) took 2593 ms. Trace:
I20260709 13:01:08.068535  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2578) took 1615 ms. Trace:
I20260709 13:01:08.078747  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:08.090207  1541 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:08.086041  1610 rpcz_store.cc:276] 0709 13:01:06.250099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.295602 (+ 45503us) service_pool.cc:224] Handling call
0709 13:01:06.295901 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:06.889872 (+593971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.037176 (+1147304us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b54000bf9d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:08.037461 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1345046272}
I20260709 13:01:08.088668  1684 rpcz_store.cc:276] 0709 13:01:05.472606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.472798 (+   192us) service_pool.cc:224] Handling call
0709 13:01:05.473236 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:06.635150 (+1161914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.065835 (+1430685us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54002cd654. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:08.066143 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2669957888}
I20260709 13:01:08.093046  1542 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:01:08.084056  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.093256  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.089604  1540 rpcz_store.cc:276] 0709 13:01:06.453100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.628937 (+175837us) service_pool.cc:224] Handling call
0709 13:01:06.629226 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:06.652328 (+ 23102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:06.652377 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:06.652473 (+    96us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:06.652528 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.068158 (+1415630us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b54001d79d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:08.068242 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:08.068443 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52495616}
I20260709 13:01:08.083352  5091 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:08.071972  5080 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:08.105759  5123 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:08.113776  5123 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:08.114739  5091 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:08.095736  5106 consensus_queue.cc:1059] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:01:08.116150  5123 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:08.110749  5080 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:08.118544  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:08.119266  1540 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:01:08.118571  5080 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:08.121762  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1963) took 2252 ms. Trace:
I20260709 13:01:08.122169  1536 rpcz_store.cc:276] 0709 13:01:05.869263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.884979 (+ 15716us) service_pool.cc:224] Handling call
0709 13:01:06.120581 (+235602us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b5800003998. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:01:06.120751 (+   170us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:06.121242 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:06.121273 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:06.161916 (+ 40643us) log.cc:844] Serialized 52 byte log entry
0709 13:01:06.392955 (+231039us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:06.393034 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:06.393104 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:08.005865 (+1612761us) spinlock_profiling.cc:243] Waited 6.91 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:08.121503 (+115638us) raft_consensus.cc:1685] finished
0709 13:01:08.121525 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:08.121675 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":120,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":56865,"spinlock_wait_cycles":42562688,"thread_start_us":157960,"threads_started":1,"wal-append.queue_time_us":471101}
I20260709 13:01:08.125919  5043 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:08.126753  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.125988  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:08.128422  1685 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 13:01:08.132489  5022 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:08.135967  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:08.136857  1612 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:08.137738  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.137876  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.138806  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:08.139042  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.139739  5106 raft_consensus.cc:3055] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:08.140239  5106 raft_consensus.cc:740] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:08.140908  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:08.141062  5106 consensus_queue.cc:260] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:08.134068  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.142340  5106 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:08.142419  1686 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:01:08.142225  4952 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:08.142186  5059 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:08.143684  4944 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:08.126585  5043 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:08.146129  1657 leader_election.cc:336] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.701s (SENT)
W20260709 13:01:08.147181  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.147099  5043 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:08.140254  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.148975  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.151135  5059 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:08.152426  5107 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:08.154002  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2005) took 2869 ms. Trace:
I20260709 13:01:08.154419  1614 rpcz_store.cc:276] 0709 13:01:05.285047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.285404 (+   357us) service_pool.cc:224] Handling call
0709 13:01:05.285727 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:06.775499 (+1489772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.153745 (+1378246us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54003c8cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:08.153906 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3423932288}
I20260709 13:01:08.155102  5116 raft_consensus.cc:3055] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:08.155511  5116 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:08.111742  5019 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.157125  5019 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:08.159503  1536 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:08.159613  5004 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.160337  5004 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:08.160725  5019 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:08.139341  1612 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:08.162074  1618 raft_consensus.cc:2410] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:08.150677  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:08.163625  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.164080  1540 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:01:08.164320  1614 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:08.165122  5094 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:08.166536  1536 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:08.167361  5094 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:08.167613  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.170061  5039 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:08.078473  4871 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:08.175307  4871 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:08.181932  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:08.183161  5039 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:08.166553  4827 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:08.190109  4827 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:08.190477  4827 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:08.194205  5103 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:08.194633  5103 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:08.195528  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:08.195410  5103 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:08.196460  5103 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:08.197777  5043 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:08.197208  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:08.198969  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1965) took 2329 ms. Trace:
I20260709 13:01:08.198977  5094 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:08.199527  1537 rpcz_store.cc:276] 0709 13:01:05.869885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.885080 (+ 15195us) service_pool.cc:224] Handling call
0709 13:01:05.895097 (+ 10017us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.228290 (+1333193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.198728 (+970438us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400343ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:08.198894 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3064512896}
I20260709 13:01:08.200256  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:08.201074  5094 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:08.201824  5018 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:08.202503  5018 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:08.203275  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:08.204952  5018 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:08.206111  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.219288  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.220089  1686 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:08.208280  5094 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:08.209496  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.222005  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:08.222687  1610 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:08.213143  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:08.209944  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.223604  1614 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:01:08.210883  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:08.210943  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.210652  1438 catalog_manager.cc:5697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 6 to 7, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 7 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:08.212385  4766 raft_consensus.cc:993] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: : Instructing follower 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 13:01:08.225232  1616 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:08.225414  5044 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:08.226105  5017 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:08.225980  5044 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:08.226738  5017 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:08.228416  5044 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:08.210783  5064 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:08.213629  5099 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:08.214605  1537 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:08.185150  5053 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:08.215513  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:08.218334  1657 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:08.218725  1536 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:08.207221  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.219492  1540 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:08.235579  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.220220  1617 raft_consensus.cc:2410] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 7 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 13:01:08.236255  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.236689  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.207202  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:08.220831  1541 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:08.237550  1685 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:08.224366  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.237917  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.238440  1690 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:08.226593  1539 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:08.212327  1609 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:08.241335  1690 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:08.243012  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.245584  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.245774  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:08.246001  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.246630  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.247169  1609 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:08.241801  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:08.243903  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.250111  1610 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:08.250151  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.251482  1614 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:08.257099  1615 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:01:08.257546  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.232439  5094 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.233196  5055 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:08.266237  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.261098  1686 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:01:08.308665  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 90406.6us
W20260709 13:01:08.308465  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46467.4us
W20260709 13:01:08.324748  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 78309.4us
I20260709 13:01:08.311201  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.324982  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:08.229408  5017 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:08.327710  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.232889  5064 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:08.330832  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.251961  5127 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:08.331682  1617 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:08.261184  4628 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:08.333451  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.335145  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.254086  5053 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.338369  1685 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:08.338675  1617 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:08.317912  5019 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.339464  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:08.322079  4986 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:08.323484  5094 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:08.343242  1685 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:08.345726  5019 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:08.347688  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 343480us
I20260709 13:01:08.261011  5125 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 13:01:08.258651  5119 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 13:01:08.351853  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:08.353322  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.245136  5121 raft_consensus.cc:1081] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Signalling peer 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 13:01:08.353860  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:08.307394  5018 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 13:01:08.354950  1583 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:08.309909  5088 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:01:08.357682  4952 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:08.329353  4651 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:01:08.335458  5127 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:08.364056  1685 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:08.321834  4951 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:01:08.368748  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:08.376555  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.376816  2237 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:01:08.377170  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:08.377630  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.339701  4980 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:01:08.396045  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:08.400004  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:08.405966  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.407543  2229 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:08.407722  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.366890  4973 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:08.345328  5053 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:08.367281  4871 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:08.375564  5127 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:08.445590  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.378904  4952 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:08.451514  1615 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:08.370618  5006 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
W20260709 13:01:08.460233  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 135323us
I20260709 13:01:08.462256  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.437503  4973 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:08.445222  4951 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:08.464237  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:08.491606  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:08.433537  4827 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:08.458884  4871 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:08.471104  4973 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:08.454049  5123 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:01:08.473886  4951 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:08.458619  5091 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:01:08.550892  4986 raft_consensus.cc:697] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:08.542754  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 200212us
I20260709 13:01:08.624233  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:08.551383  4916 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.644412  1612 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
W20260709 13:01:08.653126  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.653765  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:08.649818  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:08.561919  5064 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:08.647967  4944 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.649214  4916 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:08.670686  4986 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:08.672272  5011 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:08.659837  4998 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:08.674160  5064 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:08.686266  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:08.687592  5150 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:01:08.689800  5123 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:08.695511  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.658723  5044 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:08.700204  1687 raft_consensus.cc:2376] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8. Re-sending same reply.
I20260709 13:01:08.667477  5039 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:08.764178  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:08.665387  5019 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:08.765061  1617 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:01:08.763173  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 298993us
I20260709 13:01:08.728638  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.784307  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.766650  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:08.796967  2203 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:08.798517  2203 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:08.808341  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.623628  4952 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:08.704681  4943 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:08.816828  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.704110  5064 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.817538  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.724187  5123 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:08.730363  5044 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:08.839771  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.717222  5011 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:08.657671  4871 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:08.841180  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:08.862287  1685 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:08.863274  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:08.851012  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:08.820955  5040 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.711488  4998 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:08.702489  4944 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:08.843582  5150 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 13:01:08.827744  5064 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:08.844502  5123 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:08.840574  5157 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.868062  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:09.004027  1685 raft_consensus.cc:2410] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:01:09.004977  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:09.005048  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 206553us
W20260709 13:01:08.933981  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.006276  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:08.834982  5085 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:08.997100  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 336601us
I20260709 13:01:08.841825  5053 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:08.842059  5107 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:08.865127  5019 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:08.874460  5040 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.031713  4996 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.025491  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:09.041333  4996 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.033434  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:09.043552  2233 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.030857  5107 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.045463  4996 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:09.022495  5157 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.023154  5085 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:08.846454  5167 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:08.866001  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 212338us
I20260709 13:01:08.821339  4995 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.064522  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:09.063453  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:09.087745  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 67839.7us
I20260709 13:01:09.089731  1658 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:09.070916  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.069358  5033 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.052767  5085 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.092008  2233 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:09.048334  5107 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:09.049737  4996 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.052129  5157 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:09.137923  2233 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:09.123056  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:09.129626  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:09.140616  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:09.131590  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 281267us
I20260709 13:01:09.144922  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:09.096129  5039 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:09.142532  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.151578  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:09.174909  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.118827  5095 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:09.175635  1540 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:01:09.070076  5171 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.125793  5053 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:09.063510  5019 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:09.180486  1536 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:09.177910  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.129362  4995 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:09.190994  1540 raft_consensus.cc:2410] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:09.181710  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:09.183110  2235 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:01:09.192373  2235 pending_rounds.cc:85] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 7
I20260709 13:01:09.192956  2235 pending_rounds.cc:107] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:01:09.193274  2235 raft_consensus.cc:2889] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:09.131865  5167 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:09.196063  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.143513  4991 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.198644  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.198858  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.144289  5040 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:09.069504  5011 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:09.196753  1536 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:09.185204  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:09.227262  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.072892  5144 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.107s
W20260709 13:01:09.180011  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 137162us
I20260709 13:01:09.202489  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:09.202862  1535 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:09.219623  5019 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:09.208995  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 666) took 1523 ms. Trace:
I20260709 13:01:09.234751  1692 rpcz_store.cc:276] 0709 13:01:07.685940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.686919 (+   979us) service_pool.cc:224] Handling call
0709 13:01:07.687239 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.788436 (+101197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.208707 (+1420271us) spinlock_profiling.cc:243] Waited 95.8 ms on lock 0x7b5400058f54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:09.208923 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220242304}
I20260709 13:01:09.209141  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:09.142817  5119 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 13:01:09.117247  5044 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.184029  5053 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:09.200994  5167 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:09.206244  5106 consensus_meta.cc:287] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.039s	user 0.013s	sys 0.000s
I20260709 13:01:09.246697  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:09.205824  4916 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:09.176800  5158 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:01:09.218767  1508 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:09.121600  5161 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:01:09.224720  4668 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:09.184556  5180 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.226030  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:09.257833  2233 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:09.258630  1508 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:09.220456  5099 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 13:01:09.203166  1540 raft_consensus.cc:2393] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:09.127218  5033 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.237069  1684 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:09.127172  5160 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 13:01:09.148804  4996 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.215831  4951 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.228403  2201 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:09.220695  5104 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.222462  1538 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:09.251155  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:09.226032  5095 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.224098  5168 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:09.263123  4998 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:09.247207  5053 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.268883  2235 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:09.311509  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:09.247731  5178 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:09.274235  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 133729us
I20260709 13:01:09.276443  5033 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:09.184103  5171 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.278769  1540 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:09.246502  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.284457  1684 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:01:09.249588  5167 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:09.297676  2231 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:01:09.206468  4991 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.302712  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:09.320410  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:09.321564  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:09.321622  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:09.322126  1684 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:09.322397  1691 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:09.249238  5151 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:01:09.259994  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:09.314445  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:09.273087  4944 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:09.273675  5144 raft_consensus.cc:3055] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:01:09.316751  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 88440.3us
I20260709 13:01:09.329227  2201 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:09.266372  5179 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:09.296002  5169 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:09.260459  4916 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:01:09.329265  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.333549  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:09.334162  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.334923  1540 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:09.335040  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:09.316636  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:09.330490  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 70578.1us
I20260709 13:01:09.336385  1508 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:09.259689  4668 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:09.340404  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 665) took 1654 ms. Trace:
I20260709 13:01:09.335860  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.344376  1540 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:09.344122  5183 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:09.340816  1683 rpcz_store.cc:276] 0709 13:01:07.685632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.687313 (+  1681us) service_pool.cc:224] Handling call
0709 13:01:07.687580 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:07.848005 (+160425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:07.848052 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:07.848112 (+    60us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:07.848158 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.340166 (+1492008us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54000749d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:09.340214 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.340345 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":368241536}
I20260709 13:01:09.315936  5019 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:09.346309  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:09.347131  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.347784  1540 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:09.354616  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:09.355233  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 173611us
I20260709 13:01:09.357249  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1044) took 2986 ms. Trace:
I20260709 13:01:09.266208  5180 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.265276  5039 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.282740  5181 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 13:01:09.331136  5179 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:09.359293  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:09.328756  5174 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.328606  5144 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:09.339591  4944 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.307719  5162 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:01:09.373586  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:09.357766  2236 rpcz_store.cc:276] 0709 13:01:06.370714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.327551 (+956837us) service_pool.cc:224] Handling call
0709 13:01:07.343445 (+ 15894us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:07.343771 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:07.343797 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:07.345054 (+  1257us) log.cc:844] Serialized 52 byte log entry
0709 13:01:09.180892 (+1835838us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b18001ae000. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:01:09.356393 (+175501us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:09.356434 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.356913 (+   479us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:09.356975 (+    62us) raft_consensus.cc:1685] finished
0709 13:01:09.356991 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.357192 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":662,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":34839,"spinlock_wait_cycles":2434176,"thread_start_us":175294,"threads_started":1,"wal-append.queue_time_us":1994838}
I20260709 13:01:09.308423  5182 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:01:09.356696  5088 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:09.375749  5183 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:09.376405  5151 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:09.306362  5104 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.391680  5186 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:09.392560  5186 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:09.393313  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:09.393869  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.393576  5186 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.395654  1691 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:09.396495  5018 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:09.396926  5018 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:09.403023  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:09.405789  5018 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.381758  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:09.316241  5095 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.403431  4943 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:09.412525  4943 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:09.384321  4747 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:09.361318  5099 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.407234  2236 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:09.380188  1684 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:09.311183  5168 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.408207  4651 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:09.420691  1684 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 13:01:09.326251  5178 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:09.440140  1683 raft_consensus.cc:2410] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 13:01:09.412659  5095 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:09.378885  5157 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:09.408427  5018 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:09.379262  5106 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:09.470491  1684 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:09.363214  5169 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:01:09.374604  5179 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:09.417043  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:09.381778  1540 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:09.407397  5004 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:09.371793  5161 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
W20260709 13:01:09.419106  4628 consensus_meta.cc:287] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 1.075s	user 0.011s	sys 0.002s
I20260709 13:01:09.396536  1691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:01:09.382851  4944 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.513833  1691 raft_consensus.cc:606] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:09.515286  1691 raft_consensus.cc:614] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:09.436795  5099 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.461745  4991 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:09.441771  5157 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:09.518461  5099 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:09.518407  5157 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.383522  5174 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:09.468012  4973 consensus_meta.cc:287] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.832s	user 0.009s	sys 0.000s
I20260709 13:01:09.519834  4973 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.521622  2232 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:09.521734  5189 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.522270  5189 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:09.441071  5185 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.523550  5185 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.524466  5189 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:09.469026  5103 consensus_meta.cc:287] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.272s	user 0.015s	sys 0.000s
I20260709 13:01:09.527354  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:09.534516  5185 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:09.461169  5160 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:09.579442  1687 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:09.479851  5183 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.441335  5187 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:09.583420  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 166617us
I20260709 13:01:09.588788  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:09.535357  5185 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:09.585292  5176 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:09.525887  4628 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:01:09.598374  4996 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:09.584168  5094 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.227s	user 0.008s	sys 0.005s
I20260709 13:01:09.601927  4944 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:09.602245  5094 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:09.596932  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:09.597338  5178 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:09.598176  5176 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:09.479337  5144 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.601202  5185 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:09.608294  1535 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:01:09.631984  5155 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.249s	user 0.000s	sys 0.000s
I20260709 13:01:09.582736  5158 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.609098  2237 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:09.633342  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 670) took 1411 ms. Trace:
I20260709 13:01:09.633806  1534 rpcz_store.cc:276] 0709 13:01:08.221519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.221690 (+   171us) service_pool.cc:224] Handling call
0709 13:01:08.222054 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:08.222466 (+   412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:08.222494 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:08.233256 (+ 10762us) log.cc:844] Serialized 26 byte log entry
0709 13:01:08.464904 (+231648us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:08.464956 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:08.465049 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:09.633107 (+1168058us) raft_consensus.cc:1685] finished
0709 13:01:09.633129 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.633284 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1874,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":12657,"thread_start_us":231307,"threads_started":1,"wal-append.queue_time_us":1103075}
I20260709 13:01:09.636217  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.636988  1534 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:01:09.637755  1539 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:09.609905  4668 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:09.608112  5160 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:09.610705  5169 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:09.611161  5094 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:09.629115  5176 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:09.656946  5104 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:09.656157  5099 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.012s
W20260709 13:01:09.629709  5154 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.743s	user 0.004s	sys 0.001s
W20260709 13:01:09.669277  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 326807us
I20260709 13:01:09.640875  5187 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:09.644483  5180 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:09.680526  5187 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.681982  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.682968  1542 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:09.632387  5085 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.674065  5144 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:09.695729  1657 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:09.595184  5183 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.700330  5099 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.630895  5053 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.706413  5085 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:09.693787  5171 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:09.683802  5160 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:09.713872  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:09.648365  5185 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:09.719991  5123 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.689s	user 0.011s	sys 0.000s
I20260709 13:01:09.720682  5168 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:09.721624  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:09.730976  1542 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:09.723109  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:09.646144  5158 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:09.725618  5169 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:01:09.712178  5064 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.690s	user 0.007s	sys 0.004s
I20260709 13:01:09.711561  4651 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:09.734068  1537 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:09.736244  5095 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:09.738976  5197 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:01:09.739038  5053 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:09.741480  5196 log.cc:927] Time spent T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 13:01:09.656432  5193 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.117s
I20260709 13:01:09.747181  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:09.722585  5099 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.749922  5064 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:09.737614  4995 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.002s
I20260709 13:01:09.753979  5017 raft_consensus.cc:3055] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:09.732594  5123 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:09.758509  5164 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.625s	user 0.002s	sys 0.000s
I20260709 13:01:09.761147  4651 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:09.762198  5116 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.796772  2230 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:09.796438  5183 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:09.806697  5059 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:09.808255  5059 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.808877  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:09.809531  2233 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:01:09.810413  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:09.815471  1685 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:09.818550  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:09.827502  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:09.828801  1684 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 9 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 13:01:09.787895  5017 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:09.793272  4628 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:09.728690  5160 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:09.813228  5127 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.354s	user 0.002s	sys 0.011s
W20260709 13:01:09.817031  5125 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.412s	user 0.020s	sys 0.038s
I20260709 13:01:09.870532  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:09.902465  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:09.878450  4628 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:09.865209  5040 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:09.878187  5125 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.481s	user 0.021s	sys 0.038s
I20260709 13:01:09.972023  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2410) took 1749 ms. Trace:
I20260709 13:01:09.922629  5168 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:01:10.000905  2201 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.850s (SENT) [suppressed 12 similar messages]
I20260709 13:01:09.938853  5171 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:01:09.988238  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:09.972627  1689 rpcz_store.cc:276] 0709 13:01:08.223025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.223139 (+   114us) service_pool.cc:224] Handling call
0709 13:01:08.223393 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:08.340753 (+117360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:08.340802 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:08.340862 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:08.340918 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:09.971696 (+1630778us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540041e6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:09.971762 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:09.971945 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":269306240}
I20260709 13:01:09.955236  5040 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:09.950282  1684 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: [9.11-9.11]
I20260709 13:01:09.897156  5127 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:10.033874  1537 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 13:01:09.983191  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 54012.3us
I20260709 13:01:10.017009  5040 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:09.963797  5180 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:01:10.052337  1690 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:10.055092  5158 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:10.061038  5064 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:10.062448  5055 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 1.730s	user 0.012s	sys 0.002s
I20260709 13:01:10.043148  5171 raft_consensus.cc:697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:10.069473  2234 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:10.070859  4755 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.072958  4755 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:10.074720  1582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.038s	sys 0.046s
I20260709 13:01:10.053472  5185 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:10.077732  1582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.039s	sys 0.046s
W20260709 13:01:10.079600  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 900000us
I20260709 13:01:10.074666  5123 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:10.034600  5116 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:10.041517  5017 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:10.103129  4943 consensus_meta.cc:287] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.690s	user 0.003s	sys 0.009s
I20260709 13:01:10.103657  4943 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:10.096938  5160 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:10.114475  1543 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.015s	sys 0.030s
W20260709 13:01:10.116007  1543 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.015s	sys 0.030s
I20260709 13:01:10.130018  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:10.089277  5044 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:10.151840  1440 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55368 (request call id 326) took 26093 ms (client timeout 15000 ms). Trace:
I20260709 13:01:10.092873  4651 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:10.167737  4943 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:10.168460  4980 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:10.168916  4980 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:10.144454  4811 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:10.170284  1543 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:10.174607  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 693) took 1119 ms. Trace:
I20260709 13:01:10.175873  5174 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:10.203482  5177 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.974s	user 0.000s	sys 0.012s
I20260709 13:01:10.176971  1686 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:10.204579  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2419) took 1878 ms. Trace:
I20260709 13:01:10.204790  4980 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:10.205071  1686 rpcz_store.cc:276] 0709 13:01:08.326085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.326614 (+   529us) service_pool.cc:224] Handling call
0709 13:01:08.326877 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.176816 (+1849939us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400100a20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa63dc0 00007fe08faee1c0 00007fe08faef231 00007fe08faebd8f 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c
0709 13:01:10.204206 (+ 27390us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.204507 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3776979328}
I20260709 13:01:10.179095  5059 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:10.180586  2232 rpcz_store.cc:276] 0709 13:01:09.055297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.055456 (+   159us) service_pool.cc:224] Handling call
0709 13:01:09.087290 (+ 31834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.522060 (+434770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.174294 (+652234us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b5400484ed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:10.174527 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":998292224}
I20260709 13:01:10.206290  4980 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:10.207058  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:10.208199  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:10.208946  2232 raft_consensus.cc:2410] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:10.211172  2232 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:10.149019  5017 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:10.144663  5099 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:10.212513  2232 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:10.220461  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:10.153252  1440 rpcz_store.cc:269] 0709 13:00:44.058528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:44.058760 (+   232us) service_pool.cc:224] Handling call
0709 13:01:10.151744 (+26092984us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:01.648395 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:01.649123 (+   728us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:01.649171 (+    48us) write_op.cc:435] Acquired schema lock
0709 13:01:01.649188 (+    17us) tablet.cc:662] Decoding operations
0709 13:01:01.714820 (+ 65632us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:01.714899 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:01.714928 (+    29us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:01:01.763224 (+ 48296us) tablet.cc:701] Row locks acquired
0709 13:01:01.763274 (+    50us) write_op.cc:260] PREPARE: finished
0709 13:01:01.763457 (+   183us) write_op.cc:270] Start()
0709 13:01:01.763628 (+   171us) write_op.cc:276] Timestamp: P: 1783602061763391 usec, L: 0
0709 13:01:01.763656 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:01:01.771583 (+  7927us) log.cc:844] Serialized 4000 byte log entry
0709 13:01:02.569189 (+797606us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b54003535a8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe095ea89e1 00007fe095ea764c 00007fe0935233bb 00007fe09352571a 00007fe0935256a8 00007fe093525628 00007fe0935255e3 00007fe0935247fc 00007fe095957626 00007fe095951dd8 00007fe08faf8310
0709 13:01:02.569329 (+   140us) op_driver.cc:464] REPLICATION: finished
0709 13:01:02.856902 (+287573us) write_op.cc:301] APPLY: starting
0709 13:01:02.860392 (+  3490us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:02.904113 (+ 43721us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:02.904170 (+    57us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:02.921822 (+ 17652us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:02.922221 (+   399us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:01:02.922277 (+    56us) write_op.cc:312] APPLY: finished
0709 13:01:02.922705 (+   428us) log.cc:844] Serialized 77 byte log entry
0709 13:01:02.922902 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:02.923060 (+   158us) write_op.cc:454] Released schema lock
0709 13:01:02.923150 (+    90us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:02.923624 (+   474us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:02.923660 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":130435,"reported_tablets":38,"child_traces":[["op",{"apply.queue_time_us":222889,"apply.run_cpu_time_us":4425,"apply.run_wall_time_us":118710,"num_ops":10,"prepare.queue_time_us":250911,"prepare.run_cpu_time_us":50387,"prepare.run_wall_time_us":423698,"raft.queue_time_us":124624,"raft.run_cpu_time_us":51948,"raft.run_wall_time_us":278958,"replication_time_us":675755,"spinlock_wait_cycles":15643008,"thread_start_us":644555,"threads_started":4,"wal-append.queue_time_us":492836,"wal-append.run_cpu_time_us":27975,"wal-append.run_wall_time_us":1665871}]]}
W20260709 13:01:10.180637  5167 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.844s	user 0.008s	sys 0.004s
I20260709 13:01:10.230455  5004 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:10.231036  5004 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:10.223232  1438 catalog_manager.cc:5697] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:10.240514  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 107181us
W20260709 13:01:10.241551  2202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.026s	sys 0.052s
W20260709 13:01:10.242489  2202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.026s	sys 0.052s
W20260709 13:01:10.245221  1508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.000s	sys 0.075s
W20260709 13:01:10.245819  1508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.000s	sys 0.076s
I20260709 13:01:10.233119  5127 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:10.259975  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:10.246594  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 910226us
W20260709 13:01:10.237486  5019 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.011s	sys 0.021s
I20260709 13:01:10.233256  5004 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:10.270185  1438 catalog_manager.cc:5697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 4 to 6, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 6 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:10.240660  4943 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:10.237584  4811 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:01:10.277024  1540 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.013s	sys 0.063s
W20260709 13:01:10.279498  1540 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.013s	sys 0.063s
W20260709 13:01:10.274223  5019 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.011s	sys 0.021s
I20260709 13:01:10.271883  1508 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:10.283452  1508 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:10.284732  1508 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:10.289083  2232 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:10.221172  5171 consensus_queue.cc:237] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:10.296845  1438 catalog_manager.cc:5697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:10.301193  1438 catalog_manager.cc:5697] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 4 to 5, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 5 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:10.304394  1508 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:10.278712  5004 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:10.291205  5168 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:10.292346  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1062) took 1011 ms. Trace:
I20260709 13:01:10.291280  5059 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.325898  2232 rpcz_store.cc:276] 0709 13:01:09.281061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.212144 (+931083us) service_pool.cc:224] Handling call
0709 13:01:10.212407 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.291101 (+ 78694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.292254 (+  1153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.328404  2232 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:10.329241  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1063) took 1048 ms. Trace:
I20260709 13:01:10.329718  2232 rpcz_store.cc:276] 0709 13:01:09.281309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.327714 (+1046405us) service_pool.cc:224] Handling call
0709 13:01:10.328163 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.328943 (+   780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.329171 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.331961  1438 catalog_manager.cc:5697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:10.335078  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:10.237232  5167 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:10.339015  1438 catalog_manager.cc:5697] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 5 to 7, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 7 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:10.343887  1438 catalog_manager.cc:5697] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 3, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 3 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:10.282136  4943 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:10.336611  5179 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.806s	user 0.008s	sys 0.005s
W20260709 13:01:10.348312  2203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.011s	sys 0.027s
I20260709 13:01:10.335409  5180 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:01:10.360322  4944 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.052s	sys 0.070s
I20260709 13:01:10.288231  4811 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:10.340001  5011 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.075s	user 0.038s	sys 0.063s
I20260709 13:01:10.341990  5185 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:10.353511  2203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.011s	sys 0.027s
I20260709 13:01:10.346491  5121 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:10.366997  2231 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:10.332839  5004 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:10.372257  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:10.381451  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:10.339910  5059 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:10.378367  5011 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.114s	user 0.038s	sys 0.064s
I20260709 13:01:10.376004  5019 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:10.375783  4944 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.052s	sys 0.070s
I20260709 13:01:10.366482  5205 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:10.379117  5200 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:10.378901  5166 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.149s	user 0.003s	sys 0.005s
I20260709 13:01:10.348178  5168 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:01:10.383131  5180 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:10.375308  1438 catalog_manager.cc:5697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:10.358290  5179 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:10.391571  1541 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 11. Ops: []
I20260709 13:01:10.378794  5121 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:10.399735  1618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1513) took 5577 ms (client timeout 1650 ms). Trace:
I20260709 13:01:10.390713  5205 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:10.402288  5200 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:10.410794  1438 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 6 to 7, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 7 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:10.437604  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:10.575647  1534 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.900s	user 0.016s	sys 0.052s
I20260709 13:01:10.407038  1508 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:01:10.441045  1509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.052s	user 0.010s	sys 0.022s
W20260709 13:01:10.576571  1535 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.003s	sys 0.019s
I20260709 13:01:10.579973  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 677) took 1285 ms. Trace:
W20260709 13:01:10.409370  1618 rpcz_store.cc:269] 0709 13:01:04.813665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:04.813810 (+   145us) service_pool.cc:224] Handling call
0709 13:01:10.390969 (+5577159us) spinlock_profiling.cc:243] Waited 3.32 s on lock 0x7b5400389854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:10.391188 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7637859072}
W20260709 13:01:10.438290  1657 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.006s	sys 0.012s
W20260709 13:01:10.675491  1657 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.008s	sys 0.012s
W20260709 13:01:10.676196  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 787304us
I20260709 13:01:10.676589  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 902) took 1961 ms. Trace:
W20260709 13:01:10.459980  1688 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.038s	user 0.013s	sys 0.021s
W20260709 13:01:10.427412  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:10.681613  1688 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.260s	user 0.014s	sys 0.021s
W20260709 13:01:10.476373  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 475472us
W20260709 13:01:10.381376  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.590229  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1064) took 1308 ms. Trace:
I20260709 13:01:10.435747  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 883) took 4545 ms. Trace:
I20260709 13:01:10.432744  4755 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:10.412552  4811 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:10.696405  5103 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:10.397378  4827 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:10.591430  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 903) took 1587 ms. Trace:
W20260709 13:01:10.592754  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 332746us
I20260709 13:01:10.414323  4943 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:10.620550  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.628942  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2589) took 1450 ms. Trace:
I20260709 13:01:10.425316  5011 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:01:10.446693  4973 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:10.413980  5004 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:10.630856  4951 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:10.403990  5039 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.024s	sys 0.038s
I20260709 13:01:10.411020  5106 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:10.410211  5119 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.172s	user 0.025s	sys 0.019s
I20260709 13:01:10.703814  4661 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.385542  5162 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:10.726881  5119 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.488s	user 0.025s	sys 0.020s
I20260709 13:01:10.728673  5119 consensus_queue.cc:1243] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:01:10.412894  5176 consensus_meta.cc:287] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.749s	user 0.014s	sys 0.000s
I20260709 13:01:10.456496  5179 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:10.453051  5181 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:10.383786  5185 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:10.441918  5187 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.001s	sys 0.018s
W20260709 13:01:10.752588  5187 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.001s	sys 0.019s
I20260709 13:01:10.398408  5194 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:10.753610  5185 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:10.754839  5194 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:10.423187  5205 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:10.528795  5211 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:01:10.762115  5176 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:10.653131  5040 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.588s	user 0.007s	sys 0.005s
W20260709 13:01:10.661000  1534 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.017s	sys 0.053s
W20260709 13:01:10.670531  1509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.281s	user 0.010s	sys 0.022s
W20260709 13:01:10.676208  1535 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.004s	sys 0.019s
I20260709 13:01:10.677299  2237 rpcz_store.cc:276] 0709 13:01:08.715311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.715472 (+   161us) service_pool.cc:224] Handling call
0709 13:01:08.715795 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.633402 (+917607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.675789 (+1042387us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b540065aed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:10.676315 (+   526us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2045934336}
I20260709 13:01:10.680722  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:10.690169  5178 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:10.688268  2232 rpcz_store.cc:276] 0709 13:01:09.281615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.330700 (+1049085us) service_pool.cc:224] Handling call
0709 13:01:10.331121 (+   421us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:10.344933 (+ 13812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.344963 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:10.345492 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 13:01:10.588227 (+242735us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:10.588871 (+   644us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.589279 (+   408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.589483 (+   204us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:10.589583 (+   100us) raft_consensus.cc:1685] finished
0709 13:01:10.589600 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.589876 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13319,"prepare.queue_time_us":60591,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":78,"raft.queue_time_us":74911,"raft.run_cpu_time_us":487,"raft.run_wall_time_us":4691,"thread_start_us":213279,"threads_started":1,"wal-append.queue_time_us":187088}
I20260709 13:01:10.691741  4766 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:10.692762  2230 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1030) took 4639 ms (client timeout 1818 ms). Trace:
I20260709 13:01:10.699673  1538 rpcz_store.cc:276] 0709 13:01:09.178324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.179488 (+  1164us) service_pool.cc:224] Handling call
0709 13:01:10.628141 (+1448653us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b54001e4e54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:10.628406 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53136128}
I20260709 13:01:10.696532  2234 rpcz_store.cc:276] 0709 13:01:05.890023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.890208 (+   185us) service_pool.cc:224] Handling call
0709 13:01:05.890521 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.069971 (+4179450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.435471 (+365500us) spinlock_profiling.cc:243] Waited 4.17 s on lock 0x7b54005b0254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:10.435649 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9601363712}
I20260709 13:01:10.704327  1582 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:10.705049  5039 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.024s	sys 0.039s
I20260709 13:01:10.449461  5121 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:10.784325  5121 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:10.432236  5167 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:10.745807  4943 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.671387  1540 rpcz_store.cc:276] 0709 13:01:09.294632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.478204 (+183572us) service_pool.cc:224] Handling call
0709 13:01:09.478497 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.478693 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:10.279944 (+801251us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.578618 (+298674us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.578964 (+   346us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.579034 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:10.579181 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.579339 (+   158us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.579683 (+   344us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62132,"prepare.queue_time_us":984919,"prepare.run_cpu_time_us":171,"prepare.run_wall_time_us":179,"raft.queue_time_us":142635,"raft.run_cpu_time_us":801,"raft.run_wall_time_us":15590,"spinlock_wait_cycles":7852800,"thread_start_us":1059044,"threads_started":2,"wal-append.queue_time_us":256466}
I20260709 13:01:10.724730  5162 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:10.768757  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 1.41415e+06us
W20260709 13:01:10.769016  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 88740.3us
I20260709 13:01:10.769835  5119 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:10.764717  5040 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:10.770128  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:10.741185  5200 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:10.791909  4943 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:10.794559  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:10.795353  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:10.797243  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 93173us
I20260709 13:01:10.798732  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:10.799548  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.775466  5178 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:10.799813  5181 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:10.775287  4766 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:10.800201  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 341779us
I20260709 13:01:10.798782  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:10.698046  2229 rpcz_store.cc:276] 0709 13:01:09.003729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.003851 (+   122us) service_pool.cc:224] Handling call
0709 13:01:09.004175 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:10.091274 (+1087099us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b180046c660. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08f6944bc 00007fe08c9e248b 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fc455 00007fe08fae4060 00007fe08fae4bfb 00007fe08faef64b
0709 13:01:10.091451 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.091484 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:10.092046 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 13:01:10.385296 (+293250us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:10.385353 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.385723 (+   370us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:10.590467 (+204744us) raft_consensus.cc:1685] finished
0709 13:01:10.590636 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.591150 (+   514us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16669,"raft.queue_time_us":90482,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":29218,"spinlock_wait_cycles":42180352,"thread_start_us":367311,"threads_started":2,"wal-append.queue_time_us":383057}
W20260709 13:01:10.776253  2230 rpcz_store.cc:269] 0709 13:01:06.053296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:06.124652 (+ 71356us) service_pool.cc:224] Handling call
0709 13:01:10.691896 (+4567244us) spinlock_profiling.cc:243] Waited 3.57 s on lock 0x7b54005c0dd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:10.692332 (+   436us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8217614208}
I20260709 13:01:10.801249  5039 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:10.756673  5194 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:10.800226  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:10.791007  5119 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:10.807132  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2418) took 2485 ms. Trace:
I20260709 13:01:10.816849  2232 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:10.821633  5187 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:10.821491  4766 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:10.822307  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:10.822764  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:10.824064  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:10.822780  2234 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:10.808670  5040 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:10.825532  5103 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:10.827699  4943 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:10.829841  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:10.829988  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:10.796974  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:10.833913  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.832801  1690 rpcz_store.cc:276] 0709 13:01:08.321367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.322036 (+   669us) service_pool.cc:224] Handling call
0709 13:01:08.322295 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.052812 (+1730517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.806519 (+753707us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b54002ce054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:10.806881 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3978256128}
I20260709 13:01:10.802186  2237 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:10.841845  2237 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:10.842511  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 908) took 1133 ms. Trace:
I20260709 13:01:10.842852  2237 rpcz_store.cc:276] 0709 13:01:09.710045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.841038 (+1130993us) service_pool.cc:224] Handling call
0709 13:01:10.841520 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.842265 (+   745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.842432 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:10.844517  1684 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:10.845197  4766 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:10.825577  5178 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:10.834280  5119 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:10.859192  2236 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:10.847635  5103 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:10.863865  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:10.861863  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 675) took 3135 ms. Trace:
W20260709 13:01:10.879256  5017 consensus_meta.cc:287] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.548s	user 0.004s	sys 0.020s
I20260709 13:01:10.879396  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:10.880376  2230 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
W20260709 13:01:10.880658  5153 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.986s	user 0.000s	sys 0.031s
W20260709 13:01:10.881255  5153 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.022s	user 0.000s	sys 0.031s
W20260709 13:01:10.882972  5044 consensus_meta.cc:287] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.725s	user 0.010s	sys 0.007s
I20260709 13:01:10.882808  5187 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:10.883730  1689 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:10.883737  5085 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:10.885058  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:10.885541  5172 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.657s	user 0.006s	sys 0.004s
I20260709 13:01:10.886042  1689 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:10.885739  5085 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:10.887660  2233 raft_consensus.cc:2410] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:10.888671  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1995) took 1247 ms. Trace:
W20260709 13:01:10.895540  1687 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2429) took 2438 ms (client timeout 1874 ms). Trace:
W20260709 13:01:10.896081  1687 rpcz_store.cc:269] 0709 13:01:08.456460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.715660 (+259200us) service_pool.cc:224] Handling call
0709 13:01:10.895112 (+2179452us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b5400374e54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:10.895294 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1739180800}
I20260709 13:01:10.898552  5176 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:10.899868  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:10.910014  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:10.880431  1691 raft_consensus.cc:2376] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7. Re-sending same reply.
I20260709 13:01:10.868507  5200 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:10.889195  1534 rpcz_store.cc:276] 0709 13:01:09.641265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.650550 (+  9285us) service_pool.cc:224] Handling call
0709 13:01:09.650894 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.651138 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:01:10.766393 (+1115255us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.835120 (+ 68727us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.856267 (+ 21147us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:10.875425 (+ 19158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.888313 (+ 12888us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.888561 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":778020,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":1045,"raft.queue_time_us":958,"raft.run_cpu_time_us":314,"raft.run_wall_time_us":19837,"thread_start_us":1154374,"threads_started":2}
I20260709 13:01:10.846911  4661 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:10.917913  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:10.922883  1687 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:10.930960  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:10.932129  1687 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
W20260709 13:01:10.933867  5163 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.672s	user 0.009s	sys 0.025s
W20260709 13:01:10.934101  5233 log.cc:927] Time spent T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.056s	user 0.000s	sys 0.002s
W20260709 13:01:10.934805  5163 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.749s	user 0.010s	sys 0.026s
I20260709 13:01:10.936046  1618 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:10.933885  2203 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:10.943809  5200 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:10.946951  5200 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:10.947158  5156 raft_consensus.cc:3055] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:10.947722  5156 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:10.949604  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:10.951718  5040 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:10.952543  2234 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:10.953842  4952 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:10.876000  5039 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:10.913477  5187 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:10.952152  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1999) took 1298 ms. Trace:
I20260709 13:01:10.944268  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:10.869671  1688 rpcz_store.cc:276] 0709 13:01:07.726904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:08.028157 (+301253us) service_pool.cc:224] Handling call
0709 13:01:09.385166 (+1357009us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b5800010e98. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:01:09.385372 (+   206us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:09.385742 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:09.421044 (+ 35302us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.448769 (+1027725us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b18003e17c0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:01:10.682674 (+233905us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:10.683677 (+  1003us) log.cc:844] Serialized 26 byte log entry
0709 13:01:10.683963 (+   286us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:10.684001 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.684158 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:10.838410 (+154252us) raft_consensus.cc:1685] finished
0709 13:01:10.838430 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.861786 (+ 23356us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44018,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":59575,"raft.run_cpu_time_us":418,"raft.run_wall_time_us":421,"spinlock_wait_cycles":73492736,"thread_start_us":1260675,"threads_started":1,"wal-append.queue_time_us":1094486}
I20260709 13:01:10.956750  2228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 891) took 3095 ms. Trace:
I20260709 13:01:10.965318  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:10.969978  2228 rpcz_store.cc:276] 0709 13:01:07.861316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:07.861424 (+   108us) service_pool.cc:224] Handling call
0709 13:01:07.877043 (+ 15619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.787365 (+1910322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.787409 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:09.787471 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:09.787510 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.955945 (+1168435us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b54006e78d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:10.956033 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.956458 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4392587136}
W20260709 13:01:10.962198  2231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1055) took 1693 ms (client timeout 2113 ms). Trace:
I20260709 13:01:10.992156  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:10.961849  1582 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:10.954404  4952 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:10.957787  4661 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:10.996376  2231 rpcz_store.cc:269] 0709 13:01:09.268435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.334747 (+ 66312us) service_pool.cc:224] Handling call
0709 13:01:10.961650 (+1626903us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540059a954. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:10.961952 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2370688384}
W20260709 13:01:11.013761  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 145355us
I20260709 13:01:10.949254  5156 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:10.954380  5157 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:11.016839  2229 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:10.969699  1535 rpcz_store.cc:276] 0709 13:01:09.653305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.041217 (+387912us) service_pool.cc:224] Handling call
0709 13:01:10.044290 (+  3073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.044517 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:10.769811 (+725294us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.928070 (+158259us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.950788 (+ 22718us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.951396 (+   608us) log.cc:844] Serialized 14 byte log entry
0709 13:01:10.951550 (+   154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:10.951623 (+    73us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:10.951707 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.951853 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:10.952052 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22031,"prepare.queue_time_us":412221,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":133,"raft.queue_time_us":30833,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":18795,"thread_start_us":816008,"threads_started":2,"wal-append.queue_time_us":86784}
I20260709 13:01:11.003746  5059 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:11.025378  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 76009.5us
I20260709 13:01:11.031859  4628 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:11.035244  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2580) took 1906 ms. Trace:
I20260709 13:01:11.035787  1539 rpcz_store.cc:276] 0709 13:01:09.128967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.129168 (+   201us) service_pool.cc:224] Handling call
0709 13:01:09.637531 (+508363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.638250 (+   719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.034702 (+1396452us) spinlock_profiling.cc:243] Waited 508 ms on lock 0x7b540030a7d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:11.035006 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1168351616}
I20260709 13:01:11.037122  5176 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:11.037767  5176 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.038352  5176 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:11.038278  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.042346  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2593) took 1862 ms. Trace:
I20260709 13:01:11.042995  1536 rpcz_store.cc:276] 0709 13:01:09.179353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.241284 (+ 61931us) service_pool.cc:224] Handling call
0709 13:01:09.243066 (+  1782us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.371705 (+128639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:09.371735 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:09.371781 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:09.371823 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.041743 (+1669920us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54000eddd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:11.041819 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.042107 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287765376}
I20260709 13:01:11.045665  5040 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:11.046322  5238 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.075s	user 0.000s	sys 0.007s
I20260709 13:01:11.046507  5040 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.047700  5156 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.020141  5153 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:11.051281  5153 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.051986  1612 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:43412
I20260709 13:01:11.052893  1612 raft_consensus.cc:476] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Not starting forced leader election -- already a leader
I20260709 13:01:11.054476  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1998) took 1405 ms. Trace:
W20260709 13:01:11.031874  5116 consensus_meta.cc:287] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.913s	user 0.008s	sys 0.016s
I20260709 13:01:11.075410  4952 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.054893  1537 rpcz_store.cc:276] 0709 13:01:09.649149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.763612 (+114463us) service_pool.cc:224] Handling call
0709 13:01:10.033618 (+270006us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:10.034337 (+   719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.054080 (+1019743us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54001d7ed0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:11.054291 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":578171648}
W20260709 13:01:11.078776  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50841.1us
I20260709 13:01:11.079202  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 688) took 1845 ms. Trace:
W20260709 13:01:11.080765  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:11.075543  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.081857  1687 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:11.085182  1689 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
W20260709 13:01:11.086876  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:11.078778  5232 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.064s	user 0.003s	sys 0.000s
I20260709 13:01:11.093619  5104 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.101012  1688 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:01:11.101891  1688 raft_consensus.cc:606] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:11.102309  1688 raft_consensus.cc:614] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:11.105455  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.106657  1539 raft_consensus.cc:2410] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:11.109642  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.109896  1611 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:11.109699  5162 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:11.111972  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.113199  1535 raft_consensus.cc:2393] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:01:11.113588  5230 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.255s	user 0.002s	sys 0.000s
I20260709 13:01:11.115082  4661 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:11.115549  4661 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:11.115923  4661 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:11.117125  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.083331  1683 rpcz_store.cc:276] 0709 13:01:09.234062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.346092 (+112030us) service_pool.cc:224] Handling call
0709 13:01:11.053744 (+1707652us) spinlock_profiling.cc:243] Waited 92.8 ms on lock 0x7b540014a354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:11.078966 (+ 25222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213467008}
I20260709 13:01:11.091214  1691 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:11.071823  5040 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:11.088104  1690 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:01:11.128427  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.121202  4951 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.132160  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:11.132086  4951 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.133594  4986 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:11.134950  4951 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:11.134832  4986 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.137703  5187 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:11.140225  5187 raft_consensus.cc:3055] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:11.140774  5187 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:11.144215  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:11.144918  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.127539  5104 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.148841  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.149569  1614 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:11.149892  1611 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:11.148844  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:11.151265  5039 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:11.150996  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:11.151804  2234 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:01:11.155104  4871 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:11.131465  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.164085  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.156867  5176 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.127977  5116 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.174551  1691 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:01:11.175590  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.178648  1683 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:01:11.181305  1683 raft_consensus.cc:606] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:11.176455  2232 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
W20260709 13:01:11.185546  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 59063.6us
I20260709 13:01:11.186376  5210 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.187067  5210 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.186579  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2581) took 2039 ms. Trace:
I20260709 13:01:11.160612  5185 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:11.169605  4986 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:11.192361  1541 rpcz_store.cc:276] 0709 13:01:09.129285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.129582 (+   297us) service_pool.cc:224] Handling call
0709 13:01:09.129943 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.405567 (+1275624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.089523 (+683956us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54003e2954. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:11.168552 (+ 79029us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2900896128}
I20260709 13:01:11.183712  4952 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:11.226965  2237 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:11.212128  5178 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.228922  4916 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.180593  5091 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.229954  1536 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
W20260709 13:01:11.235826  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68142.7us
I20260709 13:01:11.239241  5205 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:11.240579  5039 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:11.226166  5210 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:11.231784  4916 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.198978  5116 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:11.266904   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:11.274017  5185 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:11.277644  5059 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.278680  5059 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.280161  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 904) took 1900 ms. Trace:
I20260709 13:01:11.282233  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 907) took 1819 ms. Trace:
I20260709 13:01:11.282855  2233 rpcz_store.cc:276] 0709 13:01:09.462797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.810194 (+347397us) service_pool.cc:224] Handling call
0709 13:01:11.281911 (+1471717us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54002a35d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:11.282146 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2474893696}
I20260709 13:01:11.284385  5178 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:11.280761  2229 rpcz_store.cc:276] 0709 13:01:09.380080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.852942 (+1472862us) service_pool.cc:224] Handling call
0709 13:01:10.965173 (+112231us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.026449 (+ 61276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.279834 (+253385us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b540059a950. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:11.280038 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156953216}
I20260709 13:01:11.285597  5091 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.287050  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2425) took 2911 ms. Trace:
I20260709 13:01:11.287518  1685 rpcz_store.cc:276] 0709 13:01:08.375228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.006389 (+631161us) service_pool.cc:224] Handling call
0709 13:01:09.006631 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:09.816098 (+809467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.286529 (+1470431us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b54000aaad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:11.286855 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1859565440}
I20260709 13:01:11.288947  5103 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.292527  5119 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:11.293944  5187 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.296048  5187 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:11.298269  5119 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:11.302544  5153 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:11.303431  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:11.303471  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.304286  5100 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:11.246282  4871 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 13:01:11.306173  5176 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:11.315040  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:11.321774  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.322363  5189 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.322715  4916 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:11.365617  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:11.365994  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:11.366469  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 45390.1us
I20260709 13:01:11.366685  1537 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.370548  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:11.371197  5103 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:11.327694  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.323439   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:11.324039  5185 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:11.400861  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.330734  5091 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:11.335446  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 1997) took 1693 ms. Trace:
I20260709 13:01:11.340380  5059 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:11.336310  5237 log.cc:927] Time spent T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 13:01:11.338210  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.412128  1617 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:01:11.338443  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.337843  5104 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:11.358798  1615 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:11.359735  5085 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:11.360878  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.362705  5181 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:11.364502  1535 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: [9.11-9.11]
I20260709 13:01:11.370814  1539 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:11.329341  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:11.369225  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.374029  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:11.374470  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.372769  2201 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:11.374661  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.348469  5100 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:11.375012  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.355976  5205 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:11.376644  5189 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:11.377876  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:11.381845  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:11.382579  1683 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:11.370615  4871 raft_consensus.cc:3055] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:11.383668  4661 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:11.384554  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.387761  1507 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:11.389331  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:11.389678  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:11.391506  5116 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:11.392647  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.418665  2234 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:11.414837  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:11.424371  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 259446us
I20260709 13:01:11.401924  1542 rpcz_store.cc:276] 0709 13:01:09.642063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:09.731829 (+ 89766us) service_pool.cc:224] Handling call
0709 13:01:10.628788 (+896959us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:10.629807 (+  1019us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:10.629845 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:10.631274 (+  1429us) log.cc:844] Serialized 26 byte log entry
0709 13:01:10.672444 (+ 41170us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:10.672483 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:10.673919 (+  1436us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:11.118736 (+444817us) raft_consensus.cc:1685] finished
0709 13:01:11.118755 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:11.334940 (+216185us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b54001e4e50. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:11.335223 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14301,"raft.queue_time_us":28732,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":7873,"spinlock_wait_cycles":2005658880,"thread_start_us":31485,"threads_started":1,"wal-append.queue_time_us":102778}
W20260709 13:01:11.495234  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:11.502204  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.496187  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:11.412554  5187 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:11.514115  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:11.514058  5187 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.515475  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:11.505802  1610 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
W20260709 13:01:11.516875  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 129376us
I20260709 13:01:11.517701  1536 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:11.518040  1687 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:11.518399  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:11.518821  5189 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:11.519445  5189 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:11.519506  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:11.519922  5189 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.520323  5189 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.471033  5103 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:01:11.521590  1537 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:11.522387  5103 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:11.522863  5103 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:11.523700  1613 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:11.524189  5103 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:11.525667  1540 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:11.527062  5178 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:11.527268  4661 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:11.527580  5178 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:11.527941  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 158504us
I20260709 13:01:11.528091  5178 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:11.528890  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:11.529547  1690 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:11.529960  1688 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:01:11.530380  1688 pending_rounds.cc:85] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 8
I20260709 13:01:11.530637  1688 pending_rounds.cc:107] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:01:11.471407  4871 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:11.531236  1688 raft_consensus.cc:2889] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:11.532634  1534 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:11.530592  5059 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:11.533253  5059 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.533746  5059 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:11.501924  5085 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:11.521135  5116 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:11.510772  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:11.516824  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:11.543246  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.532141  4871 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.541474  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.547528  5194 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:11.547889  1685 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:11.512888  5181 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.575961  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.576917  1610 raft_consensus.cc:2368] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8.
I20260709 13:01:11.547636  1684 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:11.517130  5187 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:11.555207  1541 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:01:11.557883  2201 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:11.560968  1689 raft_consensus.cc:2410] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:11.552947  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:11.585599  1691 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:11.584380  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:11.542661  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:11.588006  4916 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.588624  4916 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.588918  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.590081  2234 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:11.591239  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:11.587693  1686 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:11.595901  4916 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:11.597819  4747 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.600572  4973 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.601328  4973 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.592118  1534 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:11.604574  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.614028  1615 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:11.622702  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.624445  2234 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:11.577327  5194 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:11.632043  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 110638us
I20260709 13:01:11.635469  2237 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:11.633934  1542 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:11.604976  5103 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:11.641499  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.650606  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.651925  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.653139  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:11.656689  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 327545us
I20260709 13:01:11.625239  4916 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:11.598717  4747 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:11.662827  1683 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:11.632969  5194 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:11.661331  2232 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:11.668816  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.669746  1615 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:11.660795  5187 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:11.674415  5187 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:11.676039  4747 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:11.675691  5187 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:11.677464  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.678257  1610 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:11.679550  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:11.681368  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.684203  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:11.685031  1540 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:11.685859  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.686455  1537 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:11.687042  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:11.687424  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.688056  1537 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:11.685889  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:11.689205  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 146599us
I20260709 13:01:11.631291  5085 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:11.701460  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.703709  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.710304  2237 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:11.723944  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.726890  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:11.728487  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.728487  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.729552  2234 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:11.729882  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.731792  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.732687  2237 raft_consensus.cc:2410] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:01:11.734268  5119 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:11.734845  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:11.735494  1615 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:11.736382  1692 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.013s	sys 0.058s
W20260709 13:01:11.736905  1692 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.014s	sys 0.058s
I20260709 13:01:11.738447  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:11.739354  1543 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:01:11.743377  5100 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:11.744656  5100 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:11.748234  4871 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.750218  1535 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: [4.3-9.4]
I20260709 13:01:11.751654  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.729377  2233 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:11.758922  2234 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:11.761226  1541 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:11.761958  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:11.770151  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.772266  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:11.772454  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.759800  5119 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:11.796579  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.813715  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:11.833348  1539 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:11.796132  5153 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:11.783143  5181 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:11.848294  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:11.843577  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:11.797840  5119 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:11.848984  2237 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:11.763654  5185 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.077s
W20260709 13:01:11.823330  4952 consensus_meta.cc:287] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.589s	user 0.011s	sys 0.007s
I20260709 13:01:11.762284  5163 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:01:11.876128  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:11.841914  5153 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.782483  4747 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:01:11.869753  5119 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:11.896611  1657 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:11.899712  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:11.905817  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.909696  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.911713  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.912449  1614 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:11.913481  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.914778  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.894377  5040 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:11.915613  2228 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:11.915922  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.836660  5085 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:11.916671  1618 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:11.917173  4973 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:11.917572  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:11.918756  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:11.919601  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:11.919759  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 307939us
I20260709 13:01:11.920055  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.920572  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:11.845145  5156 consensus_queue.cc:1059] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:01:11.921993  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:11.924691  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:11.898061  5153 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:11.921171  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.928938  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.929692  5163 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:11.930083  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:11.930205  5163 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:11.935832  1536 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:11.939888  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.930814  1688 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:01:11.949313  2237 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:11.962687  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.967674  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:11.972079  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:11.930853  5163 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:11.927692  5258 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:11.986276  2228 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:11.987017  1687 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:11.987844  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:11.989212  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 239085us
I20260709 13:01:11.966316  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:11.992384  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.996136  1657 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:11.997457  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.000233  1683 raft_consensus.cc:2410] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:01:12.003456  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:11.930886  5040 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:12.020576  1615 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:01:12.007615  5059 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.008211  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:12.008706  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:12.009037  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.942651  4766 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:11.994275  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:12.035352  1684 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.037927  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:12.038599  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:12.038053  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:12.042330  1508 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:11.989087  5104 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.054019  1689 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:01:12.044471  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.047133  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:12.050544  4766 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.050866  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:11.963689  5121 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:12.050866  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:12.053227  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 64732.3us
W20260709 13:01:12.057716  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.060038  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:12.061908  2229 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:12.063031  1614 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:01:12.061529  1615 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
W20260709 13:01:12.066009  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 119756us
I20260709 13:01:11.988577  5178 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.043115  5116 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:12.075174  1657 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:12.095273  1689 raft_consensus.cc:2376] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10. Re-sending same reply.
I20260709 13:01:12.097399  1688 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:12.062947  4916 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:12.107090  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:12.107612  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:12.111658  2236 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:12.111924  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:12.112385  1614 raft_consensus.cc:2410] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:12.108491  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:12.104871  1584 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:12.061053  5104 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:12.124253  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 134048us
I20260709 13:01:12.129495  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:12.134451  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:12.135569  1618 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:12.136886  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:12.107110  5185 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:12.128801  1687 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:12.126974  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:12.150549  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:12.160936  1689 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 6.
I20260709 13:01:12.165738  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:12.166561  1613 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:12.126597  5156 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:12.166714  1686 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:12.165675  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:12.148701  5116 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:12.104177  5121 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.162492  5258 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:12.177408  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 68981us
I20260709 13:01:12.177724  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:12.178596  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 116869us
W20260709 13:01:12.178819  5261 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:01:12.162555  5185 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.179376  4661 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:01:12.180729  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.175631  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:12.202306  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.202386  1614 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:12.184830  1618 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:12.120115  5040 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
W20260709 13:01:12.206393  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 77060.8us
I20260709 13:01:12.186447  5116 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.222939  5116 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:12.228724  1685 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
W20260709 13:01:12.231398  5189 consensus_meta.cc:287] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.677s	user 0.016s	sys 0.000s
I20260709 13:01:12.232175  5189 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:12.233035  5121 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:12.173787  5250 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.351s	user 0.001s	sys 0.001s
W20260709 13:01:12.192019  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 96214.6us
I20260709 13:01:12.117558  4916 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:12.195744  5255 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.301s	user 0.000s	sys 0.001s
I20260709 13:01:12.194728  1615 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
W20260709 13:01:12.187780  1534 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.572s	user 0.015s	sys 0.007s
I20260709 13:01:12.238557  1534 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:12.191030  5085 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:01:12.211680  5004 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:12.195140  5267 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:12.260254  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 56414.3us
I20260709 13:01:12.203091  5185 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:12.266948  1614 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:12.270586  1613 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
W20260709 13:01:12.270910  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 239519us
I20260709 13:01:12.272487  1615 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:12.274943  1508 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:12.291258  1536 raft_consensus.cc:2410] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:01:12.290732  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 55795.1us
I20260709 13:01:12.297070  1613 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:12.300340  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:12.262425  5004 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:12.270431  5156 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.319165  1657 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:12.316758  2231 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:12.303822  4986 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.309898  5181 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:12.327224  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 52413.7us
W20260709 13:01:12.327695  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.328511  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:12.329291  1536 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:12.231503  5163 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.331993  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:12.333595  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:12.339527  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.340318  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.341123  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.267604  5267 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.332810  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:12.345712  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:12.314813  5059 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:12.318171  5004 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:12.350401  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:12.355893  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 176496us
I20260709 13:01:12.355131  1540 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:12.356961  1508 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:12.321238  5181 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.328227  5187 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:12.365552  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 199425us
I20260709 13:01:12.359210  1536 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:01:12.364390  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:12.367223  1541 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:12.369575  1543 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:12.370903  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:12.372752  1534 raft_consensus.cc:2410] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:12.372934  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:12.374311  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:12.345031  5178 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:12.375839  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:12.372051  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2046) took 1131 ms. Trace:
I20260709 13:01:12.345738  5267 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:12.376910  1538 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:12.381814  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:12.330227  4986 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:12.391203  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.392130  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.378959  1611 rpcz_store.cc:276] 0709 13:01:11.234505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.236924 (+  2419us) service_pool.cc:224] Handling call
0709 13:01:11.262133 (+ 25209us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.361356 (+ 99223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.361388 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:11.362014 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 13:01:11.617290 (+255276us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:11.617349 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.617552 (+   203us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:12.363719 (+746167us) raft_consensus.cc:1685] finished
0709 13:01:12.363747 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:12.363930 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38943,"raft.queue_time_us":41029,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":254774,"threads_started":1,"wal-append.queue_time_us":449872}
I20260709 13:01:12.392544  1538 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:12.341881  5156 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:12.392788  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.365218  5181 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:12.389235  1537 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:12.370081  5121 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:12.381636  4871 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.005s
I20260709 13:01:12.418146  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:12.383772  4628 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:12.395339  5104 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:12.395751  4952 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:01:12.425223  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47919.7us
I20260709 13:01:12.441174  1508 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:12.374684  4661 consensus_queue.cc:1059] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
W20260709 13:01:12.462811  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 159835us
I20260709 13:01:12.463698  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:12.465930  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 71713.7us
I20260709 13:01:12.464645  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:12.435235  5189 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:12.469152  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:12.469980  5280 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:12.470809  5274 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:01:12.470860  5104 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:12.473095  5275 log.cc:927] Time spent T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:01:12.475167  4986 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:12.478168  5282 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:01:12.421578  5178 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 13:01:12.478597  5273 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 13:01:12.478452  5259 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:12.454084  5289 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:01:12.484820  1684 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
W20260709 13:01:12.510579  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46286.1us
I20260709 13:01:12.452638  5121 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:12.486999  5100 consensus_meta.cc:287] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.742s	user 0.006s	sys 0.007s
I20260709 13:01:12.513273  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:12.533372  5189 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:12.530413  4916 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:12.539237  1610 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
W20260709 13:01:12.539690  5119 consensus_meta.cc:287] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.597s	user 0.012s	sys 0.000s
I20260709 13:01:12.604517  5119 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.605837  1687 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:12.540226  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.541873  5281 log.cc:927] Time spent T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:01:12.543920  1584 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:12.548705  5039 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:12.610450  5039 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:12.611646  5039 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.612854  5039 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.560274  5187 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:12.567374  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.617843  5277 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 13:01:12.618167  5040 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.441740  5144 consensus_queue.cc:1059] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.188s
W20260709 13:01:12.569837  5256 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.577s	user 0.005s	sys 0.000s
I20260709 13:01:12.570544  5163 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:12.481405  4952 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:12.486269  5280 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:12.439759  4628 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.511582  5283 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:12.596977  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 83805.1us
I20260709 13:01:12.537791  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:12.602501  5278 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:01:12.603644  5104 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:12.549880  5285 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:12.568362  5153 consensus_meta.cc:287] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.000s
I20260709 13:01:12.619702  2203 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:12.645033  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 598) took 1124 ms. Trace:
I20260709 13:01:12.642886  1535 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:12.623482  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2043) took 1541 ms. Trace:
I20260709 13:01:12.692821  1690 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:12.764880  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:12.685534  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 97074.1us
I20260709 13:01:12.680998  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:12.589967  5178 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:12.808843  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 189230us
I20260709 13:01:12.800077  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:12.641325  5103 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.990s	user 0.010s	sys 0.003s
W20260709 13:01:12.822434  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.614478  5119 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:12.694245  5283 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.803583  5280 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:12.807324  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:12.597925  5121 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:12.816659  1657 leader_election.cc:336] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.509s (SENT)
I20260709 13:01:12.839972  1657 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:12.841078  5121 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:12.843704  5119 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:12.844393  5100 raft_consensus.cc:993] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:01:12.844868  5100 raft_consensus.cc:1081] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:01:12.791980  1617 rpcz_store.cc:276] 0709 13:01:11.520653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.575552 (+ 54899us) service_pool.cc:224] Handling call
0709 13:01:11.575909 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:11.576255 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:11.816223 (+239968us) log.cc:844] Serialized 14 byte log entry
0709 13:01:12.643904 (+827681us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b18002a0c40. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c979c0a 00007fe0960b9219 00007fe08ca01a40 00007fe08c9fc4e4 00007fe08c9fc455 00007fe0934e43c7 00007fe08faf69a7 00007fe08fb060bf 00007fe08fb06038
0709 13:01:12.644385 (+   481us) log.cc:844] Serialized 14 byte log entry
0709 13:01:12.644610 (+   225us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:12.644667 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:12.644725 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.644796 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:12.644928 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28732,"prepare.queue_time_us":546127,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":103,"raft.queue_time_us":8696,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":17676,"spinlock_wait_cycles":24807168,"thread_start_us":436792,"threads_started":2,"wal-append.queue_time_us":250996}
I20260709 13:01:12.841378  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:12.841886  4661 consensus_queue.cc:1059] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:12.842453  1686 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:12.842623  5103 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.847541  5103 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.843029  4986 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.843175  5144 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:12.849051  1684 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:12.819456  4916 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:12.621132  5156 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:12.792340  1612 rpcz_store.cc:276] 0709 13:01:11.083837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.089240 (+  5403us) service_pool.cc:224] Handling call
0709 13:01:11.184487 (+ 95247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:11.363707 (+179220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.363756 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:11.364342 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 13:01:11.758639 (+394297us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:11.832699 (+ 74060us) log.cc:844] Serialized 14 byte log entry
0709 13:01:11.907538 (+ 74839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:11.921638 (+ 14100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:12.568933 (+647295us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:12.623127 (+ 54194us) raft_consensus.cc:1685] finished
0709 13:01:12.623254 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:12.623412 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":139244,"prepare.queue_time_us":57565,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":46,"raft.queue_time_us":188089,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":271,"spinlock_wait_cycles":30355840,"thread_start_us":393485,"threads_started":1,"wal-append.queue_time_us":627433}
I20260709 13:01:12.859550  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:12.794909  5104 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:12.618767  5040 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.870538  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:12.875393  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:12.878968  4973 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:12.800045  5285 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.850643  5283 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:12.805380  5153 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:12.886516  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 86554.1us
W20260709 13:01:12.928443  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 384655us
I20260709 13:01:12.862103  4916 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:12.874469  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 77664.4us
I20260709 13:01:12.889385  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:12.886226  5156 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:12.896495  5301 raft_consensus.cc:3055] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:12.900010  5291 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:12.892194  5276 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:12.902037  5103 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:12.904587  5285 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:12.905609  5059 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:01:12.917755  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.918033  5258 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:12.919883  5173 raft_consensus.cc:3055] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:12.920253  5004 consensus_meta.cc:287] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.553s	user 0.005s	sys 0.004s
I20260709 13:01:12.921883  5290 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:12.858875  4986 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:12.979636  5290 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.980100  5290 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:12.981912  4986 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:12.908998  5267 consensus_meta.cc:287] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.000s
W20260709 13:01:12.973312  5181 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.545s	user 0.002s	sys 0.007s
I20260709 13:01:12.983155  5267 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:12.925921  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.926664  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 595) took 1406 ms. Trace:
I20260709 13:01:12.926997  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:12.985911  2237 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:01:12.943320  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:12.946163  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56853.4us
I20260709 13:01:12.987001  2201 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:12.890727  5104 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:12.938395  5116 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.704s	user 0.011s	sys 0.001s
I20260709 13:01:12.988384  2201 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:12.993274  1686 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
W20260709 13:01:12.954110  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 137419us
I20260709 13:01:12.993770  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:12.967414  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:12.968429  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:12.969517  5294 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 13:01:12.970436  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:12.995646  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:12.947333  5289 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:12.945313  5301 raft_consensus.cc:740] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:12.951335  5085 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:12.972616  5059 raft_consensus.cc:697] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:12.956079  5156 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:12.959230  5291 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:12.974439  4628 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:12.954212  4952 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.975541  5153 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:12.959019  5306 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:12.975948  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:12.977802  5040 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:13.005995  5085 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:12.926038  5194 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.231s	user 0.012s	sys 0.002s
I20260709 13:01:12.988900  1540 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:12.942440  4916 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:12.987581  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:12.984349  1609 rpcz_store.cc:276] 0709 13:01:11.519769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.521749 (+  1980us) service_pool.cc:224] Handling call
0709 13:01:11.522161 (+   412us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:11.759134 (+236973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:11.759197 (+    63us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:11.770104 (+ 10907us) log.cc:844] Serialized 52 byte log entry
0709 13:01:12.114181 (+344077us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:12.141722 (+ 27541us) log.cc:844] Serialized 14 byte log entry
0709 13:01:12.152751 (+ 11029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:12.152836 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:12.921245 (+768409us) raft_consensus.cc:1685] finished
0709 13:01:12.921264 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:12.921413 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10772,"prepare.queue_time_us":529137,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":24840,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":13701,"thread_start_us":493351,"threads_started":2,"wal-append.queue_time_us":462300}
I20260709 13:01:12.994985  1541 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:12.971302  5185 consensus_meta.cc:287] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.666s	user 0.010s	sys 0.002s
I20260709 13:01:12.998042  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:13.033876  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 46363us
I20260709 13:01:12.992151  5116 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.961285  5173 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:13.000591  1539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54764
I20260709 13:01:12.997716  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:12.993852  1686 pending_rounds.cc:85] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 6
I20260709 13:01:13.069980  1686 pending_rounds.cc:107] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 10.7
W20260709 13:01:13.003547  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 60360.2us
I20260709 13:01:13.070304  1686 raft_consensus.cc:2889] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:13.071043  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:12.958851  5258 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:13.002871  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:12.993557  5267 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:13.082513  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:13.083130  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:13.083564  1536 raft_consensus.cc:2410] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:13.083405  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1577) took 1162 ms. Trace:
I20260709 13:01:12.991406  4986 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:12.998961  5276 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:13.014439  4628 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:13.017446  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:12.983311  5181 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:13.095062  5181 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:13.095670  5181 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:13.096155  5181 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.020797  4973 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:13.097383  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:13.098232  4973 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.099539  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:13.100461  5302 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:13.100895  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:13.029704  1543 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:13.030162  5059 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.034027  1689 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:12.996280  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:13.035105  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:13.036127  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:13.036181  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 702) took 1097 ms. Trace:
I20260709 13:01:13.070075  1683 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
W20260709 13:01:13.074116  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 131493us
I20260709 13:01:13.032243  5039 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:13.005784  5289 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.031558  5085 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:13.085347  1686 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:13.033037  5153 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.085261  5156 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.091917  4628 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:12.988896  5004 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.103286  1541 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:13.104858  1509 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:13.105002  1685 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:13.106817  5278 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:13.105535  1691 rpcz_store.cc:276] 0709 13:01:11.934875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.935256 (+   381us) service_pool.cc:224] Handling call
0709 13:01:11.990456 (+ 55200us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:12.028837 (+ 38381us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:12.141760 (+112923us) log.cc:844] Serialized 14 byte log entry
0709 13:01:12.572007 (+430247us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b18003e0fe0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a 00007fe08faf6a2a
0709 13:01:12.874598 (+302591us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:12.888309 (+ 13711us) log.cc:844] Serialized 26 byte log entry
0709 13:01:12.888518 (+   209us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:13.016262 (+127744us) log.cc:844] Serialized 14 byte log entry
0709 13:01:13.017728 (+  1466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.017805 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:13.032522 (+ 14717us) raft_consensus.cc:1685] finished
0709 13:01:13.032544 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.032696 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33328,"prepare.queue_time_us":267626,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":44235,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":34704,"spinlock_wait_cycles":48640128,"thread_start_us":476128,"threads_started":3,"wal-append.queue_time_us":818230}
I20260709 13:01:13.090554  4986 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.108192  5258 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.108726  4747 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:13.109313  5246 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:13.109364  5267 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:13.109838  1688 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:13.116214  5116 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:13.122783  1537 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: [6.7-7.8]
W20260709 13:01:13.123885  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 135567us
I20260709 13:01:13.122251  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:13.036655  5194 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.129626  1509 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:13.084273  1614 rpcz_store.cc:276] 0709 13:01:11.920812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.272876 (+352064us) service_pool.cc:224] Handling call
0709 13:01:12.308935 (+ 36059us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:12.330373 (+ 21438us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:12.806949 (+476576us) log.cc:844] Serialized 14 byte log entry
0709 13:01:13.082699 (+275750us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:13.082757 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:13.082805 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.082875 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.083337 (+   462us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":321094,"prepare.queue_time_us":477864,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":8402,"raft.queue_time_us":34070,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":73,"thread_start_us":212034,"threads_started":2,"wal-append.queue_time_us":181619}
I20260709 13:01:13.100009  4766 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:13.090420  5285 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:13.184244  2201 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:13.134800  1539 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:13.185017  1539 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.118044  5173 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:13.174088  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 51906.8us
I20260709 13:01:13.128649  4628 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.181243  1618 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:13.184353  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 54822us
I20260709 13:01:13.196681  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:13.037861  5185 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.033183  5306 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.186679  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:13.166136  5266 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:13.166071  5004 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:13.167634  4986 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:13.136458  4747 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:13.190237  5304 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 13:01:13.121289  5039 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:13.192766  1658 leader_election.cc:336] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.532s (SENT)
I20260709 13:01:13.229408  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:13.180179  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 177403us
I20260709 13:01:13.168329  5144 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.200539  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1574) took 1281 ms. Trace:
I20260709 13:01:13.235939  1690 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:13.171700  5246 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:13.167574  5289 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:13.206637  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:13.183563  5121 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:13.207592  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:13.132131  5153 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.193478  5278 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:13.125329  5280 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.213495  5267 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:13.228992  1538 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:13.236130  1618 rpcz_store.cc:276] 0709 13:01:11.919258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.274104 (+354846us) service_pool.cc:224] Handling call
0709 13:01:13.173158 (+899054us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7eee 0000000000375939 00007fe094a3b1e0 00007fe094b040c4 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:01:13.173314 (+   156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.200197 (+ 26883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.200453 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5545856}
I20260709 13:01:13.208110  4871 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:13.213222  5312 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:13.247771  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 212710us
I20260709 13:01:13.261812  2203 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:13.215276  5173 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.209031  5116 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:13.215034  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 44405.8us
I20260709 13:01:13.236945  5039 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.132618  5301 consensus_queue.cc:260] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.245077  4766 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.245342  5121 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.247496  5004 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:01:13.245833  5181 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:13.246730  5278 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.274634  4747 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:13.283296  5318 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:01:13.263705  5312 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:13.279304  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1552) took 2475 ms. Trace:
I20260709 13:01:13.285490  5185 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:13.286456  5266 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:13.287377  5325 raft_consensus.cc:2764] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:13.279014  5121 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:13.288430  5153 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:13.250816  5319 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:01:13.289273  4661 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:01:13.292093  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 102689us
I20260709 13:01:13.261970  4871 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:13.284029  5280 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:13.297863  2232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1078) took 1681 ms (client timeout 1509 ms). Trace:
W20260709 13:01:13.303751  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 133380us
W20260709 13:01:13.288235  1610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 605) took 1616 ms (client timeout 1683 ms). Trace:
I20260709 13:01:13.276996  5316 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.298929  1539 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.279160  5163 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:13.294394  5278 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:13.300711  1692 rpcz_store.cc:276] 0709 13:01:10.803741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:10.958145 (+154404us) service_pool.cc:224] Handling call
0709 13:01:10.958644 (+   499us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:10.959423 (+   779us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:10.959907 (+   484us) log.cc:844] Serialized 14 byte log entry
0709 13:01:11.038623 (+ 78716us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b1800064060. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:01:13.136053 (+2097430us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b1800064060. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08f6944bc 00007fe08c9e248b 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a
0709 13:01:13.268728 (+132675us) log.cc:844] Serialized 14 byte log entry
0709 13:01:13.278847 (+ 10119us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:13.278902 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:13.278958 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.279021 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:13.279240 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34531,"prepare.queue_time_us":63046,"prepare.run_cpu_time_us":158,"prepare.run_wall_time_us":8085,"raft.queue_time_us":20101,"raft.run_cpu_time_us":369,"raft.run_wall_time_us":370,"spinlock_wait_cycles":84833536,"thread_start_us":868907,"threads_started":2,"wal-append.queue_time_us":770765}
W20260709 13:01:13.323365  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 75226.5us
W20260709 13:01:13.369150  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:13.369578  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:13.369899  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.276657  5116 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.304147  5312 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:13.365329  2232 rpcz_store.cc:269] 0709 13:01:11.627966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.629051 (+  1085us) service_pool.cc:224] Handling call
0709 13:01:13.297642 (+1668591us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b5400512cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:13.297784 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43283712}
I20260709 13:01:13.297701  5314 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:01:13.363206  1610 rpcz_store.cc:269] 0709 13:01:11.672030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.679270 (+  7240us) service_pool.cc:224] Handling call
0709 13:01:13.288046 (+1608776us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b540008cad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:13.288151 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1712344320}
I20260709 13:01:13.294301  5321 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.369830  1582 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:13.362758  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:13.370491  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:13.367657  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 183491us
I20260709 13:01:13.382282  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:13.362097  5276 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.373107  5104 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.376436  1540 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:13.390571  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:13.375936  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:13.391861  2237 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:13.392150  1536 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:13.320426  4661 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:13.365857  5318 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:01:13.402588  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:13.403172  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:13.375280  1507 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:13.320779  4747 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.321255  5246 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:13.380515  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 118793us
I20260709 13:01:13.413761  2203 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:13.335876  4871 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.398444  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:13.369907  5059 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:13.380911  5314 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.410445  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:13.384431  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:13.385319  5194 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:13.400910  5004 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:13.401775  5181 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:13.434733  2203 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:13.421478  1692 raft_consensus.cc:2393] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:13.437137  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:13.390337  5280 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:01:13.393996  5185 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:01:13.455344  1683 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:13.419067  5267 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:13.430511  5314 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.399132  5259 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:01:13.403580  5163 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.394143  5276 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:13.457051  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.377259  5319 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:01:13.455931  5181 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:13.489672  5320 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 13:01:13.491488  5327 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:01:13.493515  5337 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:13.494673  5329 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:01:13.495568  5328 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:13.496493  5326 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:01:13.496376  4916 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.452454  5194 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:13.457301  5246 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:13.449862  5331 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.523290  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:13.526484  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.431730  5308 consensus_queue.cc:1059] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
W20260709 13:01:13.529403  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 277180us
W20260709 13:01:13.575284  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 190905us
W20260709 13:01:13.577164  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.394629  5321 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.468760  5153 consensus_queue.cc:1059] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:01:13.535604  5328 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:13.497062  5039 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.523861  5306 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:13.450857  5290 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:13.568670  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.529361  5259 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:13.526185  5059 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.580003  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:13.597438  5289 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:13.597944  5328 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:13.537463  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.465682  4661 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:13.602342  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 148824us
I20260709 13:01:13.630676  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:13.614811  5059 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.604592  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:13.632923  5059 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:13.633553  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:13.622915  2229 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:13.635658  5319 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:13.636118  5319 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:13.636494  5319 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:13.618233  5289 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.640770  5289 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:13.642225  5334 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:01:13.650781  5321 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:13.651890  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:13.652999  2229 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:13.655107  5290 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:13.643781  5283 consensus_meta.cc:287] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.721s	user 0.011s	sys 0.000s
I20260709 13:01:13.656834  5283 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.657559  5276 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:13.659690  5283 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:13.644601  5039 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:13.606595  5259 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:13.611698  5181 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.662112  5104 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:13.661160  5259 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.666172  5259 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:13.646615  4916 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:13.582122  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 200274us
I20260709 13:01:13.676505  4986 raft_consensus.cc:3055] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:13.679692  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:13.681056  5320 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:13.682853  1689 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:13.686237  1687 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:13.687004  5291 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:13.687482  5328 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.689457  5333 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:13.690572  5333 raft_consensus.cc:697] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:13.682665  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:13.691820  5333 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.695989  5328 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:13.698374  5330 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:01:13.699698  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:13.700807  1688 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:13.703125  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:13.704044  1691 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
W20260709 13:01:13.705936  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.696107  5258 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.707674  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:13.717038  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:13.713299  2228 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
W20260709 13:01:13.615124  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 102790us
I20260709 13:01:13.677292  4986 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:13.726240  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:13.719031  5085 consensus_meta.cc:287] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.585s	user 0.011s	sys 0.000s
I20260709 13:01:13.735140  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:13.735778  1612 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:13.711337  5306 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:13.724205  2203 connection.cc:256] RPC call timeout handler was delayed by -1.4887287710000123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29910000000000003s for a response
W20260709 13:01:13.738399  2203 connection.cc:256] RPC call timeout handler was delayed by -1.0255298729999822s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1503s for a response
I20260709 13:01:13.739252  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:13.740343  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:13.721242  5040 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:13.718158  5104 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.749579  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:13.744133  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:13.745083  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:13.720317  5320 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:13.752239  5336 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:01:13.752140  5320 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.742722  5085 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.754798  5320 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:13.755494  2232 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:13.726152  4628 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.004s
I20260709 13:01:13.756520  5085 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:13.756256  4628 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.757097  5306 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.758819  4628 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:13.759140  5314 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.759950  5306 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:13.655153  5121 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:13.761621  5349 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.762290  5349 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.732070  5280 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.763554  5280 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:13.766449  1508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.316s	user 0.055s	sys 0.061s
W20260709 13:01:13.766846  1508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.317s	user 0.055s	sys 0.061s
W20260709 13:01:13.767462  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 1.32625e+06us
I20260709 13:01:13.768980  1508 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:13.743566  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:13.774629  1615 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:13.770288  5294 raft_consensus.cc:3055] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:13.783938  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:13.812937  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64092.1us
I20260709 13:01:13.811896  5320 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:13.817061  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:13.818154  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:13.881036  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:13.882072  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 70555.9us
I20260709 13:01:13.826300  1618 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:13.871601  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.887622  1616 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:13.888661  2229 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:13.889361  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:13.780344  5259 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:13.891117  1618 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:13.893456  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:13.895007  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:13.880849  5320 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:13.888331  5310 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:13.898013  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:13.809317  5314 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:13.779124  5173 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 13:01:13.778746  5294 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:13.907943  2236 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 10 index: 11. (term mismatch)
I20260709 13:01:13.932869  2236 pending_rounds.cc:85] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 10
I20260709 13:01:13.933326  2236 pending_rounds.cc:107] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 9.11
I20260709 13:01:13.933724  2236 raft_consensus.cc:2889] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:13.908295  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:13.908553  5340 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:13.930704  2231 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:13.896570  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:13.933116  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 97295.9us
I20260709 13:01:13.825637  5004 raft_consensus.cc:697] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:13.947585  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 131106us
I20260709 13:01:13.778056  5104 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:13.940774  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:13.953661  1687 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:13.947378  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 135583us
I20260709 13:01:13.957190  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:13.957446  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:13.957974  2235 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:13.958384  2233 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:13.959496  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:13.809607  5291 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.120s
I20260709 13:01:13.926718  5320 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:13.930586  4661 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:13.888473  5178 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 13:01:13.972591  2230 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:13.970832  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:13.940451  5259 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:13.969251  5278 consensus_meta.cc:287] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.000s
W20260709 13:01:13.993000  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:13.960533  5156 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.001818  5348 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:13.968590  5314 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:13.925133  4871 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:14.020223  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:13.996534  5280 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:14.008832  5312 consensus_meta.cc:287] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.623s	user 0.009s	sys 0.005s
I20260709 13:01:13.942276  5310 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:14.016462  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:14.019892  1539 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:14.026681  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 71508us
I20260709 13:01:14.015543  1508 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:14.031499  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:13.953001  4986 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.036724  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:54764 (request call id 2029) took 1063 ms. Trace:
I20260709 13:01:14.020386  5348 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.038105  5266 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:14.038847  5266 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.040393  5348 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:14.037164  1539 rpcz_store.cc:276] 0709 13:01:12.992101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.000457 (+  8356us) service_pool.cc:224] Handling call
0709 13:01:13.978062 (+977605us) spinlock_profiling.cc:243] Waited 65.4 ms on lock 0x7b54000ada54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fae3a3d 00007fe094ab3cd0 00007fe08e038254 00007fe08e0381e2 00007fe08e038111 00007fe08e03808c 00007fe08e037342 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:14.030616 (+ 52554us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":150502144}
I20260709 13:01:14.009394  5278 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:14.044574  5278 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:14.028124  5039 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.045360  5351 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:14.045974  5351 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:14.020503  5294 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:14.046361  5351 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.046576  5361 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:14.047257  5361 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:14.047274  5294 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:14.048946  2235 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
W20260709 13:01:14.050079  5345 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:01:14.020277  5314 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:14.053762  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:14.053484  5361 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:14.054615  5361 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:14.058365  1508 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:14.049846  5283 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:14.064780  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:14.068065  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.011269  5349 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:14.071257  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.063833  5291 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:14.072907  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.073700  1539 raft_consensus.cc:2368] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
I20260709 13:01:14.031057  5312 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.074663  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.075315  1536 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:14.022271  5156 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.076445  5312 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:14.063783  5299 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:14.078003  2231 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:14.063911  5362 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:01:14.079274  1543 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:13.971239  5340 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.094871  1540 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 13:01:14.103964  5181 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.094525  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:14.105300  2230 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:14.104677  5181 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:14.108806  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.111153  5121 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:14.119388  5181 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:14.120661  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.121883  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:14.122560  2230 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:14.124797  1683 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:14.126168  1683 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:14.126624  1691 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:14.127066  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:14.127599  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:14.128266  1691 raft_consensus.cc:2368] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10.
I20260709 13:01:14.129577  5121 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:14.130095  5121 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:14.132535  5121 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:14.133355  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:14.133833  5144 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:14.134202  5144 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:14.124814  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:14.128410  1683 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:14.139062  5335 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.604s	user 0.003s	sys 0.000s
I20260709 13:01:14.148876  5163 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:14.154795  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.196775  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.150599  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2032) took 1114 ms. Trace:
I20260709 13:01:14.152064  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2460) took 1150 ms. Trace:
I20260709 13:01:14.173375  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.197626  1612 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:14.198381  1616 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:14.173550  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.174918  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:14.178360  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.179790  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.182337  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1573) took 1170 ms. Trace:
W20260709 13:01:14.200417  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.200302  1686 rpcz_store.cc:276] 0709 13:01:13.012335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.129064 (+116729us) service_pool.cc:224] Handling call
0709 13:01:13.129356 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.166537 (+ 37181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.166568 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:13.166612 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:13.166640 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.182016 (+1015376us) spinlock_profiling.cc:243] Waited 37 ms on lock 0x7b54001dd3d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:14.182075 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:14.182251 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":84988800}
W20260709 13:01:14.193898  1658 leader_election.cc:336] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.425s (SENT)
I20260709 13:01:14.183893  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:14.184743  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.184609  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:14.185683  1683 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:01:14.186543  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 710) took 2374 ms. Trace:
I20260709 13:01:14.134588  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:14.188755  1692 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:14.077471  5299 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.190559  5340 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:14.064786  5314 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:14.190712  5357 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.231s	user 0.002s	sys 0.000s
I20260709 13:01:14.191345  5259 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:14.192047  5353 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
W20260709 13:01:14.192902  5358 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:01:14.193722  1537 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:14.160354  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1569) took 1159 ms. Trace:
I20260709 13:01:14.232424  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.232585  1685 rpcz_store.cc:276] 0709 13:01:13.002615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.092905 (+ 90290us) service_pool.cc:224] Handling call
0709 13:01:13.093245 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.129204 (+ 35959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.160068 (+1030864us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b5400374954. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:14.160274 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26615552}
I20260709 13:01:14.233284  1543 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:01:14.204746  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.234025  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 114896us
I20260709 13:01:14.234244  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.235355  1536 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:01:14.235575  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 204153us
I20260709 13:01:14.205022  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.228828  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:14.210036  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:14.203840  2234 rpcz_store.cc:276] 0709 13:01:11.812462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:11.826295 (+ 13833us) service_pool.cc:224] Handling call
0709 13:01:13.298091 (+1471796us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:13.300680 (+  2589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.300724 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:13.325959 (+ 25235us) log.cc:844] Serialized 52 byte log entry
0709 13:01:13.409610 (+ 83651us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:13.575093 (+165483us) log.cc:844] Serialized 14 byte log entry
0709 13:01:13.581058 (+  5965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:13.581159 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:13.753466 (+172307us) raft_consensus.cc:1685] finished
0709 13:01:13.753484 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:14.186348 (+432864us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400512cd0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:14.186486 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9010,"prepare.queue_time_us":310851,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":3154,"raft.queue_time_us":86649,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":26027,"spinlock_wait_cycles":3353706752,"thread_start_us":167640,"threads_started":2,"wal-append.queue_time_us":123036}
I20260709 13:01:14.237944  1535 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.216070  1541 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:14.217132  1691 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:14.216954  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:14.206449  1689 rpcz_store.cc:276] 0709 13:01:13.001991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.104079 (+102088us) service_pool.cc:224] Handling call
0709 13:01:13.104377 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:13.683325 (+578948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.151735 (+468410us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54000aaad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:14.151980 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329912064}
I20260709 13:01:14.218402  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.219647  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:14.197667  1538 rpcz_store.cc:276] 0709 13:01:13.035699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.035848 (+   149us) service_pool.cc:224] Handling call
0709 13:01:14.150234 (+1114386us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54000ad054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:14.150509 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250069888}
I20260709 13:01:14.237556  1618 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:14.240254  1612 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:14.236752  1509 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:14.241962  1540 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:14.243520  1542 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:14.241863  1509 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:14.242359  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:14.247704  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 183017us
I20260709 13:01:14.243023  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:14.250710  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.250799  2229 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:14.077327  5351 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.244153  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:14.221589  5312 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:14.256536  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:14.247354  5362 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:14.264362  1536 raft_consensus.cc:2393] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:14.264926  2231 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:01:14.265806  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.259178  1543 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:14.258751  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:14.259909  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.077760  5004 consensus_queue.cc:237] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:14.264030  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:14.273371  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.251570  5267 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:14.258729  5283 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:14.278447  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:14.272670  5315 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:14.243253  5314 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 13:01:14.267104  5337 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:14.266451  5276 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:14.288349  1691 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:14.270774  5362 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.260502  5121 consensus_queue.cc:1059] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:01:14.294804  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.285851  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:14.360510  1609 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:14.360940  1543 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:14.361917  1536 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:14.281796  5354 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.254s	user 0.001s	sys 0.001s
I20260709 13:01:14.356617  5340 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:14.366936  5340 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:12.919153   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:14.371809  1687 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:14.281800  5104 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:14.297894  5258 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.373653  5258 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.296038  1535 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:14.330965  1612 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:14.364130  5362 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.383836  5144 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:14.402427  5144 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:14.411595  5299 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:14.418480  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 128127us
I20260709 13:01:14.419072  1617 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
W20260709 13:01:14.390398  5356 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.431s	user 0.002s	sys 0.005s
I20260709 13:01:14.421684  5116 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:14.423164  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:14.424075  4766 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:14.426071  5308 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:14.426234  4766 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:14.426553  5308 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.426780  4766 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:14.427078  5308 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:14.428862  4766 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.356804  5156 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:14.406064  1539 raft_consensus.cc:2410] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 12 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
W20260709 13:01:14.432582  5319 consensus_meta.cc:287] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.796s	user 0.011s	sys 0.000s
I20260709 13:01:14.432693  4766 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:14.358114  5163 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.441272  5163 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:14.443082  5040 consensus_meta.cc:287] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.665s	user 0.009s	sys 0.000s
I20260709 13:01:14.443534  5040 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:14.444183  5163 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:14.425169  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:14.446122  5163 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.294524  5267 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:14.447438  5267 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.445514  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:14.399809  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:14.452405  1616 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.454409  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.385468  5362 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.461881  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:14.462597  1542 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:14.463474  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:14.464109  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:14.464334  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:14.450691  5319 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.470947  1540 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:14.472452  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.472815  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:14.473183  1540 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 6: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:14.474550  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.476282  5291 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:14.472470  5163 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:14.475626  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.478137  1540 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:14.479965  5349 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:14.480837  5349 consensus_queue.cc:1243] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:01:14.480943  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:14.481369  5285 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:14.481722  1534 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:14.482033  5285 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:14.483433  5285 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.485725  5285 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:14.486215  5308 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:14.488726  5308 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:14.455096  1538 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:14.282449  5283 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.504333  5283 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.369123  5340 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:14.501752  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:14.492556  1536 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:14.507414  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.506610  1691 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:14.293208  5337 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.514650  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:14.515442  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.536768  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:14.561496  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.516655  5315 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:14.517023  1690 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:14.519315  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:14.519851  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:14.565368  1683 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:14.516256  1534 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:14.570941  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.566030  1691 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:14.567191  1687 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:14.285518  4952 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.226s
I20260709 13:01:14.567998  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.281539  5342 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.214s
I20260709 13:01:14.574566  5370 raft_consensus.cc:3055] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:14.575070  5370 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:14.574601  4661 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:14.575613  4661 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:14.576121  5370 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.576442  4661 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.577351  1688 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:14.578052  5370 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.580591  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.581584  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.582856  1540 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:14.583009  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:14.584230  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.595414  1508 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:14.597980  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.599586  1538 raft_consensus.cc:2410] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:14.602218  1543 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:14.602592  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:14.603447  1536 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:14.600775  1534 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 5. Current term is 6. Ops: []
I20260709 13:01:14.570122  5337 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:14.613860  1584 leader_election.cc:400] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
I20260709 13:01:14.616074  1584 leader_election.cc:403] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:14.580118  5301 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:01:14.606433  5327 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.082s	user 0.002s	sys 0.007s
I20260709 13:01:14.469686  5276 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.600170  5340 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:14.623817  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.625638  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45022.3us
I20260709 13:01:14.618096  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:14.626410  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.626441  1584 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:14.618624  4661 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.636582  1535 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:14.635882  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.640878  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:14.605436  5299 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:01:14.656006  1616 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:14.648968  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:14.681357  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.676658  1507 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:14.692464  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.693544  1691 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:14.696300  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:14.699296  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.647903  5116 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:14.644479  5104 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:14.704681  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 53698.2us
I20260709 13:01:14.718335  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:14.700749  5369 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:01:14.646883  5292 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:14.699580  1535 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:14.649986  5362 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:14.714613  1507 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:14.724905  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:14.726789  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.689494  5342 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
I20260709 13:01:14.706434  5340 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:14.721740  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:14.717039  5104 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.723481  4952 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:14.721921  5292 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:14.749805  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:14.762274  1616 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:14.722810  5116 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:14.730439  5340 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:14.752091  5181 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:14.753827  5004 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:14.716930  5348 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:01:14.769778  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.759562  1584 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:14.716451  5291 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:01:14.750633  1534 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:01:14.746884  5104 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:14.746495  5178 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:14.654765  5315 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.766660  5340 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:14.772728  5331 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:14.773273  5294 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:14.796947  1542 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:14.774405  5116 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:14.777935  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.799455  1612 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:14.778030  5276 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:14.787335  5004 consensus_queue.cc:1243] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:14.782568  1616 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:14.763072  5292 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:14.786280  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:14.788126  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:14.789999  5372 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:01:14.798384  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:14.781984  1507 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:14.803387  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.806262  1616 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:14.806094  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.809193  5285 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:14.809998  5285 raft_consensus.cc:697] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:14.810786  4766 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:14.811415  4766 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:14.811708  5374 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 13:01:14.811236  5285 consensus_queue.cc:237] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:14.812422  4766 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.817369  5004 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:14.794219  5178 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:14.819267  5178 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:14.827520  5355 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.579s	user 0.004s	sys 0.003s
I20260709 13:01:14.830292  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:14.830935  1615 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:14.832322  5283 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:14.833887  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1593) took 1109 ms. Trace:
W20260709 13:01:14.858454  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 51564.6us
I20260709 13:01:14.867486  1584 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:14.841228  5337 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.859144  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:14.842552  5320 consensus_meta.cc:287] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.866s	user 0.008s	sys 0.003s
I20260709 13:01:14.843019  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.834734  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.879493  1686 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:14.866947  1613 rpcz_store.cc:276] 0709 13:01:13.724548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.725687 (+  1139us) service_pool.cc:224] Handling call
0709 13:01:13.738152 (+ 12465us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:13.746526 (+  8374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:13.746566 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:13.751846 (+  5280us) log.cc:844] Serialized 52 byte log entry
0709 13:01:14.011736 (+259890us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:14.011789 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:14.011866 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:14.826697 (+814831us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:14.833496 (+  6799us) raft_consensus.cc:1685] finished
0709 13:01:14.833527 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:14.833792 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48918,"raft.queue_time_us":1037,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":14785,"spinlock_wait_cycles":37727104,"thread_start_us":259315,"threads_started":1,"wal-append.queue_time_us":342920}
I20260709 13:01:14.879856  1612 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:14.879855  1616 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:14.856612  1615 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:14.872495  1685 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:14.884903  5320 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.872777  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:14.889652  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:14.896958  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.892197  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:14.895035  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.901922  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.884759  5266 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:14.879356  5370 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:14.892148  5348 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:01:14.908480  1609 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:14.913844  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:14.911250  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:14.914439  1615 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:14.894625  5004 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:14.889771  5320 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:14.918000  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.917382  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:14.921978  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:14.927330  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:14.938370  1616 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:14.940120  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.941112  1616 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:14.944738  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:14.946502  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.944756  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.947939  1616 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:14.948107  1613 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:14.948897  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:14.913681  5314 consensus_queue.cc:1059] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:01:14.959347  1612 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:14.958158  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:14.965080  1613 raft_consensus.cc:2410] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:14.956099  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.913899  5331 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:01:14.967985  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.975698  1613 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:14.916401  5040 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:01:14.975548  1615 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:14.977176  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.978266  1613 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:14.969568  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:14.984705  1615 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: [9.4-11.5]
I20260709 13:01:14.980171  1616 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:14.929976  5104 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:14.948652  5388 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:14.996438  1612 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:14.999876  1613 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:01:14.983659  5387 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:15.000872  5387 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:15.001266  5387 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.002967  5340 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.914300  5266 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:14.995360  1616 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:15.016810  1612 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:01:15.021745  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 77121.8us
I20260709 13:01:15.023332  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:14.990866  5116 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:15.013315  5340 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:15.047693  5380 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 13:01:15.050084  5385 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:01:15.053737  5391 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.054658  5104 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:15.057371  5266 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:15.057969  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.064399  5163 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:15.066521  1610 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:15.068053  5387 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:15.069005  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1584) took 1362 ms. Trace:
I20260709 13:01:15.070000  5337 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:15.054356  5391 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:15.070924  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 829222us
W20260709 13:01:15.073065  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 809035us
I20260709 13:01:15.074203  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:15.069367  1610 rpcz_store.cc:276] 0709 13:01:13.706799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.706947 (+   148us) service_pool.cc:224] Handling call
0709 13:01:13.707236 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:15.066922 (+1359686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:15.068756 (+  1834us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54003499d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:15.068943 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3125208960}
W20260709 13:01:15.079550  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 56245.1us
I20260709 13:01:15.081841  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:15.083885  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:15.035039  5283 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:15.089074  5039 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.089910  5039 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.090862  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:15.092175  1509 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.515s (SENT) [suppressed 34 similar messages]
I20260709 13:01:15.092511  5039 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:15.094058  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.098183  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:15.100387  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:15.100860  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:15.111406  5085 raft_consensus.cc:2764] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:01:15.112421  5085 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:15.112794  5085 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.113152  5085 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.097857  5312 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:01:15.113759  5283 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:15.114256  5283 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.114589  5283 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.085539  5292 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:15.117563  5381 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:01:15.119387  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:15.120594  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:15.098611  5121 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:15.098366  5333 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:15.105232  5104 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:15.105598  5391 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:15.117297  5292 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:15.085323  5328 consensus_queue.cc:1059] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:01:15.136276  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:15.140175  5362 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:15.140640  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:15.141848  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:15.142007  5328 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:15.142584  5328 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:15.143033  5328 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.140831  5362 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:15.145002  5362 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:15.145558  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:15.138549  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 65438.2us
I20260709 13:01:15.123322  5392 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:15.160136  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 68004.1us
W20260709 13:01:15.164601  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.146288  5362 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:15.173337  2201 leader_election.cc:400] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10.
I20260709 13:01:15.176045  2201 leader_election.cc:403] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:15.174010  5400 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:15.174499  5039 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:15.172075  4628 raft_consensus.cc:3055] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:15.166114  5392 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:15.165676  5294 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:01:15.198093  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 56339.6us
I20260709 13:01:15.175853  5391 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:15.194273  5340 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:15.224018  5340 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:15.201913  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.193897  5396 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:15.226689  5396 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:15.226588  5399 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:15.227192  4661 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:01:15.227649  4661 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:15.190284  4628 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:15.208683  5400 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:15.208714  5395 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:15.208788  5387 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:15.208541  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:15.219235  4916 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:15.222553  5121 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:15.198431  5333 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.199131  5392 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.227954  5399 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:15.228690  5163 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:01:15.240545  1657 leader_election.cc:336] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.752s (SENT)
W20260709 13:01:15.250967  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.239229  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:15.279868  4628 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.281355  4628 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.233788  5085 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.288353  5085 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:15.227905  5396 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.305755  4661 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:15.307215  4661 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:01:15.309737  5306 raft_consensus.cc:3055] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:15.295838  5400 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.298555  5340 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:15.237370  5399 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:15.303740  5178 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.310297  5306 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:15.216122  5328 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.311482  5121 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:15.341100  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 101987us
W20260709 13:01:15.338395  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 200571us
W20260709 13:01:15.342720  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.236905  5283 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.349360  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:15.351248  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:15.346119  5403 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.355540  5319 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:15.355741  5399 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.356462  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:15.357052  1537 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.357682  5306 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:15.358706  5306 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.338884  5178 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.363520  5178 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:15.344923  5395 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:15.341315  5388 raft_consensus.cc:697] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:15.365173  5395 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:15.357084  5399 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.366500  1537 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:15.342015  5121 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:15.368279  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.370966  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:15.373365  1535 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:15.349714  5387 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.355198  5403 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.386308  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:01:15.383636  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 190789us
I20260709 13:01:15.391215  1535 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:15.378575  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:15.397589  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:15.375975  5121 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.399935  2233 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:15.366532  5294 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:01:15.379693  5402 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.403470  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:15.400715  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:15.405315  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:15.405279  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:15.409651  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:15.409890  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2072) took 1157 ms. Trace:
I20260709 13:01:15.412075  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:15.408917  1683 raft_consensus.cc:2376] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12. Re-sending same reply.
I20260709 13:01:15.414855  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:15.415721  2232 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:15.415647  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:15.416218  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:15.416870  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:15.417738  2231 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:15.411227  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:15.410238  1609 rpcz_store.cc:276] 0709 13:01:14.252603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.509176 (+256573us) service_pool.cc:224] Handling call
0709 13:01:15.409734 (+900558us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400039b54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:15.409838 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774116736}
I20260709 13:01:15.427107  5405 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:15.427789  5405 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:15.404006  5337 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 13:01:15.431910  5403 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:15.432508  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:15.432919  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.432622  5388 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.434178  4986 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.438131  5328 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:15.439049  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 52828.5us
I20260709 13:01:15.439687  5258 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:15.439837  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:15.440563  1617 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1590) took 1718 ms (client timeout 1515 ms). Trace:
W20260709 13:01:15.440994  1617 rpcz_store.cc:269] 0709 13:01:13.722528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.723137 (+   609us) service_pool.cc:224] Handling call
0709 13:01:15.440382 (+1717245us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b5400398ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:15.440493 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1559142144}
I20260709 13:01:15.441071  2203 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:15.443643  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:15.400844  5163 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:15.428800  5405 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.452571  5310 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:15.392943  5387 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:15.460902  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1603) took 1198 ms. Trace:
I20260709 13:01:15.462150  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2611) took 1370 ms. Trace:
I20260709 13:01:15.462574  1539 rpcz_store.cc:276] 0709 13:01:14.092265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.197652 (+105387us) service_pool.cc:224] Handling call
0709 13:01:15.461917 (+1264265us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54001e4454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:15.462068 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":382455168}
I20260709 13:01:15.464643  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:15.461377  1613 rpcz_store.cc:276] 0709 13:01:14.262217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:14.983905 (+721688us) service_pool.cc:224] Handling call
0709 13:01:14.999614 (+ 15709us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:15.000498 (+   884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:15.460681 (+460183us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b5400322650. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:15.460835 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35214592}
I20260709 13:01:15.466917  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:15.401702  5319 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:01:15.444759  5039 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:15.478425  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:15.479187  1690 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.480835  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:15.445325  5328 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:15.485888  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:15.503628  1614 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1586) took 1796 ms (client timeout 2128 ms). Trace:
W20260709 13:01:15.504078  1614 rpcz_store.cc:269] 0709 13:01:13.707425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:13.716729 (+  9304us) service_pool.cc:224] Handling call
0709 13:01:15.503420 (+1786691us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b5400102554. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:15.503567 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2092783488}
I20260709 13:01:15.505556  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:15.504087  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:15.507683  1691 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:15.421458  5402 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.518383  5319 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:15.418531  5395 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:15.494318  5348 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:15.526512  1683 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:01:15.516714  5392 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.493932  5085 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:15.471906  5163 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:15.547756  1684 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:15.530081  1690 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:15.533596  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:15.534873  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:15.544736  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 101149us
I20260709 13:01:15.459689  5310 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:01:15.555179  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:15.540421  5338 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.558208  5338 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:15.526067  4871 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:15.558965  5163 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.561971  5312 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:15.527335  5319 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:15.567335  5319 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.546043  5348 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:15.568958  5348 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.571128  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:15.571712  5305 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:15.572238  2231 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:15.572367  5305 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:15.573523  5305 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.574255  5407 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:15.574785  5407 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:15.575554  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:15.589298  2234 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:01:15.591779  1686 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:15.591877  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:15.596355  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.585381  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:15.617559  2233 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:15.621644  1690 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:15.625200  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:15.629474  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:15.626019  2234 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:15.575842  5407 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:15.635931  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.632411  2237 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:01:15.639134  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:15.579038  5351 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:15.603860  5173 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:15.604125  5310 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.643123  2234 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:15.644479  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:15.644670  2229 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:15.634492  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:15.650704  1691 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
W20260709 13:01:15.657630  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.630975  4871 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:15.633929  5407 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:15.670743  5387 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:15.674377  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:15.678480  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.634485  5258 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:15.694056  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:15.702375  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:15.716488  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.677878  5402 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:15.634341  5342 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
W20260709 13:01:15.747421  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45139us
I20260709 13:01:15.740566  1690 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:15.750849  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:15.696749  5104 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.701726  5004 raft_consensus.cc:3055] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:15.739941  5387 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:15.666863  5310 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:15.774197  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:15.744601  4871 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.776880  1534 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:15.780853  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:15.782172  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.804466  1535 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.809770  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:15.754088  4986 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:15.800056  1509 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:15.818425  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:15.819591  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:15.820510  1542 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:15.821715  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:15.765461  5333 consensus_meta.cc:287] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.523s	user 0.007s	sys 0.004s
I20260709 13:01:15.826696  2230 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:15.756844  5004 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:15.828195  1539 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:15.743863  5267 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:01:15.851006  1535 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:15.853632  4916 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.856346  4916 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:15.858771  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.784085  5039 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:15.903930  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:15.811689  5387 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:15.911917  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:15.760353  5104 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.924466  2236 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:15.878018  4986 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:15.843623  5333 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10.
I20260709 13:01:15.934948  2237 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:15.935989  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 303768us
I20260709 13:01:15.938428  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:15.934465  5405 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:15.940338  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:15.875833  4916 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:15.938617  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:15.943284  5410 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.241s	user 0.001s	sys 0.000s
I20260709 13:01:15.943357  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:15.944095  2230 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.827764  4871 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:15.955812  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:15.912619  5178 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:15.952472  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:15.960271  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:15.967820  1613 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:15.905368  5402 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:01:15.971262  1618 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:01:15.970525  2232 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:15.973095  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:15.980427  1616 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
W20260709 13:01:15.971125  4661 consensus_meta.cc:287] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.663s	user 0.012s	sys 0.000s
I20260709 13:01:15.989264  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:15.990140  1615 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 12: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:15.983923  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:15.990973  1609 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:15.991050  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:15.991881  1611 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:15.991673  1612 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
I20260709 13:01:15.996847  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:15.997597  1610 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:15.971742  5173 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:01:15.986527  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 226328us
I20260709 13:01:15.971843  5259 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:15.974375  5405 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:16.010337  1613 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:15.984850  5292 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:16.011534  5416 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 13:01:15.977104  5258 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:01:16.011340  5267 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:15.975378  5178 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.014140  5173 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:16.012178  5422 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:16.042408  5413 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:01:15.954711  5004 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:16.005887  5340 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
W20260709 13:01:16.084542  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.052230  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:16.073021  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:16.047210  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 348291us
W20260709 13:01:16.092267  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 271544us
W20260709 13:01:16.093111  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.023909  5292 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:16.045300  5421 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:01:16.094060  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:16.095034  1509 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:16.104136  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.095454  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:16.114106  1657 leader_election.cc:336] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.067s (SENT)
I20260709 13:01:16.114555  1657 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:16.011796  4986 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:16.100101  5399 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.721s	user 0.011s	sys 0.000s
W20260709 13:01:16.156944  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44078.8us
I20260709 13:01:16.160594  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:16.121627  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:16.079905  5121 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.666s	user 0.009s	sys 0.000s
I20260709 13:01:16.162103  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.010820  5387 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:16.130646  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:16.163712  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:16.170578  2228 raft_consensus.cc:2376] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9. Re-sending same reply.
I20260709 13:01:16.135931  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.171998  2236 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:01:16.172060  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:16.138890  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:16.172887  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.172962  1610 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:16.173633  2229 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:16.173898  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:16.174474  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.174743  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:16.175719  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:16.175966  2229 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:16.176250  1617 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:16.176980  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:16.139153  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:16.179303  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:16.062566  4871 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:16.179949  1685 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:16.172169  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:16.180711  1612 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:16.187289  5392 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:16.167227  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.164834  2231 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:16.177182  2229 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:01:16.173657  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:16.190871  1616 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.051164  5338 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:16.194051  2230 raft_consensus.cc:2410] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:16.195506  1609 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:16.197333  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:16.199431  2237 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:16.117163  5259 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:16.036031  5267 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:16.174516  1692 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:16.118129  5351 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:16.201279  1612 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:16.201835  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.201885  1610 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:16.138203  5246 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:16.165119  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:16.203794  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:16.120721  5310 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:16.189569  5362 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:16.210157  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.232525  2228 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:16.219321  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:16.233573  2230 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:01:16.197853  5421 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:01:16.185220  5104 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:16.205905  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:16.239743  1508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.064s	sys 0.096s
W20260709 13:01:16.240198  1508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.064s	sys 0.096s
I20260709 13:01:16.163786  5424 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.227829  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:16.229060  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64023.3us
I20260709 13:01:16.241991  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:01:16.246302  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.213042  5173 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.230113  5338 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:16.254060  5338 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.254632  5338 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:16.237315  5259 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:16.258072  5259 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:16.258715  5259 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:16.259191  2235 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:16.206856  5351 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:16.262041  5351 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.267377  5267 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:16.169077  5422 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
W20260709 13:01:16.268991  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.202471  2233 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:01:16.237473  5407 consensus_meta.cc:287] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.569s	user 0.013s	sys 0.000s
I20260709 13:01:16.238511  5362 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.275734  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.282786  1509 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:16.285753  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.300189  2233 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:16.311340  5340 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:16.312125  5340 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:16.271435  5267 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.316602  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:16.319494  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.320008  2229 raft_consensus.cc:2410] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:01:16.315793  1657 leader_election.cc:336] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.134s (SENT)
W20260709 13:01:16.321767  1657 leader_election.cc:336] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.199s (SENT)
I20260709 13:01:16.316809  2236 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:16.277714  5362 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.274616  5422 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:16.325100  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:16.330586  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.200791  5405 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.327816  1616 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:16.333482  2236 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:16.332109  5422 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:16.269878  5424 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.341521  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:16.244216  5292 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.338436  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.304572  5431 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:01:16.336988  5422 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:16.359409  2233 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:16.360792  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.337500  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.365634  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:16.356226  5420 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 13:01:16.313212  5340 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.369820  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:16.375850  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:16.376793  1535 raft_consensus.cc:2410] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:16.353143  5178 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:16.199120  5121 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.382822  1539 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:16.319113  5267 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:16.384621  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:16.388355  1657 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:16.390827  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:16.391752  1616 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:16.393988  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:16.394640  1609 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.395051  5104 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:16.397687  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:16.398479  1542 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:16.399552  5283 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:16.401382  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:16.368909  5342 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.037s	sys 0.050s
I20260709 13:01:16.404089  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:16.404771  1537 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:16.409587  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:16.409662  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:16.413479  1657 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:16.415160  1657 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:16.418838  1616 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:16.420434  1688 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:16.419806  1657 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:16.402217  5423 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:01:16.429536  5246 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:16.434304  1657 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:16.418898  5402 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:16.431290  4766 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:16.431821  1684 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:01:16.441671  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:16.432096  5430 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 13:01:16.406883  5342 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.038s	sys 0.051s
I20260709 13:01:16.207476   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:16.450861  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:16.430632  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:16.453586  1534 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:16.438750  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:16.444753  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 162043us
I20260709 13:01:16.455001  1536 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:16.427201  1609 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:16.446041  1617 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:01:16.459885  1611 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:01:16.446563  2203 connection.cc:256] RPC call timeout handler was delayed by -1.2969345100000282s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1786s for a response
W20260709 13:01:16.460552  2203 connection.cc:256] RPC call timeout handler was delayed by -1.0908345100000361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2015s for a response
I20260709 13:01:16.464730  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:16.446826  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:16.457125  1692 raft_consensus.cc:2410] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:01:16.432760  4747 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.401767  5407 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:01:16.445585  5391 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:01:16.449604  5163 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.482273  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:16.449786  5400 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:16.456775  5342 consensus_queue.cc:1243] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:16.483774  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:16.484611  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:16.487566  1615 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:16.488910  1534 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:16.486027  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:16.491721  1609 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:16.493119  1609 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:16.498613  1534 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:16.469859  5178 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.501345  2229 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:16.501865  1613 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:16.421962  5283 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:01:16.472357  5434 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:16.508966  1438 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40172 (request call id 230) took 29136 ms (client timeout 15000 ms). Trace:
I20260709 13:01:16.510154  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:16.510842  1611 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:16.511011  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 76779.6us
I20260709 13:01:16.456789  4766 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.520004  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:16.513895  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:01:16.516870  5419 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.486s	user 0.005s	sys 0.000s
I20260709 13:01:16.525260  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:16.525326  1657 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:16.539963  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:16.548038  1582 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:16.549665  1539 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:16.553607  1613 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:16.558053  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:16.559886  1613 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:16.562147  1613 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:16.563797  1613 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:16.565214  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 282859us
I20260709 13:01:16.568147  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:16.478148  4747 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.508966  5121 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:16.595688  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.518700  5395 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:16.510311  5178 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.602072  1534 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:16.605460  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:16.566900  5342 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:16.550961  5392 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:16.608289  1613 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:16.609205  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:16.609807  1541 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:01:16.611732  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:16.654522  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:16.502856  5338 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.503288  5400 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.615100  1508 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:16.615324  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47255.7us
I20260709 13:01:16.519091  5434 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.607087  5246 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.594772  5283 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:16.603411  5424 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:16.509502  1438 rpcz_store.cc:269] 0709 13:00:47.367134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:47.367417 (+   283us) service_pool.cc:224] Handling call
0709 13:00:48.882936 (+1515519us) spinlock_profiling.cc:243] Waited 62.6 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:01:16.503034 (+27620098us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:11.094032 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:11.094969 (+   937us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:11.095047 (+    78us) write_op.cc:435] Acquired schema lock
0709 13:01:11.095086 (+    39us) tablet.cc:662] Decoding operations
0709 13:01:11.098864 (+  3778us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:11.098955 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:11.098994 (+    39us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:01:11.100356 (+  1362us) tablet.cc:701] Row locks acquired
0709 13:01:11.100411 (+    55us) write_op.cc:260] PREPARE: finished
0709 13:01:11.100666 (+   255us) write_op.cc:270] Start()
0709 13:01:11.100872 (+   206us) write_op.cc:276] Timestamp: P: 1783602071100590 usec, L: 0
0709 13:01:11.100911 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 13:01:11.233223 (+132312us) log.cc:844] Serialized 3984 byte log entry
0709 13:01:12.140264 (+907041us) spinlock_profiling.cc:243] Waited 24.2 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fae7e28 00007fe093522581 00007fe093521d58 00007fe0935269f1 00007fe0935269a9 00007fe093526939 00007fe093526901 00007fe093525b2d 000000000038f354 000000000038f199
0709 13:01:12.209921 (+ 69657us) op_driver.cc:464] REPLICATION: finished
0709 13:01:12.479658 (+269737us) write_op.cc:301] APPLY: starting
0709 13:01:12.479798 (+   140us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:12.480165 (+   367us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:12.480253 (+    88us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:12.827591 (+347338us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:12.858200 (+ 30609us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 13:01:12.864362 (+  6162us) write_op.cc:312] APPLY: finished
0709 13:01:12.894965 (+ 30603us) log.cc:844] Serialized 77 byte log entry
0709 13:01:12.895272 (+   307us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:12.895422 (+   150us) write_op.cc:454] Released schema lock
0709 13:01:12.895501 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:12.896139 (+   638us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:12.896178 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3390275,"reported_tablets":33,"spinlock_wait_cycles":144015616,"child_traces":[["op",{"apply.queue_time_us":269337,"apply.run_cpu_time_us":6483,"apply.run_wall_time_us":416490,"num_ops":10,"prepare.queue_time_us":210955,"prepare.run_cpu_time_us":39879,"prepare.run_wall_time_us":1046494,"raft.queue_time_us":248547,"raft.run_cpu_time_us":53625,"raft.run_wall_time_us":252528,"replication_time_us":1102929,"spinlock_wait_cycles":55549696,"thread_start_us":805360,"threads_started":4,"wal-append.queue_time_us":544895,"wal-append.run_cpu_time_us":15249,"wal-append.run_wall_time_us":2100430}]]}
I20260709 13:01:16.617393  5407 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:16.653528  1538 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
W20260709 13:01:16.666742  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 55083.3us
I20260709 13:01:16.669235  5342 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.674659  1657 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:16.655351  1613 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:16.609586  5428 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.767946  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:16.687201  5338 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:16.676880  5163 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:16.689873  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:16.691777  5400 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:16.682111  5266 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.710978  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1602) took 1299 ms. Trace:
I20260709 13:01:16.712021  5337 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.718637  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:16.741883  1618 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:16.664726  1539 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:16.754755  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:16.655367  1541 pending_rounds.cc:85] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 6
I20260709 13:01:16.799942  1541 pending_rounds.cc:107] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:01:16.800412  1541 raft_consensus.cc:2889] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:16.794744  1688 rpcz_store.cc:276] 0709 13:01:15.411714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.411828 (+   114us) service_pool.cc:224] Handling call
0709 13:01:16.710680 (+1298852us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54002ce054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:16.710884 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2310135168}
I20260709 13:01:16.661696  5395 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:16.622287  4766 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:16.667768  5392 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:16.772552  1691 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:16.698860  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:16.826148  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 306241us
I20260709 13:01:16.828033  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:16.781105  5424 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:16.806996  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 132427us
I20260709 13:01:16.836032  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:16.798874  1618 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:16.837196  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 82565.8us
I20260709 13:01:16.831856  1691 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:16.762343  4916 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:16.772696  5342 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:16.849797  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:16.859867  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 254951us
I20260709 13:01:16.771710  5104 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.847240  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:16.739195  5439 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.156s
I20260709 13:01:16.863343  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:16.749342  5246 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:16.855880  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:16.798676  5436 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:16.858407  5440 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:16.769253  5333 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 13:01:16.851367  1618 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:16.880350  1618 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:16.818460  5267 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:16.833681  5392 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:16.862715  1615 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:16.878700  1657 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:16.864566  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.806000  5400 consensus_queue.cc:1059] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 13:01:16.862521  5156 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:16.867851  4661 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:16.793915  5428 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:16.881649  5348 consensus_meta.cc:287] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.307s	user 0.010s	sys 0.005s
I20260709 13:01:16.816443  5337 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:16.868690  5283 raft_consensus.cc:697] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:16.874495  4766 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:16.875962  5440 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.874464  5446 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.857165  5424 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:16.891570  5438 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:01:16.885128  5407 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:16.867074  4986 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.033s
W20260709 13:01:16.900796  5437 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.387s	user 0.002s	sys 0.000s
I20260709 13:01:16.892627  5267 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.898443  5104 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:16.916772  5440 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:16.907799  5450 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:16.906526  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 291514us
I20260709 13:01:16.967645  1618 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:01:16.968153  1618 pending_rounds.cc:85] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 6
I20260709 13:01:16.968461  1618 pending_rounds.cc:107] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:01:16.968776  1618 raft_consensus.cc:2889] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:16.916895  1611 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:16.893684  4916 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:01:16.904882  4661 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:16.947080  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 195559us
W20260709 13:01:16.947759  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:16.910087  4766 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:16.891688  5442 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:16.896673  5156 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:16.945132  4747 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:16.950080  5267 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:16.955542  5351 consensus_meta.cc:287] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.693s	user 0.008s	sys 0.002s
W20260709 13:01:16.955737  5259 consensus_meta.cc:287] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.696s	user 0.010s	sys 0.000s
I20260709 13:01:16.956658  5246 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:16.957283  5319 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:16.917019  5121 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:16.921211  5333 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:16.962025  5436 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:16.969204  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:17.013904  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.014750  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 45616.5us
W20260709 13:01:16.986320  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 123048us
I20260709 13:01:16.907124  5348 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:16.991200  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 149142us
I20260709 13:01:16.991811  5451 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 13:01:16.944083  5439 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:16.998638  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 117773us
W20260709 13:01:16.987367  5306 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:01:16.971899  5387 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:16.905524  5395 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:16.995222  4766 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:16.988935  5454 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:17.016186  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:16.985390  5402 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 13:01:17.001127  5156 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:16.923069  5446 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.127640  1685 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:17.018441  1535 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:17.000516  5426 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.003165  5453 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:17.006723  5259 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:17.049860  5387 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:01:17.056244  5362 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.716s	user 0.008s	sys 0.003s
I20260709 13:01:17.057597  5104 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:17.063323  4916 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:17.059713  1689 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:17.060241  5400 raft_consensus.cc:3055] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:17.045146  5178 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:17.128111  5395 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:17.131049  5407 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.043659  4661 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.003635  5442 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:17.050930  5267 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:17.169075  5445 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 13:01:17.047441  5432 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:17.046182  5424 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:17.013366  5441 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 13:01:17.018180  5436 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:17.120733  1536 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:17.063047  5434 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:17.005882  5351 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.124166  5310 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.022151  5333 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:17.131754  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 115616us
I20260709 13:01:17.212474  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:17.213253  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:17.134081  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:17.136657  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:17.123777  5458 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:01:17.058444  5428 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:17.073566  5468 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.221868  5468 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:17.224488  5468 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:17.012544  5121 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:17.146920  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.154322  1686 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:01:17.127784  5448 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:01:17.156908  2229 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:17.140000  5426 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:17.162492  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:17.234951  1692 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:17.060262  4747 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:17.059626  5246 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:01:17.172693  2237 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:01:17.012980  5283 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.133873  5453 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:17.143110  5451 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:17.150416  5467 raft_consensus.cc:3055] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:17.138114  5454 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:17.184894  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:17.267102  1689 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:17.189519  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:17.159864  5391 raft_consensus.cc:3055] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:17.043027  5439 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:17.191435  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:17.191588  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:17.158182  5400 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:17.128647  4986 consensus_queue.cc:1175] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" responder_term: 8 status { last_received { term: 8 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 8 index: 8 } }
I20260709 13:01:17.209594  2232 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:01:17.210391  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 62687.2us
I20260709 13:01:17.214339  1684 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:17.214443  1683 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:17.151960  5362 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.160032  5104 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:17.274701  5451 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:17.220721  5450 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:17.159780  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:17.216723  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:17.162256  4916 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.235944  2229 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:01:17.250169  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:17.254493  1616 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:17.255059  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 239403us
I20260709 13:01:17.184067  5407 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:17.258934  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:17.455374  1615 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:17.260073  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:17.200812  5267 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:01:17.268438  2228 raft_consensus.cc:2410] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 11 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:17.270354  2230 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:17.242952  5337 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:17.311573  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:17.263424  5467 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:17.216697  5310 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:17.335057  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.259174  5424 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:01:17.336342  2232 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:17.269748  5391 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:17.339483  1683 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:17.213203  2231 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:17.302771  5266 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:17.302897  5465 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:17.454278  5342 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.583s	user 0.011s	sys 0.000s
I20260709 13:01:17.475914  1689 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:01:17.303668  5446 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:17.279384  5443 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 13:01:17.481128  1683 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:17.369835  5400 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:17.383605  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 48746.3us
I20260709 13:01:17.377585  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:17.378046  5422 consensus_meta.cc:287] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.014s	user 0.010s	sys 0.003s
I20260709 13:01:17.411162  1614 raft_consensus.cc:2410] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:17.376317  5469 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.449777  1687 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:17.375576  5392 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:17.340556  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 87509.4us
I20260709 13:01:17.377885  1537 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:17.460480  5485 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:17.377099  5348 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:17.456173  1692 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
W20260709 13:01:17.456769  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 240113us
I20260709 13:01:17.458263  1686 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:01:17.464040  5259 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:17.540763  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:17.461024  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 301191us
W20260709 13:01:17.457571  5478 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.079s	user 0.007s	sys 0.000s
I20260709 13:01:17.455530  5387 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:17.304466  5442 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.490033  5467 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:17.491091  5479 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:01:17.491160  5451 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.481120  5465 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.477193  2232 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.336577  5333 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.396382  5440 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:17.518251  1683 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:01:17.515622  1684 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:17.522305  5480 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:17.524310  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:17.524065  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:17.524737  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:17.525255  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:17.542970  5246 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.564819  1618 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:17.544929  5400 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:17.475966  5471 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:17.554593  5104 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:17.553215  5484 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:17.569340  5471 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:01:17.532387  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.572573  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.452646  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:17.454360  5156 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.447347  5454 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.542347  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:17.580391  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 127795us
I20260709 13:01:17.547763  1685 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:17.409898  5453 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.481843  5342 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:17.556653  2232 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:17.585886  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 43613.9us
I20260709 13:01:17.587066  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:17.588322  1508 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:17.546195  5492 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:01:17.541312  5485 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:17.567430  5480 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.555047  5405 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
W20260709 13:01:17.538395  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.554318  5391 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.592487  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:17.593051  1540 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:17.552682  5362 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:17.597981  1538 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:01:17.603114  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.603729  1538 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:01:17.604631  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.558205  5387 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:17.555245  5348 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.565739  5467 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:17.536033  5469 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:17.610294  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.617280  1539 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:17.617931  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:17.614105  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.613147  1543 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:17.582351  5266 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:17.535115  5422 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:17.625020  1612 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:17.598492  5306 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.579939  5426 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:17.641342  1541 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:17.628115  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.630337  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:17.552037  5436 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.630242  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:17.624138  5466 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 13:01:17.636847  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.617810  5434 raft_consensus.cc:2764] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:01:17.638883  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:17.639523  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 81962.7us
I20260709 13:01:17.600834  5104 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:17.559929  5446 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:01:17.634624  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:17.643429  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:17.675004  2236 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:17.633613  5266 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.643831  1583 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:17.642226  1538 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:17.646981  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:17.597843  5337 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.621646  5461 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:17.659734  2231 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:17.650452  2229 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:17.613648  5387 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:17.580287  5246 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:17.637799  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:17.661201  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.684698  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:17.649816  5405 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:17.614508  5465 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:17.601161  5450 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:01:17.679522  5266 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:17.692090  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.650276  5004 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:17.613749  5178 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:01:17.646907  5362 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:17.696142  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.733682  5362 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:17.699707  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.736825  5362 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:17.660012  5439 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:17.625716  5348 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:17.663437  5391 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:17.676856  5283 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:17.713366  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.740329  5283 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.740797  5283 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:17.649621  5292 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:01:17.743191  1540 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:17.646231  5306 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.626099  5488 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.108s
I20260709 13:01:17.702827  5407 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:17.695806  5104 raft_consensus.cc:697] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:17.732213  1692 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:17.708504  5387 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:17.752823  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 164286us
I20260709 13:01:17.695286  5461 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:17.754956  5461 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:17.755411  1543 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:17.757390  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.749281  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:17.712256  5499 raft_consensus.cc:3055] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:17.699252  5337 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:17.716626  4916 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:17.759768  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:17.713819  5405 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:17.662130  5491 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 13:01:17.765306  1540 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:01:17.686493  5395 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:01:17.766422  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.772187  1541 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:17.781445  1543 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:17.785498  1539 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 13 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:01:17.788954  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.746470  5362 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.792375  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:17.794924  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.803018  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:17.558861  5399 consensus_queue.cc:1175] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" responder_term: 8 status { last_received { term: 8 index: 10 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:17.804411  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:17.763022  5246 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:17.807047  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.791882  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.812475  1543 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:17.814415  5342 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:17.815343  5156 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:17.815851  5494 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:01:17.768350  5499 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:17.786742  5392 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:17.806419  5447 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.837s	user 0.000s	sys 0.006s
I20260709 13:01:17.913784  2232 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
W20260709 13:01:17.822448  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 129976us
W20260709 13:01:17.847396  5432 consensus_meta.cc:287] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.644s	user 0.007s	sys 0.002s
I20260709 13:01:17.892969  5104 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.904177  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:17.827275  1539 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:17.917821  1618 raft_consensus.cc:2410] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 12 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 13:01:17.832471  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.918737  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.918991  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:17.920913  1539 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:17.847020  1540 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:17.819870  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.922091  5505 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.930497  5505 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.928303  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:17.871479  5306 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:17.932678  1618 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:17.924314  1543 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:01:17.936631  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:17.937784  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:17.915253  5499 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.942510  1540 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:17.944438  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.946205  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:17.946839  1686 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:17.947283  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.948025  1539 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:17.951828  1618 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:17.953148  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:17.954319  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:17.941493  2201 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:17.818598  5310 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:17.882225  5348 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:17.868364  5492 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.966017  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:17.955878  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:17.956599  5493 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.280s	user 0.002s	sys 0.000s
I20260709 13:01:17.853518  5342 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:17.862646  5440 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:17.964367  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:17.845899  5337 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:17.844450  5469 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:17.959746  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:17.962136  5173 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:17.968400  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:17.969285  5507 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:01:17.852306  5428 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 13:01:17.969615  5502 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:17.961968  5506 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:01:17.931949  5392 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:18.017871  5338 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:17.985562  5246 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:17.873643  5405 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.972463  5505 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:17.990942  5492 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:17.940096  5480 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:17.959038  5485 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:17.993921  5512 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:01:17.989684  5498 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 13:01:17.932492  5488 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
W20260709 13:01:17.998291  5496 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:01:18.100255  5502 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:17.805500  5362 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:17.983044  5510 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:17.916231  5432 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:18.106411  5487 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.124243  5178 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:18.124061  5487 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:18.124675  5178 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:18.024516  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 50298.3us
I20260709 13:01:18.125522  5338 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.126124  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:18.127775  5338 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:18.127707  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:17.985715  5348 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:18.026865  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:18.027344  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:17.962452  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 170157us
I20260709 13:01:18.035377  1543 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:18.027911  5509 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:01:18.040271  5460 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.027828  5156 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 13:01:17.966991  1689 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:18.045717  1691 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 6.
W20260709 13:01:18.041234  4766 consensus_meta.cc:287] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.913s	user 0.004s	sys 0.006s
I20260709 13:01:18.030567  5499 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:18.005702  5503 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.132s
I20260709 13:01:18.131075  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:18.123167  1609 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:01:18.130686  1542 raft_consensus.cc:2410] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:01:18.147711  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.130720  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.133548  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.136775  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.049917  5453 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:18.149063  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.150708  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:18.153594  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:18.154119  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:18.032231  5310 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:01:18.146216  1582 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:18.165638  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:18.168352  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.170374  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:18.032344  5342 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.174064  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:18.034360  5266 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:18.114562  5515 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:01:18.114650  5506 raft_consensus.cc:3055] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:18.174208  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:18.175866  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:18.178589  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 47585.8us
I20260709 13:01:18.182403  2230 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:18.183262  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:18.121809  5502 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:18.179160  1609 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:01:18.179839  2228 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:18.118000  5512 consensus_queue.cc:1243] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:01:18.123381  5510 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:18.184856  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.186457  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:18.215111  1689 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:18.123399  5362 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:18.158846  5163 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:18.182471  1692 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:18.215533  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:18.187095  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:18.169792  5488 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:18.106323  5392 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:18.174016  5306 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:18.227316  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:18.184265  5395 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:18.164017  5446 consensus_queue.cc:1175] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "984c9089d9804652846f3caeb609d56e" responder_term: 8 status { last_received { term: 7 index: 8 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:18.232391  2233 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:01:18.232577  1686 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:18.180019  5405 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:18.174239  5319 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:18.235679  2203 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:18.236872  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:18.238507  1688 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:01:18.231205  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:18.195397  5506 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:18.217566  5432 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:18.250876  1583 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:18.226488  5488 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.247740  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:18.256507  2230 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:18.248785  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:18.257118  2233 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.215235  5348 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:18.250792  2234 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:18.158553  4766 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.263069  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:18.224697  5454 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.185590  5426 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 13:01:18.217363  5510 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:18.261987  1689 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:01:18.291042  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:18.234382  5451 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.219877  5163 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.159197  5460 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:18.234951  5306 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:18.243302  5440 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:18.240128  5395 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.157493  5178 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:18.263872  1692 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:18.288879  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:18.251791  5319 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.254581  5337 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:18.322818  5337 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.258404  5432 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.323244  2230 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:18.323441  5432 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:18.323244  5337 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.310923  5163 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:18.311771  2234 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:18.293598  5501 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.177s
W20260709 13:01:18.330598  5004 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.588s	user 0.009s	sys 0.000s
I20260709 13:01:18.325575  5319 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:18.331094  5487 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:18.331949  1618 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:18.332971  4661 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.260942  5488 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.333498  4661 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:18.338758  5392 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:18.339927  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1616) took 1758 ms. Trace:
I20260709 13:01:18.340888  5266 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:18.273846  5292 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:18.341471  5511 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:18.341707  5292 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.342198  5292 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.342247  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 632) took 1919 ms. Trace:
I20260709 13:01:18.329345  5422 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:18.342660  1616 rpcz_store.cc:276] 0709 13:01:16.422525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.422704 (+   179us) service_pool.cc:224] Handling call
0709 13:01:16.423117 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.452332 (+1029215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.341993 (+889661us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b54001a63d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:18.342189 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1877442432}
W20260709 13:01:18.345924  1537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 707) took 1926 ms (client timeout 1905 ms). Trace:
W20260709 13:01:18.346346  1537 rpcz_store.cc:269] 0709 13:01:16.419396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.419699 (+   303us) service_pool.cc:224] Handling call
0709 13:01:18.345687 (+1925988us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b54000ae454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:18.345821 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2097449728}
I20260709 13:01:18.346904  5460 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:18.347993  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 712) took 1873 ms. Trace:
I20260709 13:01:18.348369  1536 rpcz_store.cc:276] 0709 13:01:16.474715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.474980 (+   265us) service_pool.cc:224] Handling call
0709 13:01:16.475308 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.203852 (+728544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.347743 (+1143891us) spinlock_profiling.cc:243] Waited 645 ms on lock 0x7b540030a7d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:18.347934 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1483737344}
W20260709 13:01:18.308884  5521 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 13:01:18.340422  1613 rpcz_store.cc:276] 0709 13:01:16.581211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.785620 (+204409us) service_pool.cc:224] Handling call
0709 13:01:16.786063 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:16.949180 (+163117us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:16.990225 (+ 41045us) log.cc:844] Serialized 14 byte log entry
0709 13:01:17.937068 (+946843us) spinlock_profiling.cc:243] Waited 2.8 ms on lock 0x7b18003ed760. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08f6944bc 00007fe08c9e248b 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fc455 00007fe0934e43c7 00007fe08faf69a7 00007fe08fb060bf
0709 13:01:17.952852 (+ 15784us) log.cc:844] Serialized 14 byte log entry
0709 13:01:17.953121 (+   269us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:17.953193 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:17.953255 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.339632 (+386377us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54002bd754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:18.339696 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:18.339861 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54524,"prepare.queue_time_us":117336,"prepare.run_cpu_time_us":163,"prepare.run_wall_time_us":8177,"raft.queue_time_us":18087,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":15773,"spinlock_wait_cycles":507839360,"thread_start_us":214577,"threads_started":2,"wal-append.queue_time_us":219710}
W20260709 13:01:18.353621  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 170424us
I20260709 13:01:18.312281  5471 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:18.317186  5306 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:18.350539  5395 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:18.363210  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 127624us
W20260709 13:01:18.354872  5283 consensus_meta.cc:287] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.614s	user 0.008s	sys 0.007s
I20260709 13:01:18.339502  5337 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.319878  5440 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.356317  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:18.289026  5454 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.337334  4661 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:18.360419  5467 consensus_meta.cc:287] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.736s	user 0.013s	sys 0.002s
I20260709 13:01:18.407117  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:18.409379  5454 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:18.410006  5422 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:18.410399  5422 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.410728  5422 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:18.412629  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.399902  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.384820  5461 consensus_meta.cc:287] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.629s	user 0.007s	sys 0.007s
W20260709 13:01:18.380447  5400 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.797s	user 0.009s	sys 0.006s
W20260709 13:01:18.417555  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:18.404409  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 48181.7us
I20260709 13:01:18.429411  1508 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:18.385901  5460 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.311287  5451 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.413131  5440 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:18.445734  2203 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:18.360771  5488 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:18.359917  5348 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:18.423949  5469 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:18.483083  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 53772.5us
W20260709 13:01:18.493883  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 50117.3us
I20260709 13:01:18.403242  5283 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:18.502147  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56499.4us
I20260709 13:01:18.507807  1507 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:18.507294  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:18.513280  1689 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:18.512439  2203 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:18.514601  2234 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:18.498363  4661 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.428359  5461 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.504637  5487 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:18.525799  1692 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:18.517354  5387 consensus_meta.cc:287] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.764s	user 0.006s	sys 0.007s
I20260709 13:01:18.504980  5511 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:18.506421  5492 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:18.445724  5460 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:18.518654  5501 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.504082  5467 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:18.511559  5469 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.546281  5337 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:18.549582  5517 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.190s	user 0.000s	sys 0.002s
W20260709 13:01:18.550626  5528 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:01:18.552687  5306 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:18.560328  5520 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 13:01:18.560207  5283 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:18.376310  5506 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.492772  5266 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:18.566556  5534 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:18.582489  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 70126.3us
I20260709 13:01:18.556823  5467 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:18.537009  5387 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:18.562677  5319 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:18.586635  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 79421.1us
I20260709 13:01:18.578253  2233 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:18.529194  4661 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.561473  5536 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.008s
W20260709 13:01:18.591027  5532 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 13:01:18.535692  4766 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:18.592927  5535 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:18.593272  5451 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:18.593864  4661 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:18.594609  5524 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:01:18.536104  5487 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:18.589614  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:18.597932  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:18.603585  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:18.605094  2233 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:01:18.607329  2233 pending_rounds.cc:85] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 6
I20260709 13:01:18.606835  1657 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:18.607999  2233 pending_rounds.cc:107] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:01:18.612372  2233 raft_consensus.cc:2889] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:18.611732  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:18.655521  1657 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:18.650964  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.587982  5467 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:18.642618  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.667897  1543 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:01:18.625449  5451 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:01:18.588742  5266 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:18.673010  5499 consensus_meta.cc:287] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.508s	user 0.012s	sys 0.000s
I20260709 13:01:18.595178  5527 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:18.658623  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46982.3us
I20260709 13:01:18.695993  2236 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:18.696189  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:18.672134  5319 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:18.697338  5469 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:18.587296  5454 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.071s
W20260709 13:01:18.674860  5538 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:01:18.668643  5306 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:18.670614  5460 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:18.668334  5471 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:18.699920  5454 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:18.700133  5471 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:18.679154  5526 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:01:18.679087  5422 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.700526  5454 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.656806  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:18.606555  5506 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:18.702557  5422 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:18.657269  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 60580.5us
I20260709 13:01:18.693549  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:18.704566  5454 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:18.710314  2232 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: [7.8-10.10]
W20260709 13:01:18.712186  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.723877  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:18.724984  2232 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:18.687479  5348 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:18.587491  5342 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:18.701848  1536 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
W20260709 13:01:18.722597  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.736343  5499 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.737129  5499 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.565580  5501 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.615871  5283 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:18.575057  5432 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:18.673342  5391 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.934s	user 0.012s	sys 0.000s
I20260709 13:01:18.620852  5104 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:18.752977  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 48163.6us
I20260709 13:01:18.675539  5387 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:18.684170  5453 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.005s
I20260709 13:01:18.586031  5534 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:18.761955  5422 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:01:18.654484  5535 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:18.596556  5511 consensus_queue.cc:1175] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "83f6f5810950417195f707ff308a8125" responder_term: 10 status { last_received { term: 8 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:18.761722  1430 maintenance_manager.cc:419] P 1ca716594c6348a883b88d48068b8416: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033349
I20260709 13:01:18.786762  1539 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:18.760300  5469 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:18.765053  5533 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.760640  5499 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:18.761683  5529 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.770207  2232 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
W20260709 13:01:18.796511  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 288757us
I20260709 13:01:18.771548  5488 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:18.808205  1508 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:18.767582  5283 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:18.811751  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 115682us
I20260709 13:01:18.809389  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:18.815485  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:18.811841  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 156387us
I20260709 13:01:18.772652  5395 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:18.817592  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:18.810755  5292 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.821540  5266 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:18.822903  5292 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:18.823431  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1620) took 1693 ms. Trace:
I20260709 13:01:18.823864  1685 rpcz_store.cc:276] 0709 13:01:17.130320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.130465 (+   145us) service_pool.cc:224] Handling call
0709 13:01:17.156061 (+ 25596us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.581609 (+425548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.823166 (+1241557us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b5400375354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:18.823353 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":899561728}
W20260709 13:01:18.828771  1584 connection.cc:256] RPC call timeout handler was delayed by -1.3518288279999524s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1962s for a response
I20260709 13:01:18.813268  5461 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:18.830204  5469 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.830796  5469 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.825227  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:18.833039  5469 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:18.833940  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:18.833243  5501 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:18.792548  5535 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.810612  5533 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:18.835165  1614 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 629) took 2432 ms (client timeout 2069 ms). Trace:
W20260709 13:01:18.835765  1584 connection.cc:256] RPC call timeout handler was delayed by -1.1916236389999426s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.166s for a response
I20260709 13:01:18.835851  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1635) took 1098 ms. Trace:
I20260709 13:01:18.840413  1507 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:18.853091  1614 rpcz_store.cc:269] 0709 13:01:16.403103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.403655 (+   552us) service_pool.cc:224] Handling call
0709 13:01:18.834924 (+2431269us) spinlock_profiling.cc:243] Waited 1e+03 ms on lock 0x7b54002cbd54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:18.835077 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2299273088}
W20260709 13:01:18.833134  5464 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.621s	user 0.001s	sys 0.004s
I20260709 13:01:18.853580  1618 rpcz_store.cc:276] 0709 13:01:17.737263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.955656 (+218393us) service_pool.cc:224] Handling call
0709 13:01:18.835699 (+880043us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b540028ffd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:18.835796 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839360256}
I20260709 13:01:18.837729  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:01:18.828200  5163 consensus_meta.cc:287] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.000s
W20260709 13:01:18.815642  5405 consensus_meta.cc:287] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.002s
I20260709 13:01:18.801506  5527 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:01:18.858162  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:18.858217  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:18.852492  5510 consensus_meta.cc:287] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.002s
I20260709 13:01:18.863901  1657 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:18.857060  5502 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.625s	user 0.010s	sys 0.000s
I20260709 13:01:18.817950  5544 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:18.851220  5348 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:18.887809  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:18.838758  5506 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:18.889983  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:18.898231  1543 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:18.817021  5529 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:18.900878  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.854440  5467 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.886775  5348 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.908318  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:18.914752  1685 raft_consensus.cc:2368] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8.
I20260709 13:01:18.914856  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:18.915688  2232 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:18.881959  5549 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:18.920007  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:18.918051  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 57283.8us
I20260709 13:01:18.921190  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:18.906247  5487 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:18.922674  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:18.858570  5163 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:18.873274  5510 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.915205  5348 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:18.929966  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.903098  5545 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:18.893062  5502 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.926708  2203 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:18.933912  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:18.937801  2203 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:18.942512  5534 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:18.942708  5547 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:18.943310  5547 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:18.944211  5547 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.945024  5502 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:18.954293  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:18.955228  2237 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:18.957850  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:18.958777  2237 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:18.962961  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:18.963644  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:18.940806  5395 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:18.941560  5405 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:18.942005  5499 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:18.972223  5560 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:18.973206  1657 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:18.963864  5553 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.097s	user 0.000s	sys 0.002s
I20260709 13:01:18.982089  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 711) took 2525 ms. Trace:
W20260709 13:01:18.982349  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.982568  1535 rpcz_store.cc:276] 0709 13:01:16.452800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.452981 (+   181us) service_pool.cc:224] Handling call
0709 13:01:16.474281 (+ 21300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.129242 (+654961us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:18.977802 (+1848560us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b54003458d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:18.977985 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1250926464}
W20260709 13:01:18.985049  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.987953  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:18.933550  5552 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 13:01:18.991286  5455 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:18.993224  1684 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:18.991968  5455 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:18.977790  5499 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.976811  5544 raft_consensus.cc:697] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:19.002673  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:18.987257  5395 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:18.985929  5560 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:18.940282  5546 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.160s
I20260709 13:01:19.002815  5549 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:19.077998  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.015035  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:19.071689  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.132889  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.686110  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.658535  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1136) took 1431 ms. Trace:
I20260709 13:01:19.690342  2234 rpcz_store.cc:276] 0709 13:01:18.226640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.330026 (+103386us) service_pool.cc:224] Handling call
0709 13:01:18.330310 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:18.514951 (+184641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.658173 (+1143222us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54003d3bd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:19.658448 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423172480}
W20260709 13:01:19.054396  5554 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:01:19.000080  5499 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:19.108105  5440 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.607s	user 0.012s	sys 0.000s
W20260709 13:01:19.026593  5477 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.769s	user 0.005s	sys 0.000s
I20260709 13:01:19.739277  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:19.009858  5395 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:19.096024  5163 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:19.059489  5452 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 2.001s	user 0.007s	sys 0.009s
W20260709 13:01:19.740324  1612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1625) took 3072 ms (client timeout 2955 ms). Trace:
I20260709 13:01:19.007470  5535 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:19.022480  5467 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:19.059914  5407 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:19.054557  5351 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:19.022619  5510 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:19.058218  5533 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:19.743768  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.743680  1612 rpcz_store.cc:269] 0709 13:01:16.667753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.718261 (+ 50508us) service_pool.cc:224] Handling call
0709 13:01:19.740124 (+3021863us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b5400001c54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:19.740254 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1900166528}
I20260709 13:01:19.058575  5529 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:19.744956  1690 raft_consensus.cc:1217] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Deduplicated request from leader. Original: 6.5->[7.6-8.7]   Dedup: 7.6->[8.7-8.7]
I20260709 13:01:19.062341  5488 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:19.704664  5560 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:19.752948  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.753902  1690 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: [7.6-8.7]
I20260709 13:01:19.759600  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2496) took 2613 ms. Trace:
I20260709 13:01:19.759943  1690 rpcz_store.cc:276] 0709 13:01:17.146314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.146705 (+   391us) service_pool.cc:224] Handling call
0709 13:01:17.168771 (+ 22066us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:19.744802 (+2576031us) spinlock_profiling.cc:243] Waited 93.8 ms on lock 0x7b54001e94a0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa63dc0 00007fe08faee1c0 00007fe08faef231 00007fe08faebd8f 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c
0709 13:01:19.759268 (+ 14466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.759514 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215662976}
I20260709 13:01:19.025967  5544 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:19.743793  5395 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:19.742612  5440 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:19.754726  5533 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.752892  5455 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:19.757050  5529 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.757728  5564 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:19.750854  5407 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:19.782864  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:19.788709  1690 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:19.760699  5488 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:19.806856  5474 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 2.551s	user 0.006s	sys 0.024s
I20260709 13:01:19.808862  5407 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:19.777526  5549 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:19.815016  5440 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:19.786911  5533 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:19.792675  5348 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.854s	user 0.006s	sys 0.006s
I20260709 13:01:19.822680  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 742) took 2120 ms. Trace:
I20260709 13:01:19.844448  1688 rpcz_store.cc:276] 0709 13:01:17.702303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.706194 (+  3891us) service_pool.cc:224] Handling call
0709 13:01:17.706571 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:18.250362 (+543791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.822395 (+1572033us) spinlock_profiling.cc:243] Waited 532 ms on lock 0x7b540028d7d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:19.822616 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1222504192}
W20260709 13:01:19.820716  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:19.846886  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.831691  5407 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:19.838397  1690 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:19.790879  5564 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:19.849754  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:19.850103  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:19.851148  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:19.851930  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 751) took 1100 ms. Trace:
I20260709 13:01:19.852240  1688 rpcz_store.cc:276] 0709 13:01:18.751840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.850909 (+1099069us) service_pool.cc:224] Handling call
0709 13:01:19.851860 (+   951us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.791186  5529 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:19.857015  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:19.858067  1688 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:19.834067  5499 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:19.850250  5348 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:19.870141  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1621) took 3284 ms. Trace:
I20260709 13:01:19.870939  1617 rpcz_store.cc:276] 0709 13:01:16.582606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:16.964022 (+381416us) service_pool.cc:224] Handling call
0709 13:01:16.964369 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:16.964718 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:16.964752 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:16.965213 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:01:17.144367 (+179154us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:17.144417 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:17.144492 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:19.783545 (+2639053us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:19.864176 (+ 80631us) raft_consensus.cc:1685] finished
0709 13:01:19.864197 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.867206 (+  3009us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1773,"raft.queue_time_us":3885,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":7076,"spinlock_wait_cycles":51836288,"thread_start_us":178706,"threads_started":1,"wal-append.queue_time_us":233770}
I20260709 13:01:19.854822  5407 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:19.894713  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:19.900156  1690 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:19.911312  5570 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:19.911979  5570 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:19.914254  5499 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:19.914561  5570 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:19.915196  5570 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:19.927259  5547 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.927923  5547 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:19.930310  5547 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:19.932250  1688 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:19.933665  5572 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:19.934105  1690 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:19.934257  5572 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:19.938023  5533 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:19.940531  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:19.946364  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:19.947551  5529 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:19.944765  5485 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:19.950850  5485 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:19.951380  5121 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:19.951712  5485 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:19.947739  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:19.953986  5488 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:19.953920  1688 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:19.955725  5348 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:19.956760  5563 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:01:19.958029  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2505) took 1829 ms. Trace:
I20260709 13:01:19.958381  1683 rpcz_store.cc:276] 0709 13:01:18.128805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.349924 (+221119us) service_pool.cc:224] Handling call
0709 13:01:18.350286 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:18.381712 (+ 31426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:18.381761 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:19.099259 (+717498us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b2c00144828. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:01:19.099851 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:01:19.619893 (+520042us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:19.619939 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.620080 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:19.957710 (+337630us) raft_consensus.cc:1685] finished
0709 13:01:19.957734 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.957963 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31199,"raft.queue_time_us":53371,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":6242,"spinlock_wait_cycles":4260608,"thread_start_us":293445,"threads_started":1,"wal-append.queue_time_us":645489}
I20260709 13:01:19.959200  5560 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:19.959746  1689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2511) took 1793 ms (client timeout 2094 ms). Trace:
W20260709 13:01:19.960166  1689 rpcz_store.cc:269] 0709 13:01:18.166749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.290723 (+123974us) service_pool.cc:224] Handling call
0709 13:01:19.959547 (+1668824us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54000aafd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:19.959689 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509126912}
I20260709 13:01:19.960518  1684 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:19.960623  1683 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:19.961319  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2506) took 1832 ms. Trace:
I20260709 13:01:19.961670  1683 rpcz_store.cc:276] 0709 13:01:18.129073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.960129 (+1831056us) service_pool.cc:224] Handling call
0709 13:01:19.960448 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.961102 (+   654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.961271 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.962621  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2512) took 1795 ms. Trace:
I20260709 13:01:19.963238  1684 rpcz_store.cc:276] 0709 13:01:18.167078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.961782 (+1794704us) service_pool.cc:224] Handling call
0709 13:01:19.962094 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.962281 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.962305 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.962351 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:19.962407 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.962471 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.962568 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.965556  1684 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:01:19.963302  1683 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:01:19.967506  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2513) took 1800 ms. Trace:
I20260709 13:01:19.961339  1689 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:19.969676  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 746) took 1816 ms. Trace:
I20260709 13:01:19.970070  1689 rpcz_store.cc:276] 0709 13:01:18.153160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.960932 (+1807772us) service_pool.cc:224] Handling call
0709 13:01:19.961191 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.969304 (+  8113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.969607 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.973245  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1632) took 1089 ms. Trace:
I20260709 13:01:19.976296  1689 rpcz_store.cc:276] 0709 13:01:18.883211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.972011 (+1088800us) service_pool.cc:224] Handling call
0709 13:01:19.972404 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.972630 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.972666 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.972722 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:19.972769 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.972837 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.972971 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.978649  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1633) took 1095 ms. Trace:
I20260709 13:01:19.979103  1689 rpcz_store.cc:276] 0709 13:01:18.883484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.977611 (+1094127us) service_pool.cc:224] Handling call
0709 13:01:19.978001 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.978220 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.978257 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.978311 (+    54us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:19.978355 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.978438 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.978580 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.968505  1683 rpcz_store.cc:276] 0709 13:01:18.167408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.962642 (+1795234us) service_pool.cc:224] Handling call
0709 13:01:19.962935 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.967173 (+  4238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.967437 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:19.982631  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 87788.4us
I20260709 13:01:19.973219  1688 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:19.984076  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1644) took 1064 ms. Trace:
I20260709 13:01:19.975231  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 747) took 1699 ms. Trace:
I20260709 13:01:19.984920  1684 rpcz_store.cc:276] 0709 13:01:18.276292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.964413 (+1688121us) service_pool.cc:224] Handling call
0709 13:01:19.964795 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:19.974645 (+  9850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.975141 (+   496us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.985371  5580 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.982565  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1636) took 1094 ms. Trace:
I20260709 13:01:19.985984  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:19.986639  2237 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:19.986461  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:19.988621  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1637) took 1094 ms. Trace:
I20260709 13:01:19.990082  2229 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:19.986451  5573 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:19.991930  5573 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:19.992435  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:19.993316  2233 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 6.
I20260709 13:01:19.989791  1684 rpcz_store.cc:276] 0709 13:01:18.894587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.987510 (+1092923us) service_pool.cc:224] Handling call
0709 13:01:19.987952 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.988184 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.988223 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.988281 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:19.988327 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.988409 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.988547 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.990854  1688 rpcz_store.cc:276] 0709 13:01:18.919590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.947498 (+1027908us) service_pool.cc:224] Handling call
0709 13:01:19.984001 (+ 36503us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.986254  1683 rpcz_store.cc:276] 0709 13:01:18.887799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.981559 (+1093760us) service_pool.cc:224] Handling call
0709 13:01:19.981934 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.982124 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.982161 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.982218 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:19.982263 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.982338 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.982494 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.944715  4916 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:20.005932  4916 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:19.965471  5499 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:20.007118  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1646) took 1021 ms. Trace:
W20260709 13:01:19.998670  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.008916  5407 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:20.010159  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2088) took 2063 ms. Trace:
I20260709 13:01:20.010794  5395 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:19.953260  5485 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:20.007117  4916 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.007220  5499 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.010593  1540 rpcz_store.cc:276] 0709 13:01:17.946549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.958052 (+ 11503us) service_pool.cc:224] Handling call
0709 13:01:17.958362 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:18.123732 (+165370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:18.123763 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:18.124263 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 13:01:18.176015 (+ 51752us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:18.176073 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.009737 (+1833664us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54003e2954. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:20.009784 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.009875 (+    91us) raft_consensus.cc:1685] finished
0709 13:01:20.009887 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.010101 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":165,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":71,"spinlock_wait_cycles":379438976,"thread_start_us":51287,"threads_started":1,"wal-append.queue_time_us":51660,"wal-append.run_cpu_time_us":1703,"wal-append.run_wall_time_us":1010612}
W20260709 13:01:20.017719  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.008098  1688 rpcz_store.cc:276] 0709 13:01:18.977344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.995360 (+1018016us) service_pool.cc:224] Handling call
0709 13:01:19.995760 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:19.998179 (+  2419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:19.998224 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:19.998291 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:19.998343 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:19.998447 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:19.998616 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:19.986088  5580 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.011008  5560 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:20.013067  5395 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:20.008790  5446 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.045s
W20260709 13:01:20.057113  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 72885.4us
I20260709 13:01:20.056833  5395 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.057458  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:20.060631  2234 raft_consensus.cc:2393] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:20.062139  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:20.061805  5499 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:20.063244  2234 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:20.058179  5173 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:20.054936  5578 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:20.115684  1658 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.523s (SENT) [suppressed 12 similar messages]
I20260709 13:01:20.056250  5575 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.110760  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:20.118296  5173 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.128767  2234 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:20.129796  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:20.130354  2234 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:20.132292  5580 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:20.138298  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 1.89308e+06us
I20260709 13:01:20.138734  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:20.136519  1441 catalog_manager.cc:5697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:20.140539  2234 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:20.140599  1441 catalog_manager.cc:5697] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 4 to 5, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:01:20.142913  1441 catalog_manager.cc:5697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 3 to 5, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:01:20.144891  1441 catalog_manager.cc:5697] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 4 to 6, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:01:20.147753  5362 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:20.148308  1507 leader_election.cc:400] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8.
I20260709 13:01:20.148705  1507 leader_election.cc:403] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:01:20.148653  5431 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:20.148365  5362 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.150265  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:20.150841  1683 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 6.
I20260709 13:01:20.151695  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:20.151970  5362 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:20.149646  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:20.153151  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:20.153542  1691 raft_consensus.cc:2393] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:20.146661  1441 catalog_manager.cc:5697] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 3 to 4, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:20.155177  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:20.160008  5348 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:20.161338  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.163113  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.165284  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:20.165966  1691 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.169452  1683 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 7: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:20.169617  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:20.170083  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:20.170347  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:20.171088  1539 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:20.171521  1683 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:20.172320  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:20.173380  1540 raft_consensus.cc:2410] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:20.173237  1507 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:20.174573  5542 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:20.182934  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.182761  1441 catalog_manager.cc:5697] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 6 to 7, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:01:20.184510  1691 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:20.185777  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.186561  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:20.186875  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:20.187273  1613 raft_consensus.cc:2368] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10.
I20260709 13:01:20.187729  1540 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:20.188674  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.189532  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:20.189989  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:20.185961  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:20.191843  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:20.192646  1539 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:20.193171  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1640) took 1287 ms. Trace:
I20260709 13:01:20.176568  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:20.199079  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:20.199990  1535 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:20.200704  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:20.198084  1441 catalog_manager.cc:5697] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 3 to 4, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 4 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:20.201689  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:20.202339  1543 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:20.202487  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.203554  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:20.203783  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.204435  1614 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:20.206033  1611 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:01:20.207998  1610 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:20.209846  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:20.210525  1537 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:20.063907  5395 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.196033  1617 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:20.196967  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.194765  5426 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:20.213495  1612 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:20.217777  5426 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:20.195289  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:20.219198  1539 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:20.174700  5544 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:20.182549  5492 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.177810  5580 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:01:20.158138  5575 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.120957  5578 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:20.294394  1541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2636) took 2702 ms (client timeout 1962 ms). Trace:
I20260709 13:01:20.221177  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:20.195816  1684 rpcz_store.cc:276] 0709 13:01:18.905621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.994956 (+1089335us) service_pool.cc:224] Handling call
0709 13:01:20.006111 (+ 11155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.006578 (+   467us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:20.053674 (+ 47096us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.192777 (+139103us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:20.192826 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:20.192867 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.192928 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.193034 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":51253,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":1594,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":10768,"thread_start_us":185388,"threads_started":2}
I20260709 13:01:20.242699  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 745) took 2215 ms. Trace:
I20260709 13:01:20.348450  1537 pending_rounds.cc:85] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 2
I20260709 13:01:20.285156  1688 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:20.256057  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:20.292483  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:20.345527  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:20.227355  1441 catalog_manager.cc:5697] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:20.241401  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.247212  1689 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:20.298683  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:20.349857  1538 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:20.246960  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 753) took 1218 ms. Trace:
I20260709 13:01:20.359309  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:20.373406  1541 rpcz_store.cc:269] 0709 13:01:17.591631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.592262 (+   631us) service_pool.cc:224] Handling call
0709 13:01:20.294081 (+2701819us) spinlock_profiling.cc:243] Waited 39.5 ms on lock 0x7b54000e97d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:20.294253 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90776448}
W20260709 13:01:20.352702  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.245342  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:20.395112  1609 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:20.375403  1537 pending_rounds.cc:107] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:01:20.380833  1684 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22357a3949344a1792e5bae4a0291987"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:01:20.353652  5548 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:20.396750  1537 raft_consensus.cc:2889] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:20.396791  1618 raft_consensus.cc:2410] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 10 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:20.221346  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.397941  1683 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:20.398350  1616 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:20.398670  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.381551  5178 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.399556  1683 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:20.383378  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:20.390750  5173 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.389616  5470 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:20.281190  5436 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:20.198432  5362 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:20.389588  5575 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:20.389724  5468 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:20.342489  5588 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:20.284792  5395 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:20.276532  5432 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:20.236675  5283 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:20.392270  1617 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:20.440160  1609 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:20.252998  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:20.402436  5310 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.376461  2237 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:20.444305  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:20.394910  1441 catalog_manager.cc:5697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 4 to 5, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 5 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:20.396246  1684 raft_consensus.cc:606] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:20.445056  1684 raft_consensus.cc:614] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:20.375276  1692 rpcz_store.cc:276] 0709 13:01:18.027602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.355524 (+327922us) service_pool.cc:224] Handling call
0709 13:01:18.360214 (+  4690us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:18.534817 (+174603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.242460 (+1707643us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b540003e654. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:20.242635 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380371840}
I20260709 13:01:20.443996  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:20.200362  5483 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:20.253607  5540 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.500s	user 0.011s	sys 0.017s
I20260709 13:01:20.451185  1441 catalog_manager.cc:5697] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 4 to 6, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 6 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:01:20.378575  1690 rpcz_store.cc:276] 0709 13:01:19.028161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.849797 (+821636us) service_pool.cc:224] Handling call
0709 13:01:20.246756 (+396959us) spinlock_profiling.cc:243] Waited 49.1 ms on lock 0x7b540014a354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:20.246894 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112956672}
I20260709 13:01:20.223312  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.383127  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:20.233345  5581 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 13:01:20.324741  5567 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:20.279281  5582 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 13:01:20.278342  5579 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:01:20.378954  1539 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:20.220391  5426 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:20.376255  1686 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:20.404064  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.419302  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.195289  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:20.211485  5440 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:20.400213  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 284557us
I20260709 13:01:20.477859  1441 catalog_manager.cc:5697] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 5 to 7, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:20.393785  5266 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:20.418443  5548 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.299163  5481 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.502108  5481 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.503190  1685 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9. Ops: [6.4-8.5]
I20260709 13:01:20.422928  5470 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:20.422946  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.299721  5422 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.508109  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.508940  1616 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:20.379245  5573 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:20.325753  5586 log.cc:927] Time spent T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:01:20.284180  5348 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8.
I20260709 13:01:20.418996  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:20.393978  5338 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.254868  5559 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:20.311278  1657 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:20.440981  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 194131us
W20260709 13:01:20.393266  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 94577us
I20260709 13:01:20.441943  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.442118  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:20.402169  5457 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.448654  5436 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:20.449518  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:20.449800  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:20.450114  1692 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:01:20.453070  2234 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:20.403378  5572 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:20.459265  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.394984  5492 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.468216  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:20.471920  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:20.474944  1691 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:20.247889  5461 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:20.484342  5575 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:20.537806  1618 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:20.549995  1613 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:20.549914  5457 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:20.550817  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.551684  1618 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:20.552188  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:20.552723  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.504568  5533 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:20.553802  5533 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.548905  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.555557  1611 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:20.556178  2237 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:20.556849  1611 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:20.557068  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:20.506579  5440 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:20.547900  5387 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:20.558028  2237 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:20.557876  5575 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.547029  1610 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:01:20.558430  5440 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.559080  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:20.559921  2237 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:20.560231  5575 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:20.560743  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:20.535228  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 726) took 1674 ms. Trace:
I20260709 13:01:20.561605  1615 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:01:20.532763  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1133) took 2307 ms. Trace:
I20260709 13:01:20.490449  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:20.489171  5395 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:20.467680  5442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.325s	user 0.024s	sys 0.022s
I20260709 13:01:20.457935  5483 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:20.498919  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 956) took 3296 ms. Trace:
I20260709 13:01:20.298956  5542 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.484815  1441 catalog_manager.cc:5697] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 6 to 7, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } }
I20260709 13:01:20.504709  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:20.505435  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:20.503456  5548 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:20.510622  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:20.437783  5468 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:20.514972  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:20.517926  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:20.469647  5432 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:01:20.525902  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.558075  5387 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:20.488188  5578 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:20.441344  5588 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.456871  5310 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:20.470140  5567 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.426162  5362 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:20.561625  1543 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 10
W20260709 13:01:20.979379  1537 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.006s	sys 0.023s
I20260709 13:01:21.172343  5583 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:01:21.217571  1537 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.011s	sys 0.031s
I20260709 13:01:21.217684  5583 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:21.218444  5461 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:20.646417  5575 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:20.855305  5451 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.219792  5575 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.220155  5451 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.221834  2233 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:20.531509  5338 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.224807  5338 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:20.536373  1540 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
W20260709 13:01:21.008102  1542 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2634) took 3417 ms (client timeout 1798 ms). Trace:
I20260709 13:01:20.572155  1618 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
W20260709 13:01:21.226401  1542 rpcz_store.cc:269] 0709 13:01:17.591106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.591228 (+   122us) service_pool.cc:224] Handling call
0709 13:01:21.007755 (+3416527us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b54003444d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:21.007990 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1153230720}
I20260709 13:01:21.227710  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:21.212776  1691 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:01:20.946439  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 63836.5us
I20260709 13:01:21.230306  2233 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:20.744797  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:20.553269  1692 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.231353  1684 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:21.232359  1616 raft_consensus.cc:2368] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10.
W20260709 13:01:21.016166  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:20.612221  2237 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:01:20.582499  2234 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
W20260709 13:01:20.684522  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:20.535405  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 224247us
I20260709 13:01:21.219112  5588 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:20.541818  1582 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:21.233839  2231 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:20.551198  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.233340  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:21.262982  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 721234us
I20260709 13:01:21.255609  1657 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:21.265997  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:21.267022  1657 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:21.256448  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.234368  2237 pending_rounds.cc:85] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 5
I20260709 13:01:21.268858  2237 pending_rounds.cc:107] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 13:01:21.269148  2237 raft_consensus.cc:2889] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:21.269903  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1154) took 1198 ms. Trace:
I20260709 13:01:21.270231  2237 rpcz_store.cc:276] 0709 13:01:20.071637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.611429 (+539792us) service_pool.cc:224] Handling call
0709 13:01:20.611800 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.269665 (+657865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.269847 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.265362  1616 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:20.549664  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 718) took 2879 ms. Trace:
W20260709 13:01:21.273921  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.274129  1538 rpcz_store.cc:276] 0709 13:01:17.670617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.741011 (+ 70394us) service_pool.cc:224] Handling call
0709 13:01:20.349502 (+2608491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.381569 (+ 32067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.549378 (+167809us) spinlock_profiling.cc:243] Waited 2.56 s on lock 0x7b54000e97d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:20.549586 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5881447808}
I20260709 13:01:21.257938  1686 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:20.542201  1609 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
W20260709 13:01:21.145444  5591 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.653s	user 0.003s	sys 0.000s
I20260709 13:01:20.543246  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.260241  1618 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:21.285487  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.286445  1618 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:21.287631  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.289172  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.290043  1618 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:21.291226  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:20.745566  5333 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:20.615491  5436 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:21.254801  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 759) took 1090 ms. Trace:
I20260709 13:01:21.232367  5533 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:21.174959  5178 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.754s	user 0.007s	sys 0.003s
W20260709 13:01:21.225647  5442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.094s	user 0.042s	sys 0.044s
I20260709 13:01:20.508131  5422 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.276177  5531 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:20.733115  2236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 752) took 2573 ms (client timeout 1853 ms). Trace:
I20260709 13:01:21.025882  5492 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:20.850356  5173 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:21.225634  5104 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.255817  2235 rpcz_store.cc:276] 0709 13:01:17.202668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.202855 (+   187us) service_pool.cc:224] Handling call
0709 13:01:17.203199 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:17.476750 (+273551us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:17.544472 (+ 67722us) log.cc:844] Serialized 14 byte log entry
0709 13:01:17.795050 (+250578us) log.cc:844] Serialized 14 byte log entry
0709 13:01:17.795230 (+   180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:17.795287 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:17.795360 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.496340 (+2700980us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b180000b920. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9cd2a6 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419
0709 13:01:20.498577 (+  2237us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54005eddd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:20.498642 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.498830 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44577,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":2801,"raft.queue_time_us":100315,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":150,"spinlock_wait_cycles":633833472,"thread_start_us":278946,"threads_started":2,"wal-append.queue_time_us":396012,"wal-append.run_cpu_time_us":1972,"wal-append.run_wall_time_us":1066807}
I20260709 13:01:20.721870  5572 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:20.680795  5481 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:20.592021  5559 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:21.261583  2232 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:21.255272  5387 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.065083  5470 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:20.562706  2230 rpcz_store.cc:276] 0709 13:01:18.225848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.324121 (+ 98273us) service_pool.cc:224] Handling call
0709 13:01:18.324464 (+   343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:18.324827 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:18.324858 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:20.299804 (+1974946us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b2c000cc068. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:01:20.300604 (+   800us) log.cc:844] Serialized 52 byte log entry
0709 13:01:20.504416 (+203812us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:20.504451 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.504661 (+   210us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.532439 (+ 27778us) raft_consensus.cc:1685] finished
0709 13:01:20.532457 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.532703 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17510,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":56881024,"thread_start_us":203360,"threads_started":1,"wal-append.queue_time_us":229907}
I20260709 13:01:21.299101  2234 rpcz_store.cc:276] 0709 13:01:20.164387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.253987 (+1089600us) service_pool.cc:224] Handling call
0709 13:01:21.254301 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.254489 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.254512 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:21.254554 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:21.254595 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.254647 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.254744 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:20.518734  5266 consensus_queue.cc:1059] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.773s
I20260709 13:01:21.276459  5156 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:21.276522  5338 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:20.680235  5465 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.623s
I20260709 13:01:21.218798  5583 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:21.319252  2236 rpcz_store.cc:269] 0709 13:01:18.152225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.153346 (+  1121us) service_pool.cc:224] Handling call
0709 13:01:20.724842 (+2571496us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54006586d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:20.725015 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1245800832}
I20260709 13:01:21.272993  5432 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.323107  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 763) took 1157 ms. Trace:
I20260709 13:01:21.298663  5333 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:21.281738  5548 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:01:21.282210  5461 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.261667  5395 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.319998  5531 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.263811  5542 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:21.307590  5577 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.817s	user 0.006s	sys 0.035s
I20260709 13:01:21.330065  5338 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:21.341305  5577 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.007s	sys 0.036s
I20260709 13:01:21.327541  5572 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.341463  5583 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:21.340898  5531 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:21.341670  5436 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:21.346695  5461 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:21.356299  5567 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:21.361260  5564 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:21.363792  5559 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.365583  5468 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:21.369652  5572 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:21.407575  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:21.410303  1508 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.008s	sys 0.012s
W20260709 13:01:21.411487  1508 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.009s	sys 0.013s
W20260709 13:01:21.412398  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 877638us
I20260709 13:01:21.258956  1535 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:21.412992  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:21.415407  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 149399us
I20260709 13:01:21.416622  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:20.561904  1536 rpcz_store.cc:276] 0709 13:01:18.860525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:18.860707 (+   182us) service_pool.cc:224] Handling call
0709 13:01:18.883704 (+ 22997us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:18.899757 (+ 16053us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:18.899794 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:20.012174 (+1112380us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b2c00056c38. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:01:20.012768 (+   594us) log.cc:844] Serialized 52 byte log entry
0709 13:01:20.058542 (+ 45774us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:20.066176 (+  7634us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.066451 (+   275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:20.066551 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:20.534794 (+468243us) raft_consensus.cc:1685] finished
0709 13:01:20.534932 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:20.535146 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15522,"prepare.queue_time_us":7100,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":23788,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":7150,"spinlock_wait_cycles":3387136,"thread_start_us":45370,"threads_started":1,"wal-append.queue_time_us":208840}
I20260709 13:01:21.419281  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.420930  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.422094  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.423614  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.424669  1538 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
W20260709 13:01:21.304453  1614 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.027s	sys 0.049s
I20260709 13:01:21.300932  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:21.425663  1614 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.027s	sys 0.050s
I20260709 13:01:21.428346  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.430361  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2113) took 1041 ms. Trace:
I20260709 13:01:21.431049  1610 rpcz_store.cc:276] 0709 13:01:20.388955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.559454 (+170499us) service_pool.cc:224] Handling call
0709 13:01:20.559803 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:20.560457 (+   654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.560494 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:20.560982 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 13:01:21.061121 (+500139us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:21.061166 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.061276 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.429703 (+368427us) raft_consensus.cc:1685] finished
0709 13:01:21.429764 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.430112 (+   348us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":117,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":499655,"threads_started":1,"wal-append.queue_time_us":784432}
I20260709 13:01:21.297521  1611 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:21.273376  1613 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:21.292994  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.281736  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.296422  1618 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:21.436192  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:21.437884  1687 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:21.348018  5451 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:21.332156  2235 rpcz_store.cc:276] 0709 13:01:20.165618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.321970 (+1156352us) service_pool.cc:224] Handling call
0709 13:01:21.322388 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.322655 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.322695 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:21.322754 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:21.322811 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.322883 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.323031 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.461130  5432 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:21.462440  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:21.462996  1609 raft_consensus.cc:2410] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:21.463640  1616 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:21.463075  5333 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.445304  5310 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:21.469153  2235 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:21.469543  5362 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:21.464740  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2114) took 1045 ms. Trace:
I20260709 13:01:21.470283  5542 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:21.447686  5422 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:21.327071  5104 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:21.347591  5338 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:21.351418  5395 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:21.453637  2232 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.454568  5483 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:21.455435  5266 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:21.451493  2230 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:21.329866  5156 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:21.470757  1613 rpcz_store.cc:276] 0709 13:01:20.389195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.550956 (+161761us) service_pool.cc:224] Handling call
0709 13:01:21.434836 (+883880us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.464648  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2091) took 1117 ms. Trace:
I20260709 13:01:21.464877  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.465039  5333 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:21.417632  1542 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:21.470096  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1157) took 1073 ms. Trace:
I20260709 13:01:21.464671  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.485602  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 765) took 1314 ms. Trace:
I20260709 13:01:21.480583  2235 rpcz_store.cc:276] 0709 13:01:20.396591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.460273 (+1063682us) service_pool.cc:224] Handling call
0709 13:01:21.460976 (+   703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.469742 (+  8766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.470010 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:21.483731  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.480943  5266 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.495898  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:21.496398  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 225351us
I20260709 13:01:21.497939  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:21.498872  1538 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:21.501575  5266 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:21.505818  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 237391us
I20260709 13:01:21.507496  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.507676  1657 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:21.489804  2230 rpcz_store.cc:276] 0709 13:01:20.166177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.327141 (+1160964us) service_pool.cc:224] Handling call
0709 13:01:21.450814 (+123673us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.479647 (+ 28833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.481038 (+  1391us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.511909  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.510524  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1635) took 2626 ms. Trace:
I20260709 13:01:21.514096  1689 rpcz_store.cc:276] 0709 13:01:18.884048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:19.980245 (+1096197us) service_pool.cc:224] Handling call
0709 13:01:20.246940 (+266695us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.378257 (+131317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.510017 (+1131760us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b540014a350. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:21.510262 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612286208}
I20260709 13:01:21.521164  5156 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:21.523633  2236 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: [8.9-9.10]
I20260709 13:01:21.524652  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 963) took 1239 ms. Trace:
I20260709 13:01:21.525240  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.525049  2236 rpcz_store.cc:276] 0709 13:01:20.285239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.480969 (+1195730us) service_pool.cc:224] Handling call
0709 13:01:21.522844 (+ 41875us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:21.524285 (+  1441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.524572 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.539806  1609 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:21.504179  5104 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:21.549533  1441 catalog_manager.cc:5697] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 2 to 7, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:01:21.552110  1441 catalog_manager.cc:5697] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:21.553732  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.553949  1610 raft_consensus.cc:2368] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10.
I20260709 13:01:21.477710  1534 rpcz_store.cc:276] 0709 13:01:20.347163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.347291 (+   128us) service_pool.cc:224] Handling call
0709 13:01:20.380491 (+ 33200us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:20.423707 (+ 43216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.423741 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:20.426261 (+  2520us) log.cc:844] Serialized 52 byte log entry
0709 13:01:20.744200 (+317939us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:20.744250 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.463753 (+719503us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b54001d7ed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:21.463854 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.463942 (+    88us) raft_consensus.cc:1685] finished
0709 13:01:21.463956 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.464352 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":151,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":18057,"spinlock_wait_cycles":43929856,"thread_start_us":295998,"threads_started":1,"wal-append.queue_time_us":875484}
I20260709 13:01:21.558568  1616 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:21.559935  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1642) took 1317 ms. Trace:
I20260709 13:01:21.560724  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:21.561578  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 257839us
I20260709 13:01:21.563524  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.564649  1609 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:21.560389  1614 rpcz_store.cc:276] 0709 13:01:20.242169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.260646 (+ 18477us) service_pool.cc:224] Handling call
0709 13:01:20.553809 (+293163us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:01:20.554024 (+   215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.554248 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:21.426511 (+872263us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.559210 (+132699us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.559403 (+   193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:21.559462 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:21.559547 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.559643 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.559835 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30043,"prepare.queue_time_us":807008,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":85,"raft.queue_time_us":31670,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":231,"spinlock_wait_cycles":10441472,"thread_start_us":938934,"threads_started":2}
I20260709 13:01:21.568140  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:21.561729  1534 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:01:21.561658  1535 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:21.569147  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.578341  5604 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:01:21.585755  5483 consensus_queue.cc:1059] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:21.588356  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.589505  1618 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:21.589707  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:21.590695  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.591689  1618 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:21.592293  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2123) took 1119 ms. Trace:
I20260709 13:01:21.592866  1609 rpcz_store.cc:276] 0709 13:01:20.472918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.590895 (+1117977us) service_pool.cc:224] Handling call
0709 13:01:21.591556 (+   661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.591915 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.591942 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:21.592004 (+    62us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:21.592042 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.592095 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.592204 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.594317  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.611508  5483 raft_consensus.cc:3055] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:21.612180  5483 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:21.593276  1618 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:01:21.622073  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2097) took 1182 ms. Trace:
I20260709 13:01:21.623565  1536 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:21.624213  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2104) took 1103 ms. Trace:
I20260709 13:01:21.624594  1536 rpcz_store.cc:276] 0709 13:01:20.520462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.556815 (+1036353us) service_pool.cc:224] Handling call
0709 13:01:21.624105 (+ 67290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.624958  5104 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:21.625566  5104 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:21.626953  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2125) took 1063 ms. Trace:
I20260709 13:01:21.626803  5104 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.627365  1609 rpcz_store.cc:276] 0709 13:01:20.563646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.621105 (+1057459us) service_pool.cc:224] Handling call
0709 13:01:21.626274 (+  5169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.626613 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.626638 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:21.626678 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:21.626707 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.626762 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.626883 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.622658  1535 rpcz_store.cc:276] 0709 13:01:20.439277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.446457 (+  7180us) service_pool.cc:224] Handling call
0709 13:01:21.621852 (+1175395us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.630820  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:21.631853  1535 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:21.632274  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2105) took 1106 ms. Trace:
I20260709 13:01:21.632591  1535 rpcz_store.cc:276] 0709 13:01:20.526375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.630455 (+1104080us) service_pool.cc:224] Handling call
0709 13:01:21.632215 (+  1760us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.633919  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:21.635335  5604 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:21.635959  5604 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:21.636117  1614 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:21.636415  5604 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.639417  1614 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:21.640245  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 761) took 1475 ms. Trace:
I20260709 13:01:21.640882  2231 rpcz_store.cc:276] 0709 13:01:20.165044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.263703 (+1098659us) service_pool.cc:224] Handling call
0709 13:01:21.294068 (+ 30365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.294406 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.294431 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:21.294873 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 13:01:21.586835 (+291962us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:21.586977 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.587089 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.639820 (+ 52731us) raft_consensus.cc:1685] finished
0709 13:01:21.639847 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.639998 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3751,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":73,"thread_start_us":273711,"threads_started":1,"wal-append.queue_time_us":342462}
W20260709 13:01:21.619824  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 112333us
I20260709 13:01:21.643864  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2124) took 1105 ms. Trace:
I20260709 13:01:21.636904  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:21.622911  1441 catalog_manager.cc:5697] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 6 to 7, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:21.663205  2228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1155) took 1531 ms. Trace:
I20260709 13:01:21.660576  1618 rpcz_store.cc:276] 0709 13:01:20.510651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.592255 (+1081604us) service_pool.cc:224] Handling call
0709 13:01:21.592732 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.615333 (+ 22601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.615603 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.614157  5483 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:21.672375  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.635303  5559 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.676175  2232 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 13:01:21.676890  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 965) took 1341 ms. Trace:
I20260709 13:01:21.663756  2228 rpcz_store.cc:276] 0709 13:01:20.131577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.213298 (+1081721us) service_pool.cc:224] Handling call
0709 13:01:21.273139 (+ 59841us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.276652 (+  3513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.276687 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:21.280478 (+  3791us) log.cc:844] Serialized 26 byte log entry
0709 13:01:21.510016 (+229538us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:21.556600 (+ 46584us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.619809 (+ 63209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.662720 (+ 42911us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.662763 (+    43us) raft_consensus.cc:1685] finished
0709 13:01:21.662780 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.662996 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3133,"prepare.queue_time_us":67958,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":94,"raft.queue_time_us":19606,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":214,"thread_start_us":240075,"threads_started":2,"wal-append.queue_time_us":322501}
I20260709 13:01:21.679191  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.679534  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:21.680105  1535 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:21.680460  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2100) took 1217 ms. Trace:
I20260709 13:01:21.681864  5470 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:21.680832  1536 rpcz_store.cc:276] 0709 13:01:20.463468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.666646 (+1203178us) service_pool.cc:224] Handling call
0709 13:01:21.680399 (+ 13753us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.681928  5457 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:21.681102  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.677513  2232 rpcz_store.cc:276] 0709 13:01:20.335188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.452180 (+1116992us) service_pool.cc:224] Handling call
0709 13:01:21.452989 (+   809us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.676629 (+223640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.676835 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.689157  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.690104  1536 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:21.690979  5505 raft_consensus.cc:3055] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:21.691265  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.691525  5505 raft_consensus.cc:740] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:21.682667  5470 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:21.697686  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:21.693243  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.699536  1657 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:21.704623  5607 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:21.705488  5607 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:21.706909  5533 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:21.706868  5607 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.709954  5559 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.707486  5533 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:21.713706  5533 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.714423  5338 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.716149  5533 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:21.716205  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.705286  5575 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:21.719655  1540 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2652) took 1530 ms (client timeout 1673 ms). Trace:
I20260709 13:01:21.693302  5505 consensus_queue.cc:260] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:21.778668  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:21.772876  1540 rpcz_store.cc:269] 0709 13:01:20.188947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.189220 (+   273us) service_pool.cc:224] Handling call
0709 13:01:21.719155 (+1529935us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b540030c0d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:21.719415 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":795553024}
I20260709 13:01:21.780035  1689 raft_consensus.cc:2410] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:21.698990  5470 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:21.726030  5614 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:01:21.783004  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 962) took 1497 ms. Trace:
W20260709 13:01:21.727133  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.728928  5431 consensus_queue.cc:1059] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:21.783367  2234 rpcz_store.cc:276] 0709 13:01:20.284980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.331143 (+1046163us) service_pool.cc:224] Handling call
0709 13:01:21.331609 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:21.465726 (+134117us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.465820 (+    94us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:21.466883 (+  1063us) log.cc:844] Serialized 26 byte log entry
0709 13:01:21.544824 (+ 77941us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:21.544872 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.556293 (+ 11421us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.782612 (+226319us) raft_consensus.cc:1685] finished
0709 13:01:21.782633 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.782841 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125694,"raft.queue_time_us":133627,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":303,"spinlock_wait_cycles":1973120,"thread_start_us":77370,"threads_started":1,"wal-append.queue_time_us":190514}
I20260709 13:01:21.784950  5431 raft_consensus.cc:3055] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:21.730365  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:21.732621  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2524) took 1574 ms. Trace:
I20260709 13:01:21.785391  5431 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:21.791361  5431 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.792738  4916 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:21.734925  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 764) took 1568 ms. Trace:
W20260709 13:01:21.742059  1509 connection.cc:256] RPC call timeout handler was delayed by -1.1972220600000014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29550000000000004s for a response
I20260709 13:01:21.778791  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:21.810487  2233 rpcz_store.cc:276] 0709 13:01:20.165924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.293545 (+127621us) service_pool.cc:224] Handling call
0709 13:01:21.734488 (+1440943us) spinlock_profiling.cc:243] Waited 845 ms on lock 0x7b54004bb254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:21.734671 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1943077760}
I20260709 13:01:21.810052  1540 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:21.812053  5178 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.812317  5422 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:21.812598  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2103) took 1299 ms. Trace:
W20260709 13:01:21.803243  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 103904us
I20260709 13:01:21.814131  1613 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:21.789116  5505 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:21.803153  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:21.813118  5422 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:21.743793  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2116) took 1351 ms. Trace:
I20260709 13:01:21.752349  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 650) took 1567 ms. Trace:
I20260709 13:01:21.753809  5483 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:21.817608  1611 rpcz_store.cc:276] 0709 13:01:20.391959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.569437 (+177478us) service_pool.cc:224] Handling call
0709 13:01:20.691267 (+121830us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.433838 (+742571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.743364 (+309526us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b54002bdc54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:21.743624 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393376000}
W20260709 13:01:21.756611  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.754585  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:21.758831  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:21.760084  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:21.819499  1610 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:21.763103  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 741) took 4503 ms. Trace:
I20260709 13:01:21.764003  1618 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:21.820549  1618 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:21.766669  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.776284  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2102) took 1300 ms. Trace:
I20260709 13:01:21.724288  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2094) took 1331 ms. Trace:
I20260709 13:01:21.781247  5578 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:21.817690  1615 rpcz_store.cc:276] 0709 13:01:20.184895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.185014 (+   119us) service_pool.cc:224] Handling call
0709 13:01:20.185315 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.562075 (+376760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.751677 (+1189602us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54000889d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:21.752098 (+   421us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864537472}
I20260709 13:01:21.759562  5387 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.819937  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.847630  1616 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 7: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:21.787171  1683 rpcz_store.cc:276] 0709 13:01:20.158117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.399000 (+240883us) service_pool.cc:224] Handling call
0709 13:01:20.399379 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.424876 (+ 25497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.731871 (+1306995us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b54000f3d20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa63dc0 00007fe08faefa02 00007fe08faebdd6 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:01:21.732387 (+   516us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32077184}
W20260709 13:01:21.808161  1543 consensus_meta.cc:287] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.590s	user 0.016s	sys 0.007s
I20260709 13:01:21.820427  1687 rpcz_store.cc:276] 0709 13:01:17.259785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:17.534136 (+274351us) service_pool.cc:224] Handling call
0709 13:01:17.534480 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.438606 (+3904126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.762760 (+324154us) spinlock_profiling.cc:243] Waited 3.9 s on lock 0x7b54000749d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:21.763010 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8975096576}
W20260709 13:01:21.821787  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.849592  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.849967  1690 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:21.849123  1539 rpcz_store.cc:276] 0709 13:01:20.476173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.715646 (+1239473us) service_pool.cc:224] Handling call
0709 13:01:21.776205 (+ 60559us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.850291  1613 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:21.850814  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.860047  1617 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:21.792824  5431 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:21.851207  1612 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 7.
I20260709 13:01:21.803158  5442 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:21.871591  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:21.876603  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 103332us
I20260709 13:01:21.850423  5422 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:21.883276  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:21.882965  1612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1638) took 1634 ms (client timeout 1604 ms). Trace:
I20260709 13:01:21.813086  1540 rpcz_store.cc:276] 0709 13:01:20.513474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.780245 (+1266771us) service_pool.cc:224] Handling call
0709 13:01:21.780727 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.812208 (+ 31481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.812517 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:21.884187  1612 rpcz_store.cc:269] 0709 13:01:20.240357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.240591 (+   234us) service_pool.cc:224] Handling call
0709 13:01:21.729943 (+1489352us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:01:21.874802 (+144859us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4975872}
I20260709 13:01:21.887054  1538 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:21.883618  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.813028  5178 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.889112  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.890203  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.896608  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:21.894548  1615 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:21.898536  1542 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:21.873680  1543 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:21.874060  5442 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:21.847326  1541 rpcz_store.cc:276] 0709 13:01:20.392296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.392450 (+   154us) service_pool.cc:224] Handling call
0709 13:01:20.392925 (+   475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:20.393128 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:20.419510 (+ 26382us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.456140 (+ 36630us) log.cc:844] Serialized 14 byte log entry
0709 13:01:20.551284 (+ 95144us) spinlock_profiling.cc:243] Waited 8.05 ms on lock 0x7b18004a8d20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9cd2a6 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f
0709 13:01:20.552917 (+  1633us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b480030c858. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:01:21.689866 (+1136949us) spinlock_profiling.cc:243] Waited 4.09 ms on lock 0x7b18004a8d20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:01:21.694970 (+  5104us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b18004a8d20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9cd2a6 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1
0709 13:01:21.696330 (+  1360us) spinlock_profiling.cc:243] Waited 8.32 ms on lock 0x7b18004a8d20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:01:21.722338 (+ 26008us) spinlock_profiling.cc:243] Waited 8.33 ms on lock 0x7b18004a8d20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c979c0a 00007fe0960b9219 00007fe08ca01a40 00007fe08c9fc4e4 00007fe08c9fc455 00007fe0934e43c7 00007fe08faf69a7 00007fe08fb060bf 00007fe08fb06038
0709 13:01:21.722769 (+   431us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.723113 (+   344us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:21.723222 (+   109us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:21.723401 (+   179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.723744 (+   343us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.724060 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1247095,"prepare.queue_time_us":1313684,"prepare.run_cpu_time_us":166,"prepare.run_wall_time_us":5314,"raft.queue_time_us":9131,"raft.run_cpu_time_us":4100,"raft.run_wall_time_us":102907,"spinlock_wait_cycles":147870208,"thread_start_us":20096,"threads_started":1,"wal-append.queue_time_us":47883}
I20260709 13:01:21.901104  1657 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:21.908026  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.906453  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.914139  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2096) took 1489 ms. Trace:
I20260709 13:01:21.911486  1582 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:21.892881  5559 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:21.921690  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:21.917018  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:21.914737  1543 rpcz_store.cc:276] 0709 13:01:20.424717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.424890 (+   173us) service_pool.cc:224] Handling call
0709 13:01:21.906224 (+1481334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.923558  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.924499  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:21.924983  1543 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:21.926280  1539 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:01:21.929095  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 54519us
I20260709 13:01:21.906908  5442 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:21.933023  1542 raft_consensus.cc:2376] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10. Re-sending same reply.
W20260709 13:01:21.939203  5583 consensus_meta.cc:287] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.584s	user 0.007s	sys 0.005s
I20260709 13:01:21.943346  5178 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:21.944216  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:21.945544  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 88494.2us
I20260709 13:01:21.945943  1690 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: [10.10-12.11]
W20260709 13:01:21.950851  1692 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.719s	user 0.005s	sys 0.017s
I20260709 13:01:21.952615  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:21.953917  1687 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:21.955013  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:21.955824  1540 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:21.956207  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:21.957558  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2092) took 1609 ms. Trace:
I20260709 13:01:21.958302  5340 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:21.959213  5156 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:21.965603  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 54260.6us
I20260709 13:01:21.904129  1618 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.917830  5338 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:21.976686  1617 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:21.897466  5604 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:21.971333  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2106) took 1318 ms. Trace:
W20260709 13:01:21.971302  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 75305.1us
I20260709 13:01:21.937201  5387 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:21.886399  5578 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:21.981467  5104 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:21.989928  1539 rpcz_store.cc:276] 0709 13:01:20.608379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.858253 (+1249874us) service_pool.cc:224] Handling call
0709 13:01:21.912825 (+ 54572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.926811 (+ 13986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.927053 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:21.996428  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2099) took 1537 ms. Trace:
I20260709 13:01:21.997195  1535 rpcz_store.cc:276] 0709 13:01:20.454550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.718242 (+1263692us) service_pool.cc:224] Handling call
0709 13:01:21.719145 (+   903us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.719874 (+   729us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:21.726833 (+  6959us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.991208 (+264375us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:21.991272 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:21.991324 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.991410 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.991540 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6310,"prepare.queue_time_us":7450,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":3408,"thread_start_us":259906,"threads_started":1}
I20260709 13:01:21.992455  1692 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
W20260709 13:01:22.000110  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 180871us
I20260709 13:01:21.999020  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 762) took 1691 ms. Trace:
I20260709 13:01:21.975690  1537 rpcz_store.cc:276] 0709 13:01:20.347527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.347664 (+   137us) service_pool.cc:224] Handling call
0709 13:01:20.348152 (+   488us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:20.438283 (+ 90131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:20.438314 (+    31us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:20.440231 (+  1917us) log.cc:844] Serialized 78 byte log entry
0709 13:01:21.218264 (+778033us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:21.906796 (+688532us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.956633 (+ 49837us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.956913 (+   280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.957026 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:21.957100 (+    74us) raft_consensus.cc:1685] finished
0709 13:01:21.957117 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.957333 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":505955,"prepare.queue_time_us":510815,"prepare.run_cpu_time_us":139,"prepare.run_wall_time_us":145,"raft.queue_time_us":31462,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":10472,"thread_start_us":919903,"threads_started":2,"wal-append.queue_time_us":864700}
I20260709 13:01:21.993696  5104 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.009938  5104 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:22.005314  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 98081.9us
I20260709 13:01:22.015287  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:22.018465  1686 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:22.019143  1540 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:01:22.022009  1509 connection.cc:256] RPC call timeout handler was delayed by -1.0275067439999361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27130000000000004s for a response
I20260709 13:01:22.031606  2233 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: [7.8-10.10]
I20260709 13:01:22.033155  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:22.034855  5604 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:22.035223  1538 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:22.036533  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 62582us
I20260709 13:01:22.037031  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:22.037830  2229 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:22.042613  5395 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:01:22.036510  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:22.034981  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:22.047753  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45548.9us
I20260709 13:01:22.051484  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1649) took 1524 ms. Trace:
I20260709 13:01:22.054591  1618 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.036510  2228 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:22.052047  1692 rpcz_store.cc:276] 0709 13:01:20.494550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:20.551822 (+ 57272us) service_pool.cc:224] Handling call
0709 13:01:22.013700 (+1461878us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:22.055636  1689 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:22.002514  2237 rpcz_store.cc:276] 0709 13:01:20.165302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.271053 (+1105751us) service_pool.cc:224] Handling call
0709 13:01:21.271328 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:21.271525 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:21.524514 (+252989us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.676093 (+151579us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:21.685758 (+  9665us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:21.817516 (+131758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.820209 (+  2693us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:21.856946 (+ 36737us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":306292,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":110,"raft.queue_time_us":3274,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":3188,"thread_start_us":388829,"threads_started":2,"wal-append.queue_time_us":249292}
I20260709 13:01:22.064841  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:22.066078  1540 raft_consensus.cc:2410] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:22.067402  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:22.061811  1541 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.041225  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:22.074966  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:22.073239  2232 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:22.076725  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:22.061528  5333 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:22.086885  1538 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:22.094579  2233 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:22.008203  5340 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:22.096582  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 69247.9us
I20260709 13:01:22.092445  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:22.109823  2229 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:22.098573  1537 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:22.098354  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:22.112288  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:22.113654  2237 raft_consensus.cc:2376] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11. Re-sending same reply.
I20260709 13:01:22.090216  5436 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:22.076871  5604 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:01:22.085245  5470 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:22.086613  5559 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:22.131716  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 90609.8us
I20260709 13:01:22.141498  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:22.135813  2203 leader_election.cc:336] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.069s (SENT)
I20260709 13:01:22.148787  5461 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:22.149724  5461 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:22.155014  5531 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.758s	user 0.008s	sys 0.010s
I20260709 13:01:22.155658  5531 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.157653  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:22.158960  1692 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:22.159435  5559 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:22.161580  5156 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:22.161885  5559 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:22.167294  5572 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.167996  5572 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.173975  1540 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:22.175871  1540 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:22.176841  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:22.177511  2237 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:22.178675  2203 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:22.158761  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:22.149024  5620 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:01:22.178715  1541 raft_consensus.cc:2410] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:22.181219  2234 raft_consensus.cc:2410] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:01:22.110041  5333 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.133845  2232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:01:22.188228  2232 raft_consensus.cc:606] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:22.188864  2232 raft_consensus.cc:614] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:22.188097  2230 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:22.147298  5436 raft_consensus.cc:697] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:22.196460  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:22.201996  1536 raft_consensus.cc:2410] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:01:22.206957  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:22.210129  5457 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:22.213559  1583 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:22.214629  5619 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.356s	user 0.000s	sys 0.004s
I20260709 13:01:22.214957  5424 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:22.216193  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:22.217031  1689 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:01:22.217926  1689 raft_consensus.cc:606] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:22.218362  1689 raft_consensus.cc:614] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:22.218667  5492 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:22.219120  5492 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:22.219535  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2126) took 1577 ms. Trace:
I20260709 13:01:22.220350  5492 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.220983  5578 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:22.221199  5436 consensus_queue.cc:237] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:22.206765  5440 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:22.227427  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:22.228689  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.229102  5583 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:22.136250  5607 consensus_queue.cc:1059] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:01:22.230583  5156 raft_consensus.cc:3055] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:22.231398  5156 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:22.232580  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:22.232933  5156 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.219951  1609 rpcz_store.cc:276] 0709 13:01:20.642402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.628388 (+985986us) service_pool.cc:224] Handling call
0709 13:01:21.628913 (+   525us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:01:21.668352 (+ 39439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:21.668397 (+    45us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:01:21.678446 (+ 10049us) log.cc:844] Serialized 104 byte log entry
0709 13:01:21.884295 (+205849us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:21.911341 (+ 27046us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.919291 (+  7950us) log.cc:844] Serialized 14 byte log entry
0709 13:01:21.920763 (+  1472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:21.920839 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.213850 (+293011us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:22.219293 (+  5443us) raft_consensus.cc:1685] finished
0709 13:01:22.219313 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.219465 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70129,"prepare.queue_time_us":70472,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":34370,"raft.queue_time_us":48313,"raft.run_cpu_time_us":533,"raft.run_wall_time_us":46333,"spinlock_wait_cycles":62335232,"thread_start_us":205304,"threads_started":1,"wal-append.queue_time_us":179212}
I20260709 13:01:22.234493  5156 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.234706  4916 consensus_queue.cc:1059] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:22.234906  1687 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:01:22.235764  4916 consensus_queue.cc:1243] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:01:22.231649  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55053.4us
I20260709 13:01:22.235937  1687 raft_consensus.cc:606] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:22.236306  5583 raft_consensus.cc:3055] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:22.236450  1687 raft_consensus.cc:614] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:22.236609  2201 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.236795  5583 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:22.236922  5333 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:22.237795  5333 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:22.238410  5333 raft_consensus.cc:697] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:22.237929  5583 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.239364  5583 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.239596  5333 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:22.241931  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 83268.2us
I20260709 13:01:22.241096  2201 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.243194  5338 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:22.243919  1584 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:22.245318  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:22.245604  2230 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:22.246582  5457 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:22.246739  1687 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c8377b7234e4472ea27005f16dede669"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:01:22.247440  1687 raft_consensus.cc:606] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:22.247826  2230 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
I20260709 13:01:22.247936  1687 raft_consensus.cc:614] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:22.248251  4916 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:22.248999  4916 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:22.249600   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:01:22.250087  5395 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:22.250288  4916 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:22.258461  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.268071  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:22.275417  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.275591  1618 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:01:22.276108  1618 pending_rounds.cc:85] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 9
W20260709 13:01:22.278192  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.278899  1692 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
W20260709 13:01:22.280742  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 97990.1us
I20260709 13:01:22.277382  1689 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:22.284099  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:22.291437  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:22.286311  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:22.288916  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:22.290818  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:22.282749  1618 pending_rounds.cc:107] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:01:22.298550  1618 raft_consensus.cc:2889] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:22.301895  1610 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:22.302520  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2143) took 1027 ms. Trace:
I20260709 13:01:22.302851  1610 rpcz_store.cc:276] 0709 13:01:21.274686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.754083 (+479397us) service_pool.cc:224] Handling call
0709 13:01:22.302457 (+548374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:22.290208  5461 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:22.307746  5424 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:22.309520  5604 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:22.316763  5629 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:01:22.319006  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:22.320883  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:22.309017  5104 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.322721  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:22.323171  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:22.323553  5104 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:22.325165  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.323566  2237 raft_consensus.cc:2410] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:01:22.325626   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:01:22.327188  5631 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 13:01:22.334338  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 207290us
I20260709 13:01:22.313233  5572 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:22.336921  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 728) took 1112 ms. Trace:
I20260709 13:01:22.342774  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:22.344058  2237 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:01:22.342555  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:22.292639  5492 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.340699  2230 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.340490  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:22.346434  2237 pending_rounds.cc:85] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 10
I20260709 13:01:22.346544  2234 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
W20260709 13:01:22.353166  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 47667.2us
I20260709 13:01:22.350950  2237 pending_rounds.cc:107] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:01:22.354241  2237 raft_consensus.cc:2889] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:22.352543  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 91226.9us
I20260709 13:01:22.360688  1690 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:22.342245  1539 rpcz_store.cc:276] 0709 13:01:21.224290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.005803 (+781513us) service_pool.cc:224] Handling call
0709 13:01:22.044912 (+ 39109us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:22.045526 (+   614us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:22.045866 (+   340us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.137141 (+ 91275us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:22.185550 (+ 48409us) log.cc:844] Serialized 52 byte log entry
0709 13:01:22.185967 (+   417us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:22.186286 (+   319us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.186658 (+   372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.186762 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.336748 (+149986us) raft_consensus.cc:1685] finished
0709 13:01:22.336761 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.336875 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":987,"prepare.queue_time_us":11477,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":76,"raft.queue_time_us":4008,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":11079,"spinlock_wait_cycles":4483840,"thread_start_us":90805,"threads_started":1,"wal-append.queue_time_us":181390}
I20260709 13:01:22.365342  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:22.368055  2231 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:22.368757  1689 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: [8.10-8.10]
W20260709 13:01:22.371668  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 89129.8us
W20260709 13:01:22.375653  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.375977  2229 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:22.369565  1683 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 7.
I20260709 13:01:22.344662  5531 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:22.381951  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:22.385463  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.347045  5104 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.387696  5104 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:22.389086  5431 consensus_meta.cc:287] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.514s	user 0.007s	sys 0.005s
I20260709 13:01:22.375295  5578 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:22.400024  5470 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:22.400517  5578 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.402722  5578 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:22.402807  5470 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:01:22.403959  5630 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.182s	user 0.001s	sys 0.002s
I20260709 13:01:22.385764  1508 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:22.386126  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:22.404090  5310 raft_consensus.cc:3055] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:22.413749  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:22.414073  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:22.415436  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.417837  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.418679  1690 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:22.419282  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:22.420563  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 53774.4us
I20260709 13:01:22.406824  5431 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:22.421923  5565 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:22.422353  5565 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.422744  5565 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:22.428270  2201 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.433617  5533 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:22.434115  5533 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:22.434459  5533 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:22.406440  5104 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:22.420265  5483 consensus_meta.cc:287] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.602s	user 0.010s	sys 0.002s
I20260709 13:01:22.444046  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:22.446558  1689 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:22.447353  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:22.426631  5266 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.924s	user 0.010s	sys 0.014s
W20260709 13:01:22.448582  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.447028  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:22.409189  5310 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:22.455610  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.456313  1618 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:22.458491  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:22.459154  2228 raft_consensus.cc:2376] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10. Re-sending same reply.
I20260709 13:01:22.459408  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.460814  1610 raft_consensus.cc:2410] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:22.463490  5340 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:22.471100  2229 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:22.411520  5442 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:22.477785  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 98314us
I20260709 13:01:22.485847  2236 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:01:22.489449  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 59814.6us
I20260709 13:01:22.494146  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.451713  5338 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.495270  1610 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:22.491195  1508 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.466447  5483 raft_consensus.cc:2764] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:01:22.514791  1609 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:22.515213  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.490449  5577 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.451848  5266 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.496963  5578 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.537175  1610 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:22.542258  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.544636  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.547881  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.530923  5340 raft_consensus.cc:2764] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:22.554775  1534 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:22.558305  1583 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:22.562917  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.544469  5310 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:22.565981  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 57260.8us
I20260709 13:01:22.566416  5310 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.567054  1584 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:22.575968  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2107) took 1354 ms. Trace:
I20260709 13:01:22.576471  1534 rpcz_store.cc:276] 0709 13:01:21.221866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.587947 (+366081us) service_pool.cc:224] Handling call
0709 13:01:22.575735 (+987788us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b54000ed8d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:22.575881 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2218394368}
I20260709 13:01:22.579754  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.581900  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2111) took 1359 ms. Trace:
I20260709 13:01:22.582566  1508 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.582293  1543 rpcz_store.cc:276] 0709 13:01:21.222953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:21.925491 (+702538us) service_pool.cc:224] Handling call
0709 13:01:21.953997 (+ 28506us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:21.960801 (+  6804us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:21.961386 (+   585us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.110835 (+149449us) log.cc:844] Serialized 14 byte log entry
0709 13:01:22.110924 (+    89us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:22.131117 (+ 20193us) log.cc:844] Serialized 52 byte log entry
0709 13:01:22.131332 (+   215us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:22.131387 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:22.131488 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:22.361504 (+230016us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:22.581635 (+220131us) raft_consensus.cc:1685] finished
0709 13:01:22.581658 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:22.581814 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16838,"prepare.queue_time_us":869,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":1827,"raft.queue_time_us":19712,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":8746,"spinlock_wait_cycles":12144768,"thread_start_us":137346,"threads_started":1,"wal-append.queue_time_us":176872}
I20260709 13:01:22.558578  5431 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:01:22.585067  5542 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:22.598894  5542 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:22.600907  5542 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.605333  1618 raft_consensus.cc:2410] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:01:22.606173  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:22.607208  5562 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:22.607450  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:22.607782  5562 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.608299  5562 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:22.539181  5578 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.610610  1610 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:22.610966  5578 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:22.612843  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:22.614023  1689 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:22.585012  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.620654  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.621354  2201 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.621702  1609 raft_consensus.cc:2410] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:22.622907  5646 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:22.623442  5646 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.623951  5646 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:22.624653  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.552778  5483 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:22.625759  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:22.626456  1613 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:22.626929  5442 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:22.540333  5338 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.628720  1614 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:22.629733  5470 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:22.629828  5338 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:22.630599  5338 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:22.631042  5338 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.631412  5338 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.633909  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:22.633653  5646 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.635232  5533 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.625520  5483 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:22.636898  5562 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:22.638290  5483 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:22.638835  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:22.639513  1689 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:22.643302  4916 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.645648  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.646718  2203 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:22.647214  5533 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:22.648311  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.648631  1689 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:22.648279  5533 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:22.535308  5577 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:22.639032  5562 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:22.630033  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 63118.7us
I20260709 13:01:22.649678  5607 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:22.655974  5607 raft_consensus.cc:697] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:22.649384  2203 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:22.656312  5577 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:22.650861  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.657016  5607 consensus_queue.cc:237] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:22.655210  4916 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:22.658528  4916 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:22.658943  4916 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:22.649981  2201 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.649597  5578 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:22.656325  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:22.661939  1508 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.663000  1611 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:22.652673  2230 raft_consensus.cc:2410] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:22.659328  4916 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:22.668053  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 109864us
I20260709 13:01:22.672217  1583 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:22.672457  1611 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:22.673547  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:22.674400  1689 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:22.662142  5533 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.669462  5340 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:22.675575  5533 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.675742  5340 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:22.675935  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:22.676327  5340 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.676523  2228 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.678704  1584 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:22.660449  5578 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:22.660938  5431 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:22.664304  5572 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.650807  5266 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:22.685724  5310 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:22.684603  5431 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.680972  5642 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:22.686549  5436 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:22.697691  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.671911  5559 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:01:22.702172  1611 raft_consensus.cc:2410] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:01:22.708464  1689 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:22.709831  2201 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.711776  5310 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:22.712046  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 62778.4us
I20260709 13:01:22.713165  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.713881  1618 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:22.700627  5642 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.719740  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:22.700981  5431 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.702899  5578 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:22.721119  5428 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:01:22.721575  5310 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:22.724270  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45707.3us
W20260709 13:01:22.724529  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.696456  5470 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:01:22.689640  5572 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.731521  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.716619  5436 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:22.720346  5583 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:22.735728  1584 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:22.747615  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.724007  5642 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.748764  1618 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:22.749828  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.752164  5436 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.735260  5578 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:22.765748  5531 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:22.767568  5634 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
W20260709 13:01:22.770154  5643 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:01:22.774614  5578 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.781014  1618 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:01:22.784639  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:22.788249  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:22.789146  1689 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:22.791184  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 53460.4us
I20260709 13:01:22.786461  5565 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:22.793212  5333 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:22.793735  5333 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:22.794135  5333 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:22.794865  5565 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:22.791595  5559 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.755937  5492 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:01:22.756922  5646 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:22.815022  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:22.816030  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56806.4us
I20260709 13:01:22.818081  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:22.805367  5470 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:22.823011  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.829578  5531 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:22.819154  5492 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:22.847162  2203 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:22.848380  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.848580  5424 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.849124  5424 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:22.850090  5492 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:22.851230  5424 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:22.852658  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.826000  5470 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:22.853663  5470 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:22.805928  5559 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:22.866185  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 62126.3us
W20260709 13:01:22.868151  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.838477  5583 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:22.876678  5340 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.877511  5651 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:22.887144  1689 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
W20260709 13:01:22.890316  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:22.890535  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:22.891187  5660 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:01:22.892359  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45319us
I20260709 13:01:22.893571  5660 raft_consensus.cc:697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:22.886857  5338 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.894572  5660 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.898051  5583 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.898557  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:22.906392  5340 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:22.908175  2228 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:22.914054  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.908250  2201 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.918754  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:22.919649  2231 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:22.924371  1508 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.903752  5338 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:22.920748  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:22.938020  1534 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:22.934676  1610 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:22.880622  5533 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.947446  1690 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:01:22.959388  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 50540.8us
I20260709 13:01:22.962958  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:22.963639  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:22.936878  5428 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:22.966995  2232 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:22.889456  5651 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:22.975939  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:22.962212  5661 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:22.977666  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:22.980976  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:22.982091  5656 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:22.982862  5656 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:22.983472  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 60895us
I20260709 13:01:22.970436  5428 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:23.003331  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:23.008585  5578 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:23.012674  1611 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:23.023672  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:23.024932  1611 raft_consensus.cc:2410] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:23.030332  1690 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:01:23.030804  1690 pending_rounds.cc:85] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 7
I20260709 13:01:23.031023  1690 pending_rounds.cc:107] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:01:23.031244  1690 raft_consensus.cc:2889] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:22.995599  5661 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:23.033589  5659 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 13:01:23.035408  5655 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:23.035956  5655 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:23.036311  5655 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:23.036337  5470 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:23.041543  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:23.042408  1683 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:23.042411  5572 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:23.049027  5431 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.049796  5428 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.051710  5661 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.062806  1692 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:23.067663  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.068457  1610 raft_consensus.cc:2368] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8.
I20260709 13:01:22.990132  5646 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:01:23.075608  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:23.082814  1684 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.085011  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:22.991330  5642 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.086621  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:23.088824  2231 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:23.067335  5340 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.092089  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:23.094635  1611 raft_consensus.cc:2376] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12. Re-sending same reply.
I20260709 13:01:23.092235  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:23.092813  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:23.095878  1616 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:23.092605  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:23.073362  5533 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:23.069519  5559 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:23.079380  5572 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:23.095992  1618 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:23.106528  5470 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:23.106907  5642 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:23.114414  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:23.116041  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 132897us
I20260709 13:01:23.139657  5431 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:23.142994  5652 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.381s	user 0.001s	sys 0.001s
I20260709 13:01:23.144255  1686 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:23.145295  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:23.146140  1686 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:23.146332  5654 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.381s	user 0.004s	sys 0.000s
I20260709 13:01:23.146939  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:23.148543  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:23.149667  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:23.151017  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:23.152403  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:23.154053  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:23.154763  1616 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:23.113663  5340 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.154091  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:23.156736  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:23.157264  1686 raft_consensus.cc:2376] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8. Re-sending same reply.
I20260709 13:01:23.158303  1691 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:23.160284  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:23.162076  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:23.162235  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:23.162931  1618 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:23.163261  1687 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:23.164673  5656 raft_consensus.cc:3055] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:23.165115  5656 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:23.165879  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:23.166080  1690 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:01:23.166005  5656 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.167141  5656 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.167488  1684 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:23.167949  1691 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:23.168870  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.169873  5340 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:23.157950  4916 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.172175  4916 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:23.110842  5572 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.175554  5572 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:23.178263  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:23.179919  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:23.181597  5542 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.578s	user 0.012s	sys 0.000s
I20260709 13:01:23.158200  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:23.182021  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:23.169962  5651 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:23.197712  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.185613  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:23.207231  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:23.208595  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:23.185675  1686 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:23.157104  5333 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.211696  1688 raft_consensus.cc:2410] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 14 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3.
I20260709 13:01:23.209661  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:23.219322  1687 raft_consensus.cc:2393] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:23.214829  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:23.215508  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:23.212286  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:23.222014  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:23.222656  1543 raft_consensus.cc:2376] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10. Re-sending same reply.
I20260709 13:01:23.182092  5542 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.225627  5542 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:23.230371  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:23.232580  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:23.233491  1685 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:23.235438  5333 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:23.229235  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:23.213165  5505 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:23.241756  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:23.243525  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:23.244364  1541 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:23.247244  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:23.248385  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:23.249094  1617 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
W20260709 13:01:23.251695  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 62203.2us
W20260709 13:01:23.256454  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2950825789999953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
W20260709 13:01:23.256927  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2804254209999613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2001s for a response
W20260709 13:01:23.257242  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2381254209999497s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2048s for a response
W20260709 13:01:23.257548  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2071671009999818s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31970000000000004s for a response
I20260709 13:01:23.254390  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:23.258494  2203 connection.cc:256] RPC call timeout handler was delayed by -1.35354961500002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 13:01:23.260076  2230 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:23.258910  5457 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:23.261875  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:23.262264  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:23.262620  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:23.262849  1534 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:23.240028  5338 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.271064  2236 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:23.272250  5683 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:01:23.272634  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:23.273885  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:23.274716  1541 raft_consensus.cc:2410] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 14 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:23.275305  5428 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:23.264796  1540 raft_consensus.cc:2376] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9. Re-sending same reply.
I20260709 13:01:23.290612  1582 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:23.292752  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.293226  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:23.307384  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:23.322996  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:23.325819  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:23.327284  1538 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:23.330127  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:23.330722  1535 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:23.333040  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44257.8us
I20260709 13:01:23.333107  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:23.325666  5457 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:01:23.324959  5440 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:23.335471  1614 raft_consensus.cc:2376] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10. Re-sending same reply.
W20260709 13:01:23.370322  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 111241us
I20260709 13:01:23.367781  1508 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:23.305711  5542 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:01:23.305179  5559 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:01:23.298743  5505 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:01:23.392149  1582 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:23.393182  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:23.346928  5457 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:23.336643  5642 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:01:23.407958  1614 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:23.412081  1538 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:23.423400  1617 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 10. (term mismatch)
I20260709 13:01:23.423946  1617 pending_rounds.cc:85] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 9
I20260709 13:01:23.424213  1617 pending_rounds.cc:107] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 8.10
I20260709 13:01:23.424491  1617 raft_consensus.cc:2889] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:23.426069  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:23.328826  5655 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.400897  5492 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:01:23.407886  5686 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:01:23.422552  5470 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:01:23.430668  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:23.435662  5505 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:23.436525  5642 raft_consensus.cc:3055] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:01:23.413857  5436 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:23.442456  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:23.443817  1584 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:01:23.444317  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:23.472059  1614 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.446820  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:23.472776  1618 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:23.452848  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 90212.8us
I20260709 13:01:23.384773  5340 consensus_queue.cc:1059] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:01:23.417599  5457 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:23.425473  5440 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.462776  1535 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:23.463749  1534 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
W20260709 13:01:23.519608  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:23.466434  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 202290us
W20260709 13:01:23.470367  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 102709us
W20260709 13:01:23.527896  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0941254210000011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22080000000000002s for a response
I20260709 13:01:23.465296  5338 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:23.528267  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0930418449999593s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
W20260709 13:01:23.528653  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0778644150000218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17420000000000002s for a response
W20260709 13:01:23.529317  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0081815620000043s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1819s for a response
W20260709 13:01:23.551337  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 76360.5us
W20260709 13:01:23.551608  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 107907us
I20260709 13:01:23.553839  2228 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:23.470135  5642 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:23.558763  1509 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:23.466544  5424 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 13:01:23.477869  5505 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:23.577075  1539 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:23.478091  5436 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:23.571485  2232 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:23.598080  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:23.599047  2228 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:23.601825  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:23.602980  1539 raft_consensus.cc:2410] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 12 because replica has last-logged OpId of term: 11 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
W20260709 13:01:23.606961  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:23.611322  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 59135.2us
I20260709 13:01:23.590481  5559 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.001957  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:23.555502  5338 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.003222  1509 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:24.004560  5338 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:23.554008  5440 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:23.632161  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:23.623220  5642 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.008104  5642 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:24.010459  5440 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:23.652508  5531 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:01:24.011229  5700 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.674021  5470 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:24.011927  5470 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:24.011823  5700 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.677902  5436 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.013197  5470 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.014276  5700 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:23.679086  5655 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:23.666074  5505 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.673821  5696 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:23.698061  5684 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.296s	user 0.004s	sys 0.000s
I20260709 13:01:23.724952  5457 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:24.019177  5696 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.020819  5457 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:23.804301  5695 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.804677  5542 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:23.804386  5693 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:23.806865  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:24.021862  5695 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:23.852124  5670 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.518s	user 0.005s	sys 0.000s
I20260709 13:01:24.022694  5642 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:23.865638  1614 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:23.899104  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:24.023840  1535 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:24.022079  5693 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:23.896865  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.024869  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:24.025527  1535 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:24.026043  5693 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:23.944064  1685 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:24.026397  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:24.022761  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 215963us
I20260709 13:01:24.027153  1535 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:23.945549  1686 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:23.947679  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:24.027714  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.028062  2237 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:23.936295  5656 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.028432  5695 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:24.029124  5457 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:24.025269  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.029719  5457 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.030956  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:24.031658  1536 raft_consensus.cc:2368] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
I20260709 13:01:24.032513  5457 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:23.963073  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 358427us
I20260709 13:01:24.034855  1658 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:23.966024  1506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.033s	sys 0.023s
I20260709 13:01:24.037233  5578 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:01:24.037683  1506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.034s	sys 0.023s
I20260709 13:01:23.970712  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:24.038821  5578 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:23.975800  2236 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:23.978161  1582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.032s	sys 0.039s
W20260709 13:01:24.040037  1582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.032s	sys 0.039s
I20260709 13:01:23.978962  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:24.041790  2233 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:24.042353  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:24.042973  1613 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:24.043923  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:24.044510  1613 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:24.044838  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:24.021941  5542 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:23.990947  2231 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:24.004562  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:23.950537  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:24.154590  1543 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:23.959390  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:24.045509  1611 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:23.979444  1692 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:24.045854  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:24.156675  1609 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:24.046144  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:23.992502  5266 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:24.392177  5266 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:23.952136  5483 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:24.005697  5559 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:24.045310  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:24.046957  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:24.028846  5656 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.048839  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:24.044519  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:24.054090  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:24.409943  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:24.410686  1687 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:24.412235  2201 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:24.413125  5572 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:24.014632  5470 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:24.057221  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:24.058947  1584 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:24.430540  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 371699us
I20260709 13:01:24.430862  5457 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:24.431265  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:24.432595  1584 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:24.433519  5338 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:24.433928  5338 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:24.434432  5338 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:24.064278  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:24.437335  2234 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:24.438356  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:24.438922  2234 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:01:24.064486  1690 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:01:24.441439  1690 pending_rounds.cc:85] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 8
I20260709 13:01:24.441896  1690 pending_rounds.cc:107] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:01:24.442301  1690 raft_consensus.cc:2889] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:24.065013  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:24.444062  1683 raft_consensus.cc:2368] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
W20260709 13:01:24.067565  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.070411  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:24.453663  5436 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:24.454492  5436 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:24.073462  2232 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:24.078224  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.079876  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:24.082149  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.083295  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:24.460824  2237 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:24.087769  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:24.461882  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:24.462399  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:24.032938  5696 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:24.089807  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 52498.8us
W20260709 13:01:24.092304  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.098655  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:24.006620  5690 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:23.630945  2228 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:24.120255  1684 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:24.122728  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:24.126206  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:24.131222  5686 raft_consensus.cc:3055] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:24.139519  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 792) took 1307 ms. Trace:
I20260709 13:01:24.097160  2233 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:24.160193  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:24.157866  1610 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:24.166208  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:24.238047  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:24.207427  5715 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:01:24.236618  5701 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.247627  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:24.167338  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:24.248832  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 244339us
I20260709 13:01:24.250897  5711 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:24.248934  5692 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:24.264577  5542 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:24.265288  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 70411.9us
I20260709 13:01:24.268282  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:24.304807  5661 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:24.314504  1689 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:01:24.326954  1581 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.015s	sys 0.028s
I20260709 13:01:24.341243  5428 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.346524  5722 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:24.354776  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:24.362648  5492 raft_consensus.cc:2764] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:01:24.374559  1543 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:24.379045  5700 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:24.384665  5431 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:24.393891  5483 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:24.393340  5266 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.396124  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:24.399658  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:24.400054  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2132) took 1146 ms. Trace:
I20260709 13:01:24.401767  1613 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:24.055787  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:24.413581  5572 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.416821  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:24.426745  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2136) took 1100 ms. Trace:
I20260709 13:01:24.434072  1584 leader_election.cc:400] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
I20260709 13:01:24.062912  1615 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:24.463104  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:24.468609  5698 consensus_queue.cc:1059] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:01:24.469441  5699 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:01:24.514340  5698 raft_consensus.cc:3055] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:24.469579  5642 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:01:24.514887  5698 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:24.515301  5642 consensus_queue.cc:1243] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:24.515903  5698 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.471843  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:24.470384  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:24.518335  2228 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 8: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:24.475382  2231 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:24.519546  2228 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:24.476351  2235 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:24.517164  5483 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.520864  2228 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:01:24.466498  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:24.521636  5483 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:24.522428  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 56043.1us
I20260709 13:01:24.477140  1611 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:24.477732  1609 raft_consensus.cc:2410] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 11 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 13:01:24.478214  1509 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:01:24.525105  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46983us
I20260709 13:01:24.525218  5711 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:24.525871  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:24.526826  1509 leader_election.cc:400] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 7 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8.
I20260709 13:01:24.478708  5692 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:24.481750  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:24.528362  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:24.528607  5692 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:24.482923  1581 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.016s	sys 0.028s
I20260709 13:01:24.532526  5693 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:24.532918  5693 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:24.533272  5693 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:24.534992  1535 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:24.483212  5428 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.537789  5707 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:24.538295  5707 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:24.538746  5707 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:24.483974  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 129273us
I20260709 13:01:24.540688  2203 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:24.537308  5542 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:24.541582  5656 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:24.473912  2230 rpcz_store.cc:276] 0709 13:01:22.831784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.837741 (+  5957us) service_pool.cc:224] Handling call
0709 13:01:22.856157 (+ 18416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.260646 (+404489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.139147 (+878501us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b540036a6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:24.139412 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":928326912}
I20260709 13:01:24.541937  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:24.484570  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.543119  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:24.486135  5266 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:24.544059  1542 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:24.497839  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:24.486213  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2137) took 1156 ms. Trace:
I20260709 13:01:24.508317  2236 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:24.545642  1543 rpcz_store.cc:276] 0709 13:01:23.329754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.155640 (+825886us) service_pool.cc:224] Handling call
0709 13:01:24.249821 (+ 94181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.484801 (+234980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.485973 (+  1172us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b54003e1f54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:24.486139 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":286144896}
W20260709 13:01:24.508981  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 75100.6us
I20260709 13:01:24.547258  1615 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:24.476212  2229 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:24.463716  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:24.550108  1540 raft_consensus.cc:2368] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
I20260709 13:01:24.550956  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:24.498528  1508 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:24.552132  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 53703.2us
I20260709 13:01:24.552877  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:24.469257  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:24.583307  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 85612.6us
I20260709 13:01:24.584161  1616 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:24.500653  1538 rpcz_store.cc:276] 0709 13:01:23.254059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.422877 (+168818us) service_pool.cc:224] Handling call
0709 13:01:23.443510 (+ 20633us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:23.443819 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:23.443845 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:23.444327 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 13:01:23.609210 (+164883us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:23.609258 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:23.609342 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:24.399812 (+790470us) raft_consensus.cc:1685] finished
0709 13:01:24.399832 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.399998 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":832,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"thread_start_us":164494,"threads_started":1,"wal-append.queue_time_us":261209}
I20260709 13:01:24.507704  1683 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:24.529234  1541 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:24.586866  1689 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:24.587988  1537 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:24.500463  1536 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:01:24.487615  1539 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:24.595978  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2173) took 1193 ms. Trace:
I20260709 13:01:24.637215  1615 rpcz_store.cc:276] 0709 13:01:23.402656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.404257 (+  1601us) service_pool.cc:224] Handling call
0709 13:01:24.595730 (+1191473us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b54003c8cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:24.595902 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1462755200}
I20260709 13:01:24.591215  1582 leader_election.cc:400] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
I20260709 13:01:24.638464  1582 leader_election.cc:403] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:01:24.578970  2234 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
I20260709 13:01:24.639268  2233 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 6. Current term is 7. Ops: []
I20260709 13:01:24.508455  1534 rpcz_store.cc:276] 0709 13:01:23.326377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:23.485661 (+159284us) service_pool.cc:224] Handling call
0709 13:01:23.559841 (+ 74180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:23.560099 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:23.584814 (+ 24715us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.397749 (+812935us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b1800349000. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c979c0a 00007fe0960b9219 00007fe08f6a2d70 00007fe08f696293 00007fe08f68bcea 00007fe08f68c23a 00007fe08faf6a2a 00007fe08fb060bf 00007fe08fb06038
0709 13:01:24.426019 (+ 28270us) log.cc:844] Serialized 14 byte log entry
0709 13:01:24.426364 (+   345us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:24.426412 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:24.426467 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.426542 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:24.426678 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6485,"prepare.queue_time_us":17533,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":92,"raft.queue_time_us":191,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":155,"spinlock_wait_cycles":6013696,"thread_start_us":27366,"threads_started":1,"wal-append.queue_time_us":34455}
I20260709 13:01:24.596990  1535 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:24.590333  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2138) took 1249 ms. Trace:
I20260709 13:01:24.641104  1539 rpcz_store.cc:276] 0709 13:01:23.340508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.486938 (+1146430us) service_pool.cc:224] Handling call
0709 13:01:24.487275 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:24.590088 (+102813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:24.590279 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:24.642019  1613 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:24.650552  5730 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:24.517295  5698 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:24.658087  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 74292.9us
I20260709 13:01:24.514256  5492 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:01:24.655186  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 107695us
I20260709 13:01:24.663180  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:24.485328  5700 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:24.542132  5578 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 13:01:24.635630  5690 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:24.664840  1658 leader_election.cc:400] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10.
I20260709 13:01:24.674314  1658 leader_election.cc:403] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:24.485472  5431 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:24.541953  5542 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:24.484368  5661 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:24.506506  5572 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:24.654472  1540 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:24.473666  5686 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:24.661396  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 134687us
I20260709 13:01:24.636782  5699 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:24.665246  5696 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
W20260709 13:01:24.671156  1507 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.599s (SENT)
I20260709 13:01:24.591527  5559 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:24.582759  5723 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.589710  5428 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:24.656144  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 113184us
I20260709 13:01:24.542209  5656 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.117s
I20260709 13:01:24.652765  5642 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.720413  5642 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:24.720966  5642 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.690289  1542 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:24.476863  5701 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.660979  5457 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
W20260709 13:01:24.723351  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 60293.8us
I20260709 13:01:24.723397  1618 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:24.679925  5655 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:24.700614  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 240622us
I20260709 13:01:24.662344  5492 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:24.699196  1509 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:24.728308  2234 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:24.703593  1535 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:01:24.706700  5731 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:24.708294  1612 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:24.582940  5721 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:24.705121  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:24.681226  5542 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:23.384665   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:24.734185  5695 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.738200  5695 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:24.741412  5733 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:24.741927  5733 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.742280  5733 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:24.745740  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46678.3us
I20260709 13:01:24.746536  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:24.747588  5695 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:24.748044  5695 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:24.748447  5695 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:24.724313  1584 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:24.725518  1583 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:24.676880  5711 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10.
I20260709 13:01:24.702934  5690 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:24.702502  5699 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.709040  5559 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:24.776714  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 233644us
W20260709 13:01:24.749604  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 106241us
I20260709 13:01:24.483325  5722 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:24.755489  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 84400.5us
I20260709 13:01:24.723611  5483 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:24.749348  1509 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:24.735127  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 76357.7us
I20260709 13:01:24.780993  5428 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:01:24.677330  5436 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:01:24.775616  1543 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: [8.9-10.10]
I20260709 13:01:24.753801  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:24.758349  1440 catalog_manager.cc:5697] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 4 to 6, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 6 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:01:24.785817  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 61582.8us
I20260709 13:01:24.799005  1584 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:24.786912  5732 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:24.803299  5693 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.804414  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:24.805240  1683 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:24.806656  5701 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:24.789600  5505 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:24.788471  5695 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:24.808080  5505 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.808624  5505 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:24.811676  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.785336  5733 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.813607  5492 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.815016  5492 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:24.816294  5695 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:24.814119  5733 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:24.791196  5436 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:01:24.793562  2237 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:24.765004  5696 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:24.821945  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:01:24.794039  1507 leader_election.cc:336] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.182s (SENT)
I20260709 13:01:24.824716  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:24.795830  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:24.786825  1509 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:24.781067  5699 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:24.833073  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:24.781384  5559 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:24.862632  2234 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:24.791385  5722 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:24.878429  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 82674.5us
I20260709 13:01:24.873077  2234 raft_consensus.cc:2410] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:01:24.878625  1440 catalog_manager.cc:5697] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 7 to 10, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 10 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:01:24.874775  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:24.885746  2233 raft_consensus.cc:2410] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:24.830309  5695 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:24.832361  5696 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:24.873510  5732 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:24.914027  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 115099us
I20260709 13:01:24.917032  1584 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:24.917752  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.760733  5700 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.874233  5711 raft_consensus.cc:2764] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:01:24.931098  1440 catalog_manager.cc:5697] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 5 to 8, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 8 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:01:24.931680  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:24.886397  5691 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:24.769766  5731 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.758522  5686 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:24.941465  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 119599us
I20260709 13:01:24.888576  5578 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:24.932771  5457 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:24.818972  5431 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:24.927295  5696 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:24.938057  5266 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:24.913885  5698 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:24.953830  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:24.954941  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 166492us
W20260709 13:01:24.958376  1441 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 341) took 34132 ms (client timeout 15000 ms). Trace:
I20260709 13:01:24.960033  1440 catalog_manager.cc:5697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:24.796450  5483 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:24.961843  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 85546.1us
I20260709 13:01:24.942889  5642 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:24.969659  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:24.943498  5700 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:24.943553  5701 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:24.984892  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.930819  5542 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:24.977845  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:24.972294  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 178274us
W20260709 13:01:24.991598  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.950764  5578 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:24.935297  5505 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:25.006678  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.973393  5642 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:25.020314  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.973696  5721 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:24.931341  5732 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:24.970932  5698 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:25.032548  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 54819.2us
I20260709 13:01:25.036953  1537 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:25.010667  1440 catalog_manager.cc:5697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.015537  5578 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.039467  1541 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:25.039739  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 70150.7us
I20260709 13:01:25.072067  2203 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:25.001199  5730 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:01:25.061765  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:24.930776  5690 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.027781  5711 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:25.037616  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 73246.7us
I20260709 13:01:24.966651  5340 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:01:24.944175  5691 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:25.076452  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 144842us
W20260709 13:01:25.043951  2258 debug-util.cc:398] Leaking SignalData structure 0x7b0800224b60 after lost signal to thread 434
W20260709 13:01:25.086531  2258 debug-util.cc:398] Leaking SignalData structure 0x7b0800242bc0 after lost signal to thread 1475
W20260709 13:01:25.087888  2258 debug-util.cc:398] Leaking SignalData structure 0x7b08002c0200 after lost signal to thread 1567
W20260709 13:01:25.088166  2258 debug-util.cc:398] Leaking SignalData structure 0x7b08002bfa20 after lost signal to thread 1568
W20260709 13:01:25.089183  2258 debug-util.cc:398] Leaking SignalData structure 0x7b0800376140 after lost signal to thread 1642
I20260709 13:01:25.026811  5693 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:25.032274  5738 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:25.083143  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:25.089468  2258 debug-util.cc:398] Leaking SignalData structure 0x7b08002db5a0 after lost signal to thread 1643
I20260709 13:01:25.089978  1584 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:25.104815  2258 debug-util.cc:398] Leaking SignalData structure 0x7b08003e9600 after lost signal to thread 1716
W20260709 13:01:25.105137  2258 debug-util.cc:398] Leaking SignalData structure 0x7b0800365fc0 after lost signal to thread 1717
W20260709 13:01:24.964463  1441 rpcz_store.cc:269] 0709 13:00:50.819410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:00:50.819651 (+   241us) service_pool.cc:224] Handling call
0709 13:01:24.945347 (+34125696us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:22.095100 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:22.174755 (+ 79655us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:22.174810 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:01:22.174846 (+    36us) tablet.cc:662] Decoding operations
0709 13:01:22.212578 (+ 37732us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:22.212712 (+   134us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:22.212748 (+    36us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:01:22.214425 (+  1677us) tablet.cc:701] Row locks acquired
0709 13:01:22.216053 (+  1628us) write_op.cc:260] PREPARE: finished
0709 13:01:22.234238 (+ 18185us) write_op.cc:270] Start()
0709 13:01:22.234422 (+   184us) write_op.cc:276] Timestamp: P: 1783602082234141 usec, L: 0
0709 13:01:22.234449 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:01:22.307038 (+ 72589us) log.cc:844] Serialized 5936 byte log entry
0709 13:01:22.577830 (+270792us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fae7e28 00007fe093522581 00007fe093521d58 00007fe0935269f1 00007fe0935269a9 00007fe093526939 00007fe093526901 00007fe093525b2d 000000000038f354 000000000038f199
0709 13:01:22.752764 (+174934us) op_driver.cc:464] REPLICATION: finished
0709 13:01:22.825082 (+ 72318us) write_op.cc:301] APPLY: starting
0709 13:01:22.846300 (+ 21218us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:22.902937 (+ 56637us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:22.902976 (+    39us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:22.946678 (+ 43702us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:22.947079 (+   401us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=30
0709 13:01:22.947126 (+    47us) write_op.cc:312] APPLY: finished
0709 13:01:22.953611 (+  6485us) log.cc:844] Serialized 107 byte log entry
0709 13:01:22.965956 (+ 12345us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:22.971511 (+  5555us) write_op.cc:454] Released schema lock
0709 13:01:22.978689 (+  7178us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:23.020756 (+ 42067us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:23.020789 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":140344,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":48724,"apply.run_cpu_time_us":6510,"apply.run_wall_time_us":176796,"num_ops":15,"prepare.queue_time_us":244285,"prepare.run_cpu_time_us":23025,"prepare.run_wall_time_us":482939,"raft.queue_time_us":112421,"raft.run_cpu_time_us":27884,"raft.run_wall_time_us":38201,"replication_time_us":518126,"spinlock_wait_cycles":38923648,"thread_start_us":430881,"threads_started":4,"wal-append.queue_time_us":192278,"wal-append.run_cpu_time_us":20980,"wal-append.run_wall_time_us":1480518}]]}
I20260709 13:01:25.105029  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.105042  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.001160  5734 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 13:01:25.105327  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.081007  1440 catalog_manager.cc:5697] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 7 to 8, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 8 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:01:25.106478  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.107056  1440 catalog_manager.cc:5697] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 4 to 6, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:01:25.107600  1610 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:25.109282  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:25.110097  1617 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:25.075479  5461 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:25.085990  5642 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:01:25.113896  5340 consensus_queue.cc:1243] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:25.155635  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:25.155900  5738 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:25.159302  5738 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:25.162909  5732 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:25.164757  5505 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:25.117908  5735 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:01:25.166159  5731 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:25.166843  5698 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.168164  5698 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.103196  5470 raft_consensus.cc:3055] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:25.118083  2233 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:25.169957  5470 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:25.170912  1616 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:25.171525  5470 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.112943  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:25.114283  5661 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:01:25.178654  5338 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.744s	user 0.010s	sys 0.000s
W20260709 13:01:25.178843  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 106841us
I20260709 13:01:25.179240  5338 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:25.183869  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:25.186575  5745 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:01:25.187243  1507 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.982s (SENT) [suppressed 67 similar messages]
I20260709 13:01:25.130915  1611 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:25.141613  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 65915.8us
I20260709 13:01:25.148671  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2678) took 1346 ms. Trace:
I20260709 13:01:25.190842  1541 rpcz_store.cc:276] 0709 13:01:23.802320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.587737 (+785417us) service_pool.cc:224] Handling call
0709 13:01:25.039288 (+451551us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.071292 (+ 32004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.148462 (+ 77170us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b54000ed3d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:25.148608 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":889427200}
W20260709 13:01:25.113507  2258 debug-util.cc:398] Leaking SignalData structure 0x7b08003a7800 after lost signal to thread 2261
W20260709 13:01:25.191680  2258 debug-util.cc:398] Leaking SignalData structure 0x7b0800328b00 after lost signal to thread 2262
I20260709 13:01:25.116520  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:25.116354  1440 catalog_manager.cc:5697] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 7 to 9, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 9 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.180922  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:25.205068  2237 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
W20260709 13:01:25.209331  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 119429us
I20260709 13:01:25.210101  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:25.210508  1584 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:25.160496  5742 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.083s	user 0.002s	sys 0.001s
I20260709 13:01:25.210711  1615 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:25.214269  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:25.215880  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:25.220115  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:25.220701  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:25.221288  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:25.222007  1692 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:25.232434  2236 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:25.134884  5736 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:01:25.223703  1440 catalog_manager.cc:5697] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 7 to 8, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 8 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.242228  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:25.220760  1688 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
W20260709 13:01:25.129136  5707 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.590s	user 0.003s	sys 0.009s
I20260709 13:01:25.247329  1584 leader_election.cc:400] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 7 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8.
I20260709 13:01:25.217083  5745 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:25.257557  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:25.258726  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:25.259037  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:25.259400  1690 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:25.259562  1613 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:25.131803  5730 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.251634  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:25.273887  1614 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:25.130247  5655 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:25.251308  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.280755  1611 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:25.263533  1440 catalog_manager.cc:5697] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 7, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 7 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.209369  5695 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:25.266680  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.142493  5483 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:25.146900  5691 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:25.146811  5542 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:25.282262  2235 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:25.284198  2201 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:25.393178  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:25.285068  1615 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:25.217999  5457 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:25.217217  5431 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:25.218416  5738 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:25.300561  1582 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:25.311272  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.234208  5733 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.322539  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:25.271454  5740 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:25.217665  5737 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:25.145869  5461 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:25.394717  1440 catalog_manager.cc:5697] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:01:25.395978  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.279918  5266 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:25.258759  5642 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:25.399534  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 99117us
I20260709 13:01:25.403251  1613 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
W20260709 13:01:25.404345  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 223511us
I20260709 13:01:25.422827  2203 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:25.260458  5744 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:01:25.393313  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 109228us
I20260709 13:01:25.405272  2234 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:25.415463  1536 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:25.417168  1440 catalog_manager.cc:5697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.323310  5436 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:01:25.287545  5692 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.134s
I20260709 13:01:25.262864  5707 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:25.288478  5730 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.397969  5470 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:25.546006  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50179.7us
W20260709 13:01:25.485121  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 62380.3us
I20260709 13:01:25.412199  5731 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:25.463227  1614 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
W20260709 13:01:25.525908  5559 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.638s	user 0.009s	sys 0.002s
I20260709 13:01:25.398188  5691 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:25.554847  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 334202us
W20260709 13:01:25.540143  5743 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:01:25.513654  5723 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:25.560629  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:25.571815  2203 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:25.573053  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:25.418540  5745 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:25.497541  5492 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.682s	user 0.012s	sys 0.000s
I20260709 13:01:25.418154  5737 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:25.457940  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:25.547869  5686 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:25.562733  1440 catalog_manager.cc:5697] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 8, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 8 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:25.579769  1617 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:25.582481  2228 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:25.582857  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 125046us
I20260709 13:01:25.575480  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:25.409648  5695 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:25.555596  5713 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:25.418823  5733 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:25.569537  5723 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.590943  5734 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:25.592677  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:25.612035  5572 consensus_meta.cc:287] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.885s	user 0.008s	sys 0.000s
I20260709 13:01:25.574613  5745 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:25.598040  5750 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:01:25.599200  5747 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.602532  2229 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:25.569876  5559 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.604943  5707 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:25.605608  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:25.611013  5730 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:25.423998  5338 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:25.616667  1609 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:25.523613  5711 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
W20260709 13:01:25.617673  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.555071  5731 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.405809  5457 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.577940  5722 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:25.526350  5578 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.580399  5655 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:25.576746  5744 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:25.574259  5740 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:01:25.472900  5531 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.159s
W20260709 13:01:25.648216  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.651244  2234 raft_consensus.cc:2410] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:01:25.582022  5737 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:25.427206  5266 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:25.603423  5700 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.605s	user 0.008s	sys 0.001s
I20260709 13:01:25.683836  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2679) took 1881 ms. Trace:
W20260709 13:01:25.685696  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 418541us
I20260709 13:01:25.696762  5730 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:25.718751  1536 rpcz_store.cc:276] 0709 13:01:23.802626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.589374 (+786748us) service_pool.cc:224] Handling call
0709 13:01:24.657727 (+ 68353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.556720 (+898993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.683609 (+126889us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b540030a7d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:25.683774 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1742114688}
I20260709 13:01:25.719642  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:25.638060  5723 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.569662  5691 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.636837  5734 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.688648  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:25.691797  5701 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:25.695483  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.672621  5745 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:25.702932  1440 catalog_manager.cc:5697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:01:25.704125  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 128705us
W20260709 13:01:25.739264  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50707.6us
I20260709 13:01:25.740301  2203 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:25.700577  5722 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:25.754433  1440 catalog_manager.cc:5697] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 6, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:01:25.759479  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:25.760004  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.689744  5747 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:25.713042  5741 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 13:01:25.714871  5733 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:25.780035  5737 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.720643  2231 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:25.721289  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:25.673525  5572 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.794756  5744 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:25.795804  5572 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:25.731832  5483 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.733736  1683 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:25.796643  5483 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:25.740813  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 153217us
I20260709 13:01:25.715786  5656 raft_consensus.cc:2764] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:01:25.744110  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:25.798978  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:25.799852  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 55860.1us
I20260709 13:01:25.800544  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:25.800617  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:25.718222  5461 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:25.801951  5531 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:25.802481  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:25.802708  5531 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:01:25.803282  5440 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:25.761668  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:25.804153  1613 raft_consensus.cc:2410] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:25.767725  1440 catalog_manager.cc:5697] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 4 to 5, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:01:25.735078  5734 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.770903  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.726621  5698 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:25.742129  5701 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.726486  5492 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 13:01:25.777112  1611 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:25.778563  5457 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:25.781186  5753 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 13:01:25.725848  5338 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:01:25.696341  5731 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:25.783128  5691 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:25.785107  5749 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.258s	user 0.003s	sys 0.000s
I20260709 13:01:25.764868  5692 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.850701  5692 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.852834  5692 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:25.786075  5756 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:01:25.786002  5763 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:01:25.774134  5655 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:25.762940  5722 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.789733  5761 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:25.790266  5700 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:25.790812  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:25.793035  5747 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:25.760749  1617 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:25.805380  1537 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:25.862362  5655 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.772715  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:25.871476  5655 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:25.871855  1541 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:25.872875  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:25.805687  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:25.874130  1683 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:25.801409  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:25.834211  5440 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:25.734936  5730 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:25.750108  5340 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:25.824116  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 91707.9us
W20260709 13:01:25.825134  2203 connection.cc:256] RPC call timeout handler was delayed by -1.0964092419999361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16110000000000002s for a response
I20260709 13:01:25.826658  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.827499  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:25.883172  1542 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:25.829470  5738 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:01:25.828868  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.832020  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:25.888499  1615 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:25.888767  1539 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:25.832664  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:25.836081  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:25.836921  1538 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:25.837793  1535 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:01:25.838815  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:25.845093  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:25.838773  5457 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:25.849576  5758 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:25.846840  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:25.798933  5737 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:25.857023  2233 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:25.832825  5707 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:25.861195  1612 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
W20260709 13:01:25.905989  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 65185.1us
I20260709 13:01:25.835568  5698 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:25.877641  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 76301.5us
I20260709 13:01:25.757014  5746 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:25.885015  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:25.883775  5340 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.875938  5713 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:01:25.890877  1691 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:25.907248  1542 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:25.907235  1509 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:25.887414  1618 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:25.884300  5747 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:25.896034  5461 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:25.923852  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:25.928683  5739 raft_consensus.cc:2764] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:01:25.930858  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.931210  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.931805  1615 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:25.932366  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.932906  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.934620  1615 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:25.933248  1616 raft_consensus.cc:2410] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:25.937978  2203 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:25.939459  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:25.909955  5707 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:25.913800  5531 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:25.916164  5340 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:25.901681  5457 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:25.924182  5699 consensus_meta.cc:287] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.037s	user 0.002s	sys 0.010s
I20260709 13:01:25.947328  1615 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:01:25.925441  5542 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:25.923139  5483 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:25.923543  5559 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:25.946754  5696 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.961s	user 0.012s	sys 0.000s
W20260709 13:01:25.955636  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 79585.5us
I20260709 13:01:25.956643  1615 pending_rounds.cc:85] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 8
I20260709 13:01:25.966842  1615 pending_rounds.cc:107] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:01:25.948521  1618 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:25.936532  5747 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:25.967880  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:25.943758  5737 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:25.968611  5747 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:25.959244  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:25.969794  1616 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:25.942209  5578 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:25.972852  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 59594.2us
I20260709 13:01:25.948395  5707 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:25.976660  5759 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:01:25.977547  5764 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:25.979326  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:25.979593  5762 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 13:01:25.963791  5699 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.995245  5699 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:25.980505  5738 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:25.974664  5696 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.996549  5738 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.997087  5738 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:25.998055  5266 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:25.998172  5696 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:25.999800  5457 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:25.958851  5440 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:25.972581  5542 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:25.949689  5572 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:25.950204  5531 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:25.987135  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 793) took 1111 ms. Trace:
I20260709 13:01:25.950007  5721 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:01:25.987342  1615 raft_consensus.cc:2889] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:26.007884  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:26.009719  2231 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:26.003504  1689 rpcz_store.cc:276] 0709 13:01:24.875999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.876458 (+   459us) service_pool.cc:224] Handling call
0709 13:01:24.876763 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:24.886438 (+  9675us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:25.073709 (+187271us) spinlock_profiling.cc:243] Waited 4.63 ms on lock 0x7b180030c520. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c979c0a 00007fe0960b9219 00007fe08ca01a40 00007fe08c9fc4e4 00007fe08c9fc455 00007fe0934e43c7 00007fe08faf69a7 00007fe08fb060bf 00007fe08fb06038
0709 13:01:25.396000 (+322291us) log.cc:844] Serialized 14 byte log entry
0709 13:01:25.673083 (+277083us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:25.684655 (+ 11572us) log.cc:844] Serialized 26 byte log entry
0709 13:01:25.694435 (+  9780us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:25.694475 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:25.694542 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:25.974759 (+280217us) raft_consensus.cc:1685] finished
0709 13:01:25.986921 (+ 12162us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:25.987074 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50749,"prepare.queue_time_us":256586,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":40,"raft.queue_time_us":54189,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":12537,"spinlock_wait_cycles":28261120,"thread_start_us":453896,"threads_started":3,"wal-append.queue_time_us":262022}
I20260709 13:01:26.015856  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:26.015930  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:26.016633  2234 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:26.005532  5531 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:26.016831  2229 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:26.020617  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:26.021363  2232 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:26.033854  5721 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:26.034487  5721 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:26.037701  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64117.6us
I20260709 13:01:26.037659  1683 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:26.025380  5572 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:26.061074  5572 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.061677  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:26.065549  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 44366.7us
I20260709 13:01:26.067305  2201 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:26.067344  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:26.071202  2231 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:26.071816  5699 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.072659  5699 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.016633  5764 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:26.002745  5542 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:25.977255  5698 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.082439  5661 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:25.986384  5730 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:26.085194  1507 leader_election.cc:336] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.599s (SENT)
I20260709 13:01:26.089020  5713 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:26.086901  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:26.073768  5457 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:26.070571  5696 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.018894  5745 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:26.104199  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 163267us
I20260709 13:01:26.105968  2203 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:26.092061  5695 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:26.035414  5721 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.105182  5777 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:01:26.106276  5758 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:26.137833  5758 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:26.137953  5745 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:26.137547  5764 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.138847  5758 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:26.139325  5655 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.122625  5772 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:26.143085  5722 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.127055  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:26.098639  5457 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.149945  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:26.153481  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:26.154151  2236 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:26.156606  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:26.156939  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:26.164433  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.160601  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:26.167568  1610 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:26.168669  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:26.168951  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:26.170017  1534 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:26.170650  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:26.171439  1540 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:26.173082  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:26.135130  2232 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:26.146179  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:26.176564  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:26.176687  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:26.134215  1617 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:26.150779  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:26.102558  5661 raft_consensus.cc:697] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:26.145073  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:26.182811  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:26.183652  1616 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:26.182528  2234 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:26.185746  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:26.186410  2230 raft_consensus.cc:2376] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7. Re-sending same reply.
I20260709 13:01:26.155699  5457 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:26.187336  1613 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: [8.9-10.10]
I20260709 13:01:26.155251  5772 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:26.109702  5696 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.194797  1610 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:26.170163  5655 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:26.203665  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:26.204332  1615 raft_consensus.cc:2376] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12. Re-sending same reply.
I20260709 13:01:26.208627  1616 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:26.208966  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:26.209597  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:26.209643  1539 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:26.193099  5713 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:26.211374  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:26.194052  5721 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:26.213761  1618 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:01:26.172657  5731 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.196127  5699 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:26.189869  5572 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:01:26.203747  5737 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:26.197834  5695 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:01:26.221302  5761 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.225466  5690 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:26.226774  5731 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:26.245385  1618 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:01:26.227377  2235 raft_consensus.cc:2368] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12.
I20260709 13:01:26.229132  5713 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:26.231010  5340 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:26.233523  5774 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:26.206864  5542 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:26.203701  5692 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:01:26.207991  5772 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:26.301851  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 145330us
I20260709 13:01:26.260844  5722 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:26.243583  5690 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:26.256764  1507 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.715s (SENT)
I20260709 13:01:26.308025  1618 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: [11.5-11.5]
W20260709 13:01:26.308609  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 51915.3us
I20260709 13:01:26.234580  1540 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:26.200950  5691 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 10.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:01:26.218974  5738 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.233963  5773 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.154s
I20260709 13:01:26.307641  2232 raft_consensus.cc:2376] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11. Re-sending same reply.
I20260709 13:01:26.305012  5531 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.307576  5751 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.323246  1618 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:01:26.238008  5661 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.312201  5266 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:26.245019  5761 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.331918   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:26.334725  5531 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:26.336110  5779 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:01:26.301653  5542 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:26.373459  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:26.375234  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:26.387743  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.374648  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:26.411473  1683 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:01:26.426826  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:26.433797  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.439469  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:26.413276  5440 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:26.472414  5340 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:26.473179  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.511087  5531 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:26.451045  5723 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.727s	user 0.010s	sys 0.005s
I20260709 13:01:26.511348  5693 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:26.511448  1685 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:26.511861  5693 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.511785  5723 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.512472  5693 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:26.479792  5542 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:26.367064  5266 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.491191  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.489001  5745 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.343735  5713 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:26.331097  5751 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:26.520301  5266 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:26.493443  5761 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:26.494926  5738 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:26.425853  5772 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:26.502381  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:26.506206  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.543449  1686 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:26.413038  1618 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:26.513849  5531 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.510888  5440 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:26.555105  5440 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:26.533294  5745 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:26.476575  5707 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:26.563581  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:26.564432  1686 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:26.554147  5785 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.054s	user 0.000s	sys 0.002s
I20260709 13:01:26.522507  5723 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:26.559065  2236 raft_consensus.cc:2410] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 9.
W20260709 13:01:26.510063  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 135464us
W20260709 13:01:26.572695  2201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.000s	sys 0.013s
W20260709 13:01:26.607993  2201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.000s	sys 0.013s
W20260709 13:01:26.608419  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 541122us
W20260709 13:01:26.501293  5734 consensus_meta.cc:287] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.000s
I20260709 13:01:26.591401  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 814) took 1408 ms. Trace:
I20260709 13:01:26.612250  2233 rpcz_store.cc:276] 0709 13:01:25.183186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.206985 (+ 23799us) service_pool.cc:224] Handling call
0709 13:01:25.856841 (+649856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:25.901890 (+ 45049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.591202 (+689312us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b54002a35d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:26.591340 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1393115392}
I20260709 13:01:26.520445  5751 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:26.614292  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:01:26.615060  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.613476  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:26.643863  1688 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:26.644253  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.639456  1690 raft_consensus.cc:2410] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:01:26.644716  1658 connection.cc:256] RPC call timeout handler was delayed by -1.6782601250000653s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
W20260709 13:01:26.651115  1658 connection.cc:256] RPC call timeout handler was delayed by -1.5909601250000378s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
W20260709 13:01:26.651578  1658 connection.cc:256] RPC call timeout handler was delayed by -1.4847601250000366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
W20260709 13:01:26.651948  1658 connection.cc:256] RPC call timeout handler was delayed by -1.1742601250000462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
W20260709 13:01:26.652319  1658 connection.cc:256] RPC call timeout handler was delayed by -1.1382601250000448s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
W20260709 13:01:26.652700  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0338601250000465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22280000000000003s for a response
W20260709 13:01:26.653071  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0176601250000772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
I20260709 13:01:26.654874  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.656059  1683 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:26.656454  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:26.657310  1689 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:26.654946  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:26.659246  1539 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:26.649734  5483 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:01:26.661966  5741 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.661881  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:26.662210  5751 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:26.662680  5741 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.663149  5754 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.662900  5751 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:26.663335  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:26.663702  5754 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.664248  5751 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.664767  5741 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:26.664944  5692 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.666185  1536 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:26.665867  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:26.665833  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:26.666473  1583 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:26.677800  1540 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:26.666515  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:26.642694  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:26.680542  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 66316.5us
I20260709 13:01:26.714998  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:26.716550  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.717355  1541 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:26.718241  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:26.614727  5266 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 13:01:26.737955  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:26.665510  5483 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:26.665769  5699 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:26.739071  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:26.643945  5696 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:26.749881  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.751971  1539 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:26.705821  5747 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:26.635802  5734 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:26.758198  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:26.759574  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:26.747185  5732 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:26.766207  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.758203  1689 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:26.760947  5745 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:26.665560  5692 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.744254  5699 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:26.708626  5457 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.755653  5461 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:26.772194  5461 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:26.772575  5699 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.773947  5457 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:26.720207  5741 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 13:01:26.756601  5747 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.774044  5699 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:26.783233  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 66162.3us
I20260709 13:01:26.784950  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:26.786182  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.787297  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.795359  1616 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 13:01:26.767977  5761 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.754573  5483 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:26.796927  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.788304  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.798772  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:26.811491  1539 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:26.770462  5266 consensus_queue.cc:1243] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:26.801790  5746 raft_consensus.cc:3055] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:26.816094  1538 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:26.816452  5746 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:26.781330  5747 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:26.817569  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:26.817394  5746 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.818202  1535 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:26.818538  5746 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:26.821309  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 510619us
I20260709 13:01:26.776252  5732 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:26.821907  5707 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:26.822548  5732 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.824167  5721 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:26.824720  5721 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.816156  5761 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:26.798801  1612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1683) took 1695 ms (client timeout 1792 ms). Trace:
W20260709 13:01:26.827082  1612 rpcz_store.cc:269] 0709 13:01:25.103571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.103752 (+   181us) service_pool.cc:224] Handling call
0709 13:01:26.798640 (+1694888us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b5400389854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:26.798747 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1500565888}
I20260709 13:01:26.827668  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.829286  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:26.780830  5754 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:26.807674  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:26.768177  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.867578  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:26.867679  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 53236.1us
I20260709 13:01:26.870956  1686 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:26.871693  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.872608  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:26.826613  5578 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.013s
W20260709 13:01:26.875906  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 68308.6us
I20260709 13:01:26.876749  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:26.874692  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.879238  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:26.876821  1657 leader_election.cc:336] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.609s (SENT)
I20260709 13:01:26.884407  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:26.885394  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:26.885346  1657 leader_election.cc:336] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.683s (SENT)
I20260709 13:01:26.885537  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:26.829120  5730 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:26.830921  5661 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:26.891623  1691 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:26.892215  1692 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:26.887414  1657 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:26.895819  1689 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:26.896385  1584 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:26.900575  2228 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:01:26.900753  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:26.788808  5690 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.901620  2236 raft_consensus.cc:2410] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:26.897588  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:26.904387  2231 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:26.906499  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:26.891023  5338 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:26.888038  5686 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:26.908478  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:26.831769  5542 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:01:26.908209  5461 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.919550  5686 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:26.914654  5730 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:26.915655  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:26.920017  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:26.928515  2228 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:26.928625  1691 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:26.929474  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:26.915436  1690 raft_consensus.cc:2368] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12.
I20260709 13:01:26.930500  2229 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:26.892457  5436 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:26.924211  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:26.937322  2230 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:26.929687  2232 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: [8.9-10.10]
I20260709 13:01:26.927278  1538 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:26.927604  1687 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:26.926127  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:26.941579  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:26.927023  2234 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:01:26.943506  2231 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:01:26.944051  2231 pending_rounds.cc:85] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 8
I20260709 13:01:26.944329  2231 pending_rounds.cc:107] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:01:26.944619  2231 raft_consensus.cc:2889] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:26.945019  2231 pending_rounds.cc:107] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:01:26.947592  1684 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:26.921792  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:26.945302  2231 raft_consensus.cc:2889] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:26.907737  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.930995  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:26.934696  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:26.951839  2236 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:26.936033  2235 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:26.916821  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:26.950704  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:26.918929  5656 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.956436  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:26.908748  5761 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:26.914340  5692 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:26.981416  2228 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:26.959694  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:26.934145  5642 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:26.925948  5686 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:26.977162  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:26.987972  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 53884.3us
I20260709 13:01:26.927139  5266 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:26.939852  5733 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:26.919981  5808 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:01:26.959556  1584 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:26.924896  5461 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:26.995900  1429 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.237s	user 0.000s	sys 0.000s
I20260709 13:01:26.997505  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:27.003430  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 357565us
I20260709 13:01:27.001082  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:27.001212  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:27.004828  2236 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:26.914582  5661 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:27.007342  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:27.005151  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:26.959024  5763 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:27.009189  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 95035.2us
W20260709 13:01:27.011063  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 51579.2us
I20260709 13:01:27.016436  5436 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:27.016860  5436 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.017221  5436 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:27.019701  5732 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.020264  5732 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:27.022578  5732 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:27.023099  5732 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.023427  1509 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:27.023535  5732 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:27.026307  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:27.028474  1541 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:27.030416  5738 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.030625  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:27.031066  5738 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:27.031509  1540 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:27.032207  5732 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:27.033088  1657 leader_election.cc:336] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.925s (SENT)
I20260709 13:01:27.033515  1657 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:27.034673  5738 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:27.034723  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:27.035825  1536 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:27.036455  5800 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.036734  5732 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.037032  5800 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:27.037336  5732 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:27.039407  5800 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:27.040980  5711 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:26.983678  5457 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:26.930963  5731 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:01:27.043975  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:27.044375  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:26.984082  5764 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:27.045171  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:27.000120  5686 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:27.043512  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:27.043507  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:27.047899  1657 leader_election.cc:336] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.931s (SENT)
I20260709 13:01:26.940549  5693 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:27.051582  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:27.054352  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:27.056445  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:27.073612  1612 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:27.045593  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:27.059005  1584 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:27.052157  5732 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:27.044456  5784 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:01:27.071839  2233 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:26.983947  5745 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
W20260709 13:01:27.115513  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56726.2us
I20260709 13:01:27.045665  5642 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:27.078953  1614 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:27.091120  1657 leader_election.cc:336] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.021s (SENT)
I20260709 13:01:26.986815  5656 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:27.081455  2230 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:27.071662  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:27.116937  1657 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:27.046581  5457 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:27.049798  5733 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:27.137121  5733 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:27.140313  5661 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:27.001933  5692 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 13:01:27.050083  5763 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:27.146128  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:27.053159  5436 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:27.129598  5721 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:27.129873  5811 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:01:27.088991  5711 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:27.009634  5266 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:27.086310  5428 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:27.071792  5734 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:27.127580  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 84159.4us
I20260709 13:01:27.126434  1657 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:30.828308  5837 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:30.870257  5841 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:30.875581  1506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.876s	user 0.015s	sys 0.046s
I20260709 13:01:30.877853  5840 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:31.185201  5841 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.185681  5840 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.188419  5841 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:31.188668  5840 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:30.880436  5842 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.890213  5843 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:30.894062  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1709) took 3976 ms. Trace:
W20260709 13:01:30.896739  1612 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 3.823s	user 0.030s	sys 0.023s
I20260709 13:01:31.190063  1612 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:31.189690  5843 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.189822  5842 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:31.190646  1612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 700) took 4135 ms (client timeout 1857 ms). Trace:
W20260709 13:01:31.191035  1612 rpcz_store.cc:269] 0709 13:01:27.056056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.056205 (+   149us) service_pool.cc:224] Handling call
0709 13:01:31.190545 (+4134340us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:30.843463  1429 maintenance_manager.cc:643] P 1ca716594c6348a883b88d48068b8416: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 11.849s	user 0.200s	sys 0.162s Metrics: {"bytes_written":30287,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":307641,"dirs.run_cpu_time_us":544,"dirs.run_wall_time_us":3733,"drs_written":1,"lbm_read_time_us":483,"lbm_reads_lt_1ms":4,"lbm_write_time_us":111587,"lbm_writes_1-10_ms":3,"lbm_writes_10-100_ms":2,"lbm_writes_lt_1ms":29,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":272405,"threads_started":1,"update_count":15}
I20260709 13:01:31.191984  5843 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.192235  5842 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:30.930274  1543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2703) took 3888 ms (client timeout 1951 ms). Trace:
W20260709 13:01:31.194468  1543 rpcz_store.cc:269] 0709 13:01:27.041557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:30.929973 (+3888416us) service_pool.cc:211] Skipping call since client already timed out
0709 13:01:30.930167 (+   194us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:01:30.930946  5847 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:30.936784  1508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.795s	user 0.016s	sys 0.040s
W20260709 13:01:30.939347  1618 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.890s	user 0.027s	sys 0.057s
I20260709 13:01:30.960719  5849 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:30.976258  1685 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.968s	user 0.020s	sys 0.028s
W20260709 13:01:31.198837  1685 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.190s	user 0.021s	sys 0.029s
I20260709 13:01:30.976385  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2697) took 3957 ms. Trace:
I20260709 13:01:31.199656  1535 rpcz_store.cc:276] 0709 13:01:27.018865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.024353 (+  5488us) service_pool.cc:224] Handling call
0709 13:01:27.024805 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.051645 (+ 26840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.051677 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.052138 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:01:30.962083 (+3909945us) spinlock_profiling.cc:243] Waited 3.82 s on lock 0x7b180004c360. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:01:30.975873 (+ 13790us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:30.975921 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.975996 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:30.976046 (+    50us) raft_consensus.cc:1685] finished
0709 13:01:30.976071 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.976292 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26953,"raft.queue_time_us":89243,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":22084,"spinlock_wait_cycles":8779980544,"thread_start_us":13599,"threads_started":1,"wal-append.queue_time_us":3889031}
I20260709 13:01:31.200726  5847 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:30.976980  1583 connection.cc:256] RPC call timeout handler was delayed by -2.4939634219999789s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2412s for a response
W20260709 13:01:31.202157  1583 connection.cc:256] RPC call timeout handler was delayed by -2.4371146029999977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
W20260709 13:01:31.202602  1583 connection.cc:256] RPC call timeout handler was delayed by -2.4207697239999675s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
I20260709 13:01:30.979018  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1680) took 4321 ms. Trace:
I20260709 13:01:31.202880  5842 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:30.981066  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1684) took 4322 ms. Trace:
I20260709 13:01:30.986034  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:30.987980  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1692) took 4265 ms. Trace:
I20260709 13:01:30.989312  5844 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:31.030539  1536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2705) took 3904 ms (client timeout 2169 ms). Trace:
W20260709 13:01:31.204416  1536 rpcz_store.cc:269] 0709 13:01:27.126418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.030272 (+3903854us) service_pool.cc:211] Skipping call since client already timed out
0709 13:01:31.030426 (+   154us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:01:31.030923  5851 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:31.038128  1686 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:01:31.206809  1508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.065s	user 0.018s	sys 0.041s
I20260709 13:01:31.203548  1684 rpcz_store.cc:276] 0709 13:01:26.658863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.888304 (+229441us) service_pool.cc:224] Handling call
0709 13:01:26.947461 (+ 59157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.948000 (+   539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.980814 (+4032814us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b54000ab9d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:30.980995 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66850560}
W20260709 13:01:31.040081  1507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 4.030s	user 0.014s	sys 0.085s
I20260709 13:01:31.043160  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.058182  1534 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:31.208844  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2704) took 4167 ms. Trace:
I20260709 13:01:31.069514  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 772) took 4241 ms. Trace:
I20260709 13:01:31.204068  1691 rpcz_store.cc:276] 0709 13:01:26.722366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.914396 (+192030us) service_pool.cc:224] Handling call
0709 13:01:26.914727 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.929188 (+ 14461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.987622 (+4058434us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b540038aed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:30.987900 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16638848}
I20260709 13:01:31.209620  1539 rpcz_store.cc:276] 0709 13:01:26.827876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.828242 (+   366us) service_pool.cc:224] Handling call
0709 13:01:26.828517 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.889381 (+ 60864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.889415 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.889900 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:01:27.082383 (+192483us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:27.113258 (+ 30875us) log.cc:844] Serialized 14 byte log entry
0709 13:01:27.113476 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.113553 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.069160 (+3955607us) raft_consensus.cc:1685] finished
0709 13:01:31.069185 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.069383 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76173,"prepare.queue_time_us":30753,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":121126,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":12733,"thread_start_us":142569,"threads_started":1,"wal-append.queue_time_us":298447,"wal-append.run_cpu_time_us":2548,"wal-append.run_wall_time_us":1002575}
W20260709 13:01:31.074244  1690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.989s	user 0.042s	sys 0.060s
I20260709 13:01:31.083209  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 771) took 4255 ms. Trace:
I20260709 13:01:31.096591  5853 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:31.114981  1687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 4.002s	user 0.027s	sys 0.036s
I20260709 13:01:31.122426  5854 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:31.210724  1618 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.161s	user 0.028s	sys 0.058s
W20260709 13:01:27.243257  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 292592us
W20260709 13:01:31.342655  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:30.948683  2233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 1021) took 4025 ms (client timeout 1951 ms). Trace:
W20260709 13:01:31.434906  2233 rpcz_store.cc:269] 0709 13:01:26.922790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.933752 (+ 10962us) service_pool.cc:224] Handling call
0709 13:01:30.948393 (+4014641us) spinlock_profiling.cc:243] Waited 74.8 ms on lock 0x7b540065aed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:30.948555 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172062592}
W20260709 13:01:30.781689  2202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.182s	user 0.071s	sys 0.128s
W20260709 13:01:31.439991  2202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.892s	user 0.074s	sys 0.128s
I20260709 13:01:27.199569  2232 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:27.189939  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:31.474210  2236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 839) took 4307 ms (client timeout 1673 ms). Trace:
W20260709 13:01:31.474632  2236 rpcz_store.cc:269] 0709 13:01:27.166397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.166580 (+   183us) service_pool.cc:224] Handling call
0709 13:01:31.474093 (+4307513us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:30.929865  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:27.148241  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:31.480666  2230 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 838) took 4337 ms (client timeout 1777 ms). Trace:
I20260709 13:01:27.195741  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:31.482448  2237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 840) took 4302 ms (client timeout 1822 ms). Trace:
W20260709 13:01:31.482820  2237 rpcz_store.cc:269] 0709 13:01:27.180474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.180758 (+   284us) service_pool.cc:224] Handling call
0709 13:01:31.482357 (+4301599us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:27.179561  2229 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:31.484880  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 828) took 4560 ms. Trace:
I20260709 13:01:31.485234  2229 rpcz_store.cc:276] 0709 13:01:26.924918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.931326 (+  6408us) service_pool.cc:224] Handling call
0709 13:01:26.958695 (+ 27369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.484098 (+4525403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.484635 (+   537us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b5400472a54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:31.484816 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497303296}
I20260709 13:01:27.242800  5266 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:27.243019  5338 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:27.201586  5428 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:27.245330  5436 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:27.214043  5440 consensus_meta.cc:287] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.658s	user 0.010s	sys 0.001s
I20260709 13:01:31.493217  5440 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.495602  5440 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:27.142833  5457 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:27.199638  5483 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:27.258693  5578 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:31.503465  5578 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:27.246119  5656 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:27.247746  5686 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.510905  5686 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:27.892107  5692 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.003s	sys 0.034s
W20260709 13:01:31.514065  5692 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.340s	user 0.003s	sys 0.037s
I20260709 13:01:27.218283  5693 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:27.125461  5699 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:31.518129  5699 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.518649  5699 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:27.200073  5730 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.095546  5733 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.523813  5733 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.526515  5733 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:30.860807  5744 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:27.141237  5746 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:31.527869  5744 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:29.638741  5751 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.497s	user 0.075s	sys 0.122s
I20260709 13:01:31.530390  5744 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:30.752728  5739 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.373s	user 0.022s	sys 0.046s
W20260709 13:01:31.535851  5739 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 3.250s	user 0.023s	sys 0.047s
I20260709 13:01:27.135113  5747 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:27.145623  5761 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:01:31.538911  5747 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.539091  5761 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:31.539474  5761 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:27.219188  5763 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:30.901054  5773 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.546509  5773 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:30.860705  5470 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.550436  5470 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.025142  5531 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:27.121295  5642 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.555210  5642 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:30.933679  5690 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:30.898262  5701 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:30.893313  5740 raft_consensus.cc:3055] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:31.563361  5740 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:27.199241  5807 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 13:01:30.894611  5797 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 4.139s	user 0.016s	sys 0.013s
I20260709 13:01:31.134233  5855 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:31.569568  5855 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.570173  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2692) took 4909 ms. Trace:
I20260709 13:01:31.570591  1537 rpcz_store.cc:276] 0709 13:01:26.661274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.664213 (+  2939us) service_pool.cc:224] Handling call
0709 13:01:26.664546 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.673899 (+  9353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.673950 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:26.710002 (+ 36052us) log.cc:844] Serialized 26 byte log entry
0709 13:01:26.896893 (+186891us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:01:26.896936 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:26.897004 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.569936 (+4672932us) raft_consensus.cc:1685] finished
0709 13:01:31.569961 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.570111 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":491,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":137026,"threads_started":1,"wal-append.queue_time_us":245343}
I20260709 13:01:31.571979  5855 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.138554  5698 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:31.572768  5698 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.574966  5698 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:31.185698  1506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.186s	user 0.017s	sys 0.046s
I20260709 13:01:31.203400  5847 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:31.202885  1583 connection.cc:256] RPC call timeout handler was delayed by -2.2939346489999934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
W20260709 13:01:31.577629  1583 connection.cc:256] RPC call timeout handler was delayed by -2.2426373129999888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
I20260709 13:01:31.564500  5740 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.036950  5852 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:01:31.578310  1506 connection.cc:256] RPC call timeout handler was delayed by -2.4712488160000134s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 13:01:31.578888  5740 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:31.206764  5844 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.579450  5739 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.580005  5739 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.581089  5844 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.185186  5837 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.582780  5739 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.583139  5837 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:27.193007  2201 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:31.213104  5854 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:31.585752  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 4.39241e+06us
I20260709 13:01:31.587436  5854 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.213379  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.588590  1684 raft_consensus.cc:2410] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:01:31.213765  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1718) took 4168 ms. Trace:
W20260709 13:01:31.214075  1507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.204s	user 0.014s	sys 0.087s
I20260709 13:01:31.209167  1534 rpcz_store.cc:276] 0709 13:01:27.041836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:31.057458 (+4015622us) service_pool.cc:224] Handling call
0709 13:01:31.057821 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.208581 (+150760us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.208786 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.212164  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:31.213119  1687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.128s	user 0.028s	sys 0.036s
I20260709 13:01:31.211628  1538 rpcz_store.cc:276] 0709 13:01:26.827578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.828178 (+   600us) service_pool.cc:224] Handling call
0709 13:01:26.864546 (+ 36368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.939099 (+ 74553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.082889 (+4143790us) spinlock_profiling.cc:243] Waited 62.5 ms on lock 0x7b54001e5854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:31.083130 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143821568}
W20260709 13:01:31.217733  1584 connection.cc:256] RPC call timeout handler was delayed by -2.2897819049999839s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
W20260709 13:01:31.591938  1584 connection.cc:256] RPC call timeout handler was delayed by -2.0854819049999946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
I20260709 13:01:31.589404  1618 rpcz_store.cc:276] 0709 13:01:27.045764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.047979 (+  2215us) service_pool.cc:224] Handling call
0709 13:01:27.048361 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:27.048806 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:27.048839 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.049408 (+   569us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.213354 (+4163946us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:31.213402 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.213501 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.213543 (+    42us) raft_consensus.cc:1685] finished
0709 13:01:31.213555 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.213688 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73936,"raft.queue_time_us":1185,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":24624,"thread_start_us":4163601,"threads_started":1,"wal-append.queue_time_us":158908,"wal-append.run_cpu_time_us":1827,"wal-append.run_wall_time_us":1016464}
W20260709 13:01:31.592269  1584 connection.cc:256] RPC call timeout handler was delayed by -2.00848991700002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2169s for a response
W20260709 13:01:31.592572  1584 connection.cc:256] RPC call timeout handler was delayed by -1.7225924059999898s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26080000000000003s for a response
W20260709 13:01:31.592859  1584 connection.cc:256] RPC call timeout handler was delayed by -1.339111245999959s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30150000000000005s for a response
W20260709 13:01:31.590435  1507 leader_election.cc:336] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.893s (SENT)
I20260709 13:01:31.593597  1507 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:31.593729  1684 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:31.203030  1683 rpcz_store.cc:276] 0709 13:01:26.657515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.682968 (+ 25453us) service_pool.cc:224] Handling call
0709 13:01:26.683423 (+   455us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:26.771444 (+ 88021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.771495 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:26.835551 (+ 64056us) log.cc:844] Serialized 52 byte log entry
0709 13:01:26.984519 (+148968us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:27.070882 (+ 86363us) log.cc:844] Serialized 14 byte log entry
0709 13:01:27.113609 (+ 42727us) log.cc:844] Serialized 14 byte log entry
0709 13:01:27.113861 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:27.113954 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:30.978738 (+3864784us) raft_consensus.cc:1685] finished
0709 13:01:30.978764 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.978947 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122843,"prepare.queue_time_us":164731,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":87165,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":41941,"spinlock_wait_cycles":40506624,"thread_start_us":91102,"threads_started":1,"wal-append.queue_time_us":299952,"wal-append.run_cpu_time_us":2920,"wal-append.run_wall_time_us":1045561}
I20260709 13:01:31.594349  5739 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:01:31.241192  1690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.170s	user 0.043s	sys 0.060s
W20260709 13:01:31.590446  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 246911us
I20260709 13:01:31.243175  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:31.247649  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 4.12086e+06us
I20260709 13:01:31.590651  5730 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:31.596031  1686 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:31.596381  5730 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:31.248870  5859 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:31.596676  1584 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.951s (SENT) [suppressed 4 similar messages]
I20260709 13:01:31.597081  5859 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:31.597302  5730 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.598047  5859 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.590806  1691 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:31.600782  1507 connection.cc:256] RPC call timeout handler was delayed by -4.8975203109999939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
W20260709 13:01:31.252385  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 266424us
W20260709 13:01:31.598604  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.210412  5842 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:31.602367  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:31.603214  5808 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:31.603363  5860 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.603765  5808 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.212182  5849 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:31.604598  1657 connection.cc:256] RPC call timeout handler was delayed by -3.6017268329999865s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1557s for a response
I20260709 13:01:31.602376  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:31.605114  1657 connection.cc:256] RPC call timeout handler was delayed by -2.9726268329999925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2256s for a response
W20260709 13:01:31.605479  1657 connection.cc:256] RPC call timeout handler was delayed by -2.2406531330000234s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2761s for a response
I20260709 13:01:31.212563  5853 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.262230  5843 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.312204  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 805) took 4699 ms. Trace:
I20260709 13:01:31.318674  5856 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.269202  5858 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:31.320631  5840 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.327024  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2698) took 4307 ms. Trace:
I20260709 13:01:31.346361  5841 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:01:31.353339  5340 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.358811  5861 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:31.389938  5862 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:31.394584  1582 connection.cc:256] RPC call timeout handler was delayed by -2.9331671999999571s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
I20260709 13:01:31.418419  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:31.624142  1582 connection.cc:256] RPC call timeout handler was delayed by -2.7013468069999931s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
W20260709 13:01:31.624565  1582 connection.cc:256] RPC call timeout handler was delayed by -2.5974671999999828s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2412s for a response
W20260709 13:01:31.624866  1582 connection.cc:256] RPC call timeout handler was delayed by -2.5471327330000122s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1703s for a response
W20260709 13:01:31.625126  1582 connection.cc:256] RPC call timeout handler was delayed by -2.1056325699999547s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22320000000000004s for a response
W20260709 13:01:31.625383  1582 connection.cc:256] RPC call timeout handler was delayed by -1.9427325699999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2413s for a response
W20260709 13:01:31.625684  1582 connection.cc:256] RPC call timeout handler was delayed by -1.7222297089999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26080000000000003s for a response
I20260709 13:01:31.624169  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1710) took 4692 ms. Trace:
I20260709 13:01:31.626371  1689 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:31.443727  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 841) took 4259 ms. Trace:
I20260709 13:01:31.627316  2232 rpcz_store.cc:276] 0709 13:01:27.184870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.185548 (+   678us) service_pool.cc:224] Handling call
0709 13:01:27.199082 (+ 13534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:31.443370 (+4244288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.443638 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.444506  5845 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.445569  5863 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:31.452805  5864 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.467259  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1704) took 5536 ms. Trace:
I20260709 13:01:31.629386  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.479514  5793 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:31.481050  2230 rpcz_store.cc:269] 0709 13:01:27.143834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.148000 (+  4166us) service_pool.cc:224] Handling call
0709 13:01:31.480593 (+4332593us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:31.491520  5428 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.639016  5853 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:31.530403  5751 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 4.389s	user 0.077s	sys 0.122s
I20260709 13:01:31.531940  5825 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:31.520328  5578 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.562389  5457 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.630101  1684 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:31.642998  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.643528  1684 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:31.644380  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.644856  1684 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:31.578563  5852 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.189618  1692 rpcz_store.cc:276] 0709 13:01:26.917928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.927036 (+  9108us) service_pool.cc:224] Handling call
0709 13:01:26.954804 (+ 27768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.955016 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:27.041856 (+ 86840us) log.cc:844] Serialized 14 byte log entry
0709 13:01:30.848254 (+3806398us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b18004574e0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08f6944bc 00007fe08c9e248b 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a
0709 13:01:30.893616 (+ 45362us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:30.893681 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:30.893738 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:30.893811 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:30.893959 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":129866,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":5764,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":39326592,"thread_start_us":100058,"threads_started":2,"wal-append.queue_time_us":183771,"wal-append.run_cpu_time_us":1338,"wal-append.run_wall_time_us":1003022}
I20260709 13:01:31.603865  5860 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.641310  5864 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.604281  5842 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.211980  5851 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.641434  5863 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:31.642094  5825 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:31.656442  5825 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.595518  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:31.658051  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 62599.1us
I20260709 13:01:31.659211  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:31.548678  5773 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.606086  5808 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.616534  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.540555  5761 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.661597  5837 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:31.661803  1683 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:31.662104  5761 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:31.662309  5837 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:31.662722  5837 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:31.663065  5837 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:31.553046  5470 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.608780  5531 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:31.606552  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:31.606169  5849 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.622133  5656 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:31.666142  5808 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:31.623071  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.666428  5656 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.667057  1686 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:31.426151  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.667960  1688 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:31.624249  5340 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.668622  5656 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.598062  5746 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:31.669216  5825 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:31.633209  5841 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:31.670073  5841 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.670550  5340 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.629163  5862 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.653210  5860 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.655444  5863 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.672238  5841 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.627089  5861 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.672673  5862 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.618489  5843 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.674270  5861 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.635032  5845 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.674602  5843 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.622776  5840 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.683362  5840 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.669346  5746 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.677489  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:31.685917  5746 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.649720  5457 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.641098  5793 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.686933  2203 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:31.688391  5793 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:31.619863  5856 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.626389  1687 rpcz_store.cc:276] 0709 13:01:26.931948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.939972 (+  8024us) service_pool.cc:224] Handling call
0709 13:01:26.983857 (+ 43885us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:26.984268 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.984305 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:27.002715 (+ 18410us) log.cc:844] Serialized 52 byte log entry
0709 13:01:31.591105 (+4588390us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:31.616289 (+ 25184us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.623012 (+  6723us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.623327 (+   315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.623402 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.623441 (+    39us) raft_consensus.cc:1685] finished
0709 13:01:31.623453 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.624116 (+   663us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25258,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":3939,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":19507,"spinlock_wait_cycles":47872,"thread_start_us":4536856,"threads_started":3,"wal-append.queue_time_us":205031,"wal-append.run_cpu_time_us":1948,"wal-append.run_wall_time_us":1002269}
I20260709 13:01:31.655400  5851 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:31.613865  1685 rpcz_store.cc:276] 0709 13:01:26.612769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.613101 (+   332us) service_pool.cc:224] Handling call
0709 13:01:26.613396 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.613644 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:27.008229 (+394585us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.311799 (+4303570us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:31.311864 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:31.311921 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.311993 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.312135 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":171294,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":69292,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"thread_start_us":4697284,"threads_started":3,"wal-append.queue_time_us":317256,"wal-append.run_cpu_time_us":1433,"wal-append.run_wall_time_us":1082957}
I20260709 13:01:31.692178  5856 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.650767  5852 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.578975  5747 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.653303  5864 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.681466  5845 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.693382  5747 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:31.690284  5870 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:01:31.696863  5578 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:31.697005  5793 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:31.690230  5837 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:31.702471  2231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 831) took 4746 ms (client timeout 1979 ms). Trace:
W20260709 13:01:31.702884  2231 rpcz_store.cc:269] 0709 13:01:26.955684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.955990 (+   306us) service_pool.cc:224] Handling call
0709 13:01:31.702242 (+4746252us) spinlock_profiling.cc:243] Waited 4.71 s on lock 0x7b54006586d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:31.702400 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10839802240}
I20260709 13:01:31.701007  5793 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:31.700845  5699 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.722286  5747 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.724776  5699 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.726573  5837 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:31.728847  5793 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.393782  1430 maintenance_manager.cc:419] P 1ca716594c6348a883b88d48068b8416: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10011 bytes on disk
W20260709 13:01:31.207597  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:31.346357  1509 connection.cc:256] RPC call timeout handler was delayed by -2.9291900929999883s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
W20260709 13:01:31.764220  1509 connection.cc:256] RPC call timeout handler was delayed by -2.7089547350000203s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18220000000000003s for a response
W20260709 13:01:31.625977  1582 connection.cc:256] RPC call timeout handler was delayed by -1.4513325699999768s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2959s for a response
W20260709 13:01:31.765717  1582 connection.cc:256] RPC call timeout handler was delayed by -1.3663327329999788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30150000000000005s for a response
I20260709 13:01:31.518908  1438 catalog_manager.cc:5697] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 6 to 8, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 8 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:31.619941  1542 rpcz_store.cc:276] 0709 13:01:27.019179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:27.025216 (+  6037us) service_pool.cc:224] Handling call
0709 13:01:27.025604 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:30.957477 (+3931873us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b480030c858. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:01:31.141053 (+183576us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:31.141477 (+   424us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.248597 (+107120us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:31.249243 (+   646us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.249505 (+   262us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:31.249839 (+   334us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.250091 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.250186 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.326716 (+ 76530us) raft_consensus.cc:1685] finished
0709 13:01:31.326744 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.326951 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26331,"prepare.queue_time_us":4211,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":375,"raft.queue_time_us":299256,"raft.run_cpu_time_us":47010,"raft.run_wall_time_us":3830168,"spinlock_wait_cycles":34944768,"thread_start_us":174456,"threads_started":2,"wal-append.queue_time_us":135754}
W20260709 13:01:31.578011  1583 connection.cc:256] RPC call timeout handler was delayed by -2.186959010999999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1876s for a response
I20260709 13:01:31.629668  1617 rpcz_store.cc:276] 0709 13:01:25.930582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:25.931761 (+  1179us) service_pool.cc:224] Handling call
0709 13:01:25.932047 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:26.177608 (+245561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.466932 (+5289324us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b540021ff54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:31.467130 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":464327936}
W20260709 13:01:31.782020  1583 connection.cc:256] RPC call timeout handler was delayed by -2.0744092179999711s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22320000000000004s for a response
W20260709 13:01:31.782584  1583 connection.cc:256] RPC call timeout handler was delayed by -2.062266588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
W20260709 13:01:31.782974  1583 connection.cc:256] RPC call timeout handler was delayed by -1.858832389999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2178s for a response
W20260709 13:01:31.783376  1583 connection.cc:256] RPC call timeout handler was delayed by -1.1101567039999622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30150000000000005s for a response
I20260709 13:01:31.786520  1611 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:01:31.773132  1429 maintenance_manager.cc:643] P 1ca716594c6348a883b88d48068b8416: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":703,"lbm_reads_lt_1ms":4}
W20260709 13:01:31.666285  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 59812.8us
I20260709 13:01:31.796479  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1711) took 4855 ms. Trace:
I20260709 13:01:31.797497  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:31.668663  1691 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:31.798839  1688 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:31.797137  1690 rpcz_store.cc:276] 0709 13:01:26.941109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.957923 (+ 16814us) service_pool.cc:224] Handling call
0709 13:01:26.999332 (+ 41409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:26.999734 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:26.999766 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:27.000240 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:01:31.595215 (+4594975us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:31.695762 (+100547us) log.cc:844] Serialized 14 byte log entry
0709 13:01:31.795335 (+ 99573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.795581 (+   246us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:31.795730 (+   149us) raft_consensus.cc:1685] finished
0709 13:01:31.795745 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:31.796202 (+   457us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17718,"prepare.queue_time_us":21302,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":167,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":40071,"thread_start_us":4722733,"threads_started":3,"wal-append.queue_time_us":245208,"wal-append.run_cpu_time_us":2301,"wal-append.run_wall_time_us":1146997}
I20260709 13:01:31.802445  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:31.803916  1686 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:31.801100  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.799086  1689 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:31.701373  5642 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.811813  5839 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:31.822937  1538 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:31.823702  5839 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:31.812464  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:31.812747  5837 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.813550  5825 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:31.816295  1508 connection.cc:256] RPC call timeout handler was delayed by -3.5562531699999909s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16110000000000002s for a response
I20260709 13:01:31.817143  5740 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:31.818022  5340 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.839103  5340 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.818204  5747 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:31.819583  5856 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:31.822280  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:31.822664  5852 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:31.806324  1688 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:31.807404  1686 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:31.827418  1508 connection.cc:256] RPC call timeout handler was delayed by -3.4068531699999767s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
I20260709 13:01:31.816346  1685 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
W20260709 13:01:31.844823  1508 connection.cc:256] RPC call timeout handler was delayed by -3.3433836550000251s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1673s for a response
I20260709 13:01:31.830310  1542 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:31.833838  5740 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.837821  5837 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:31.845220  1508 connection.cc:256] RPC call timeout handler was delayed by -3.2484531700000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19530000000000003s for a response
W20260709 13:01:31.866281  1508 connection.cc:256] RPC call timeout handler was delayed by -3.2250531700000238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19790000000000002s for a response
W20260709 13:01:31.866722  1508 connection.cc:256] RPC call timeout handler was delayed by -3.1979768560000252s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18220000000000003s for a response
W20260709 13:01:31.867034  1508 connection.cc:256] RPC call timeout handler was delayed by -3.1008531699999935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2117s for a response
W20260709 13:01:31.867369  1508 connection.cc:256] RPC call timeout handler was delayed by -2.119853169999999s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3207s for a response
I20260709 13:01:31.850286  5747 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.852108  1438 catalog_manager.cc:5697] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 4 to 7, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 7 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:01:31.854722  1688 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.855854  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2187) took 5065 ms. Trace:
I20260709 13:01:31.858842  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.854723  5839 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:31.863842  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:31.864481  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:31.867235  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:31.869488  1611 rpcz_store.cc:276] 0709 13:01:26.790315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:26.794221 (+  3906us) service_pool.cc:224] Handling call
0709 13:01:26.794595 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:31.787458 (+4992863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:31.855396 (+ 67938us) spinlock_profiling.cc:243] Waited 4.99 s on lock 0x7b54000befd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:31.855715 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11478096768}
I20260709 13:01:31.894225  5852 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
W20260709 13:01:31.899492  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 74364.4us
I20260709 13:01:31.900580  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:31.906144  1689 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:31.905792  5699 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:01:31.897166  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:31.911134  1534 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:31.911834  5747 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.911394  5825 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:31.912608  5340 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:31.914885  5825 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.922391  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:31.923599  1613 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:31.924132  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.925704  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.926295  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:31.926733  1616 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:31.926774  1683 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 8: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:31.927145  1618 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:31.929901  5837 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.930143  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:31.930814  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.933128  1614 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:31.930145  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.934875  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.935283  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:31.936129  1540 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:31.936517  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:31.937213  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:31.937839  1534 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:31.938061  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.938356  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:31.939479  1543 raft_consensus.cc:2393] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:31.939837  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:31.940696  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:31.942106  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:31.942587  1611 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:31.943192  1535 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:31.944720  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:31.903008  5839 consensus_queue.cc:1059] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:01:31.946719  5839 raft_consensus.cc:3055] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:31.947197  5839 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:31.949579  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:31.949180  5839 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.949844  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:31.950614  1685 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:31.951243  5839 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:31.952313  1438 catalog_manager.cc:5697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 5 to 7, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:01:31.957818  5761 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:31.959033  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:31.955665  5740 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.964039  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:31.962031  1538 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:31.966876  2231 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:31.967496  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:31.962023  1686 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.967959  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:31.968704  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:31.940999  1539 raft_consensus.cc:2410] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:31.962554  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:31.961668  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:31.951380  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 349708us
I20260709 13:01:31.965090  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:31.964341  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:31.993295  1543 raft_consensus.cc:2410] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:01:31.942637  5642 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:31.995795  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:31.995262  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:31.970849  5808 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:01:31.994606  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:31.999822  1611 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:32.001583  2233 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:32.002202  1688 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:32.002775  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.004333  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.005920  2228 raft_consensus.cc:2410] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:32.006208  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.006997  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.008021  2228 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:32.008659  5837 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.009480  5837 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:31.996490  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.008955  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.014423  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.007050  1616 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:32.016341  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:32.025018  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:32.025849  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.027145  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.028715  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.029687  1683 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 7.
I20260709 13:01:32.025238  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.031355  1689 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:32.037889  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.038226  5793 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:01:32.039448  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.036051  1438 catalog_manager.cc:5697] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 3 to 7, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:32.040130  5793 consensus_queue.cc:1243] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:01:32.040256  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.040971  1539 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:32.041839  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:32.041971  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.042759  2233 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:32.043016  1617 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:32.044592  1613 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:32.045324  5730 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:32.047297  2235 raft_consensus.cc:2410] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 14 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:32.047971  5733 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:32.046618  2201 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:32.048348  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:32.048868  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.049177  5730 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:32.049835  5730 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:32.050288  5730 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.051191  5763 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.051057  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.051936  5763 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.052673  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:32.053750  5875 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.054586  5856 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:32.054725  5875 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.055359  5856 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.055541  5656 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.055696  5763 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:32.052760  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.056830  1543 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:32.056391  5656 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.057176  5763 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:32.058223  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.059828  1614 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:32.058043  5763 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.048432  5733 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.061214  2230 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:01:32.061681  5733 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:32.055796  5856 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.063458  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.064654  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:32.066087  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.066802  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.067020  1685 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:01:32.067629  5763 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:32.068039  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:32.050305  1537 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:32.061496  1686 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.075157  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.075840  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.068571  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:32.082911  1534 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:32.084530  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.065348  2231 raft_consensus.cc:2376] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8. Re-sending same reply.
I20260709 13:01:32.088649  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.094424  5837 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:32.097730  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.098161  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.098745  1538 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:32.099677  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:32.100520  2232 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:32.101944  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.103084  2232 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:32.104072  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:32.105690  5880 log.cc:927] Time spent T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:01:32.081161  2236 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:32.110016  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.110764  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.111585  2236 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:32.113051  2236 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:32.114425  1438 catalog_manager.cc:5697] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 4 to 7, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 7 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:32.118320  2229 raft_consensus.cc:2376] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10. Re-sending same reply.
I20260709 13:01:32.119585  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.121958  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.123188  1615 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:32.123611  2233 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:32.124791  2233 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:32.124549  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:32.126104  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:32.127362  1616 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:32.128582  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.130167  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.131474  1610 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:32.132642  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.133617  1615 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.087621  2237 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:32.138548  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.139668  1617 raft_consensus.cc:2410] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:01:32.140583  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.140908  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.101836  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:32.057277  5875 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:32.142766  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:32.142927  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.153625  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.143157  2234 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:32.126160  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.105769  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.105310  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.126160  1543 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:01:32.157137  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:32.126230  1507 connection.cc:256] RPC call timeout handler was delayed by -4.6965467370000056s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
I20260709 13:01:32.152715  2230 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:32.145226  1538 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:32.151417  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:32.158486  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 90028.7us
I20260709 13:01:32.159430  2235 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:32.157680  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.158442  1611 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:32.162891  1689 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:32.163287  1686 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 10. (term mismatch)
I20260709 13:01:32.163619  1616 raft_consensus.cc:2368] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9.
I20260709 13:01:32.163786  1686 pending_rounds.cc:85] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 9
I20260709 13:01:32.164458  1618 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:32.161854  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.141388  1617 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 8.
I20260709 13:01:32.167318  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.168372  1611 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:32.168979  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:32.170068  1507 leader_election.cc:336] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.138s (SENT)
I20260709 13:01:32.141927  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.169585  5845 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:32.171259  1507 connection.cc:256] RPC call timeout handler was delayed by -4.6110902309999915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
I20260709 13:01:32.171583  5845 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.172775  5837 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:32.173202  5837 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:32.173556  5656 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:32.175062  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.142154  1713 debug-util.cc:398] Leaking SignalData structure 0x7b08000b30c0 after lost signal to thread 434
I20260709 13:01:32.176626  1615 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:32.169063  1536 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:32.173640  5837 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:32.172137  1507 connection.cc:256] RPC call timeout handler was delayed by -4.4751467369999887s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2207s for a response
I20260709 13:01:32.178568  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.140594  5531 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:01:32.185873  1507 connection.cc:256] RPC call timeout handler was delayed by -4.4233636129999923s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22060000000000002s for a response
I20260709 13:01:32.186167  2231 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:32.164142  1686 pending_rounds.cc:107] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 13:01:32.245695  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:32.252682  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.267107  1617 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:01:32.224026  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 57115.3us
W20260709 13:01:32.203343  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.230050  1616 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:01:32.231011  1539 raft_consensus.cc:2393] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:32.251554  2201 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:32.186432  1507 connection.cc:256] RPC call timeout handler was delayed by -4.3116189180000219s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17410000000000003s for a response
I20260709 13:01:32.270922  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:32.244037  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 142421us
I20260709 13:01:32.265166  1686 raft_consensus.cc:2889] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:32.275135  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.279495  1611 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:32.182894  5531 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:32.283599  1507 connection.cc:256] RPC call timeout handler was delayed by -4.1961964889999877s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16110000000000002s for a response
W20260709 13:01:32.283975  1507 connection.cc:256] RPC call timeout handler was delayed by -4.1523040809999543s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1695s for a response
W20260709 13:01:32.284202  1507 connection.cc:256] RPC call timeout handler was delayed by -3.9570349319999991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19530000000000003s for a response
W20260709 13:01:32.284689  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 69587us
I20260709 13:01:32.186785  5740 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.286098  1438 catalog_manager.cc:5697] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:32.288793  5773 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:32.288326  1438 catalog_manager.cc:5697] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 2 to 8, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 8 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:32.289686  5859 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:01:32.290575  5578 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:32.290953  5578 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:32.291327  5578 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.293069  1538 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:32.286278  5740 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.218340  5852 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.295149  1536 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:32.243099  5763 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:32.298818  1507 connection.cc:256] RPC call timeout handler was delayed by -3.9266927840000108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18700000000000003s for a response
I20260709 13:01:32.289387  5773 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:01:32.299904  1507 connection.cc:256] RPC call timeout handler was delayed by -3.8067349319999835s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21200000000000002s for a response
I20260709 13:01:32.298599  1540 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:32.302878  5773 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.308092  1534 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:01:32.308691  1507 connection.cc:256] RPC call timeout handler was delayed by -3.4448553839999931s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
W20260709 13:01:32.310859  1507 connection.cc:256] RPC call timeout handler was delayed by -3.3653492669999991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
W20260709 13:01:32.311365  1713 debug-util.cc:398] Leaking SignalData structure 0x7b0800061340 after lost signal to thread 1475
W20260709 13:01:32.312196  1507 connection.cc:256] RPC call timeout handler was delayed by -3.3086492669999643s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
W20260709 13:01:32.312619  1507 connection.cc:256] RPC call timeout handler was delayed by -2.5872189180000191s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3657s for a response
W20260709 13:01:32.315127  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.316256  1507 connection.cc:256] RPC call timeout handler was delayed by -2.2970492669999771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29009999999999997s for a response
I20260709 13:01:32.204854  1618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13adc029302444abaaffd4bed11760a3"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:01:32.318454  1618 raft_consensus.cc:606] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:32.318787  1618 raft_consensus.cc:614] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:32.320945  1507 connection.cc:256] RPC call timeout handler was delayed by -2.0216492669999866s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3207s for a response
I20260709 13:01:32.329391  2201 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:32.330669  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:32.331280  1534 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:32.306914  5642 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:01:32.311728  5763 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:32.333089  2203 connection.cc:256] RPC call timeout handler was delayed by -3.4084175340000229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.163s for a response
W20260709 13:01:32.339284  2203 connection.cc:256] RPC call timeout handler was delayed by -3.2041175340000336s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1857s for a response
I20260709 13:01:32.341562  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.339588  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:32.344238  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.343381  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.345351  2235 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:32.306706  5852 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.341104  5531 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.346679  5340 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:32.357540  5690 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.357934  5340 raft_consensus.cc:3055] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:32.358528  5340 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:32.358309  5690 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.359267  5531 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:32.359663  5340 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.360771  5690 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:32.361105  5340 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.362020  5740 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:32.363600  1507 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.695s (SENT)
I20260709 13:01:32.368153  5856 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.369683  1610 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:32.370766  1507 leader_election.cc:336] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.795s (SENT)
I20260709 13:01:32.371481  5852 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:32.375545  1507 leader_election.cc:336] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.870s (SENT)
I20260709 13:01:32.375808  1618 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:32.376474  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:32.377405  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:32.377933  1611 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:01:32.378985  5340 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:32.382270  1507 leader_election.cc:336] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.961s (SENT)
I20260709 13:01:32.382449  1611 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:32.383402  1507 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.982s (SENT)
W20260709 13:01:32.383950  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.386539  1507 leader_election.cc:336] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.144s (SENT)
I20260709 13:01:32.387389  1507 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:32.388258  5340 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:32.388641  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.389518  1611 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:01:32.389938  1507 leader_election.cc:336] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.207s (SENT)
I20260709 13:01:32.391697  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:32.392652  1611 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:32.386487  1534 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:32.399034  5845 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:32.350524  5875 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:32.405835  5875 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:32.406329  5875 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:32.409770  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.412570  1508 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:32.413928  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.418182  1713 debug-util.cc:398] Leaking SignalData structure 0x7b080016cde0 after lost signal to thread 1567
W20260709 13:01:32.418645  1713 debug-util.cc:398] Leaking SignalData structure 0x7b0800191e20 after lost signal to thread 1568
W20260709 13:01:32.420142  1713 debug-util.cc:398] Leaking SignalData structure 0x7b08003d79e0 after lost signal to thread 1642
W20260709 13:01:32.420475  1713 debug-util.cc:398] Leaking SignalData structure 0x7b08003e3040 after lost signal to thread 1643
I20260709 13:01:32.422873  5733 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.425576  1617 raft_consensus.cc:2368] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11.
I20260709 13:01:32.336424  5730 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.434541  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:32.435593  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 106357us
I20260709 13:01:32.439946  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.441862  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.446079  1535 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:32.446151  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.448663  1540 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:32.452773  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.438622  5340 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:32.458360  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.453733  2236 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:32.464702  2201 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:32.468849  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.471415  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:32.475332  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.476186  1539 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:32.477950  1438 catalog_manager.cc:5697] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 1 to 5, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 5 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:32.450632  5642 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:32.480563  1438 catalog_manager.cc:5697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 5 to 6, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 6 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:01:32.483140  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.488758  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.489640  5763 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:32.491986  1582 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:32.493158  5854 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:32.493567  5845 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:32.494024  5845 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.494379  5845 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:32.457381  1540 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:32.497807  1713 debug-util.cc:398] Leaking SignalData structure 0x7b080031c580 after lost signal to thread 1716
W20260709 13:01:32.501981  1713 debug-util.cc:398] Leaking SignalData structure 0x7b08001c6780 after lost signal to thread 1717
I20260709 13:01:32.469625  5733 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:32.518750  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.518939  5854 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.519615  1686 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:32.519699  5854 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.524418  5854 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:32.528200  1689 raft_consensus.cc:1217] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Deduplicated request from leader. Original: 4.6->[6.7-10.9]   Dedup: 6.7->[8.8-10.9]
I20260709 13:01:32.528851  1689 pending_rounds.cc:85] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 7
I20260709 13:01:32.529115  1689 pending_rounds.cc:107] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:01:32.529340  1689 raft_consensus.cc:2889] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:32.531888  1583 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:32.484097  5642 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:32.540004  5642 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:32.540637  1691 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:32.548671  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.549893  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.553318  5730 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:32.555070  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.557265  5733 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:32.566967  5740 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:01:32.569839  1440 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40172 (request call id 231) took 20526 ms (client timeout 15000 ms). Trace:
W20260709 13:01:32.574800  1713 debug-util.cc:398] Leaking SignalData structure 0x7b08003b2de0 after lost signal to thread 2261
W20260709 13:01:32.566855  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 49424.4us
I20260709 13:01:32.588914  1692 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:32.590353  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.596313  5773 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.538583  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.602854  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:32.606561  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.606900  5740 consensus_queue.cc:1059] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:32.611276  5340 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:32.615882  5837 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.617091  5825 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:32.617753  1508 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:32.622207  5856 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:32.623009  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:32.679031  1686 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:32.625560  5578 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.510749  1438 catalog_manager.cc:5697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:01:32.627297  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:32.627485  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:32.628764  2230 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:32.688017  2236 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.629753  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:32.632151  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 96724.6us
I20260709 13:01:32.633104  5763 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:01:32.646531  1617 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
W20260709 13:01:32.662566  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.686532  1540 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:32.693445  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 198546us
W20260709 13:01:32.692881  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45912.9us
I20260709 13:01:32.703173  1610 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:32.707466  2228 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:32.709586  5842 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:32.712597  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.712957  1614 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:32.714075  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.716832  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.716750  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.717829  1611 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:32.719758  2236 raft_consensus.cc:2410] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:32.732261  5859 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.733139  5859 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.736436  5859 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:32.739802  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.700057  1438 catalog_manager.cc:5697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:32.744179  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 44836.5us
I20260709 13:01:32.658712  5730 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 13:01:32.701013  5854 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:32.697867  5763 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:32.702123  5825 consensus_queue.cc:1243] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 13:01:32.749306  5890 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.091s	user 0.006s	sys 0.000s
I20260709 13:01:32.749204  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.753407  1540 raft_consensus.cc:2410] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:32.753965  5773 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:32.750319  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 45005.2us
I20260709 13:01:32.755661  1583 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:32.756639  5842 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:32.757187  5842 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:32.570500  1440 rpcz_store.cc:269] 0709 13:01:12.142055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:12.177920 (+ 35865us) service_pool.cc:224] Handling call
0709 13:01:13.192146 (+1014226us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:01:13.624647 (+432501us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b5400020710. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c8640 00007fe09589fb76 00007fe0959a9c3e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:01:32.534190 (+18909543us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:26.106418 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:26.107029 (+   611us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:26.107071 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:01:26.107089 (+    18us) tablet.cc:662] Decoding operations
0709 13:01:26.119205 (+ 12116us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:26.119298 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:26.119329 (+    31us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:01:26.120623 (+  1294us) tablet.cc:701] Row locks acquired
0709 13:01:26.120643 (+    20us) write_op.cc:260] PREPARE: finished
0709 13:01:26.120820 (+   177us) write_op.cc:270] Start()
0709 13:01:26.120959 (+   139us) write_op.cc:276] Timestamp: P: 1783602086120762 usec, L: 0
0709 13:01:26.120987 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:01:26.198915 (+ 77928us) log.cc:844] Serialized 6345 byte log entry
0709 13:01:27.138307 (+939392us) op_driver.cc:464] REPLICATION: finished
0709 13:01:27.245250 (+106943us) write_op.cc:301] APPLY: starting
0709 13:01:27.245351 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:27.246463 (+  1112us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:27.246796 (+   333us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:29.482234 (+2235438us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:29.482751 (+   517us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=32
0709 13:01:29.482850 (+    99us) write_op.cc:312] APPLY: finished
0709 13:01:29.483475 (+   625us) log.cc:844] Serialized 212 byte log entry
0709 13:01:29.781325 (+297850us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:29.781513 (+   188us) write_op.cc:454] Released schema lock
0709 13:01:29.781569 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:29.782084 (+   515us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:29.782110 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1072358,"reported_tablets":39,"spinlock_wait_cycles":532988544,"child_traces":[["op",{"apply.queue_time_us":103401,"apply.run_cpu_time_us":47467,"apply.run_wall_time_us":2537792,"cfile_cache_hit":30,"cfile_cache_hit_bytes":31395,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6220,"cfile_init":2,"lbm_read_time_us":1700,"lbm_reads_lt_1ms":12,"num_ops":16,"prepare.queue_time_us":146079,"prepare.run_cpu_time_us":52822,"prepare.run_wall_time_us":224613,"raft.queue_time_us":371570,"raft.run_cpu_time_us":9436,"raft.run_wall_time_us":77503,"replication_time_us":1017145,"thread_start_us":773417,"threads_started":5,"wal-append.queue_time_us":594199,"wal-append.run_cpu_time_us":70042,"wal-append.run_wall_time_us":2554840}]]}
W20260709 13:01:32.777242  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.780215  1582 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:32.764249  1438 catalog_manager.cc:5697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 7 to 9, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 9 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:01:32.782920  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:32.784652  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.750154  5763 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:32.790496  1583 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:32.761538  5854 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:32.796176  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.792115  5856 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:32.799706  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:32.813529  1582 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:32.796689  5733 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:32.816323  5733 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:32.825203  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 126363us
I20260709 13:01:32.802331  5690 raft_consensus.cc:3055] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:32.820927  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.838310  5340 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:32.845996  5842 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.849705  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.850435  5730 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:32.853998  5578 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:32.814875  5856 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:32.860687  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.828816  1438 catalog_manager.cc:5697] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 6 to 8, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 8 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:32.853143  5740 raft_consensus.cc:3055] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:32.865288  1542 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: [9.5-10.6]
W20260709 13:01:32.867208  5893 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:01:32.846163  5854 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.872737  1540 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 13:01:32.873620  1535 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:32.874557  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:32.830737  5733 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:32.877085  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:32.840426  5340 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 13:01:32.824662  5875 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.880862  1536 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:32.830835  5690 raft_consensus.cc:740] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:32.866160  5856 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:32.866739  5740 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:32.875145  5837 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:32.883378  1438 catalog_manager.cc:5697] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 5 to 6, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 6 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:01:32.889994  1658 connection.cc:256] RPC call timeout handler was delayed by -4.390638570999954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19720000000000001s for a response
W20260709 13:01:32.892160  1658 connection.cc:256] RPC call timeout handler was delayed by -3.6427385709999953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2803s for a response
I20260709 13:01:32.893347  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:32.893532  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:32.895206  2230 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:32.901018  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 105315us
I20260709 13:01:32.902544  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:32.904479  2201 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:32.905761  1689 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.907342  1685 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:32.908646  2231 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:32.910318  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:32.896994  5773 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:32.909873  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:32.880330  5642 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:32.898752  5531 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:32.914529  1688 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.873831  5763 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.914032  1692 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:32.921715  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 173196us
I20260709 13:01:32.915875  1438 catalog_manager.cc:5697] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:01:32.924949  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 307305us
I20260709 13:01:32.909662  5837 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:32.924199  5531 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:32.928912  1438 catalog_manager.cc:5697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:32.930760  5531 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:32.930495  1438 catalog_manager.cc:5697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 5 to 6, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 6 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:32.933895  1583 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:32.934150  5844 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:32.933924  5642 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:32.934855  5844 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:32.935398  5642 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:32.935900  5642 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:32.936684  1692 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:32.937081  5844 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:32.927680  1614 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:32.943140  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:32.944320  5531 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:01:32.938103  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 150099us
I20260709 13:01:32.938508  1688 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:32.904974  5859 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:01:32.949685  5740 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:32.951139  5740 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.951875  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:32.954504  5875 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:32.905942  5793 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
W20260709 13:01:32.926769  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 97461.6us
I20260709 13:01:32.959066  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:32.959506  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:32.964783  2230 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:32.926118  5578 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:32.928259  5837 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:32.936846  5642 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.914780  5845 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:32.985801  5908 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:32.986367  5908 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:32.986765  5908 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:32.987509  5531 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:32.988148  5907 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:32.990065  5903 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:32.990559  5903 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:32.990599  5844 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:01:32.998420  5907 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:32.923137  5773 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:32.991706  5842 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:32.993773  5763 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.994166  5859 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:32.991089  5903 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:32.979990  5837 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:32.971828  5578 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.035079  2232 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:32.975433  1508 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:33.032284  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 122474us
W20260709 13:01:33.035684  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 101871us
I20260709 13:01:32.973655  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:33.037531  1689 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
W20260709 13:01:33.040241  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 68960.5us
I20260709 13:01:33.030225  5842 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:33.049109  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 61278.8us
I20260709 13:01:33.036592  5578 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.048877  5730 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.056023  5733 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.058030  5859 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:01:33.051407  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 84609.6us
I20260709 13:01:33.058527  5859 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:33.052016  5900 consensus_queue.cc:1059] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:33.059621  5900 raft_consensus.cc:3055] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:33.060160  5900 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:33.059577  5859 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:33.044984  5531 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:33.061127  5900 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.065850  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:33.068001  2203 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:01:33.082362  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:33.050616  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 98861.3us
I20260709 13:01:33.045261  5842 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:33.061216  5859 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:33.091015  2201 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:33.096589  5875 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:33.097462  5875 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:33.098467  5875 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.100555  5913 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:33.101049  5845 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:33.093741  5905 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:33.102074  5905 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:33.103173  5844 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:33.103042  5905 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.070955  5909 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:33.104329  5909 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:33.105289  5909 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.106367  5340 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.107062  5340 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.069605  5773 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:33.101120  5913 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:33.110646  2201 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:33.103580  5844 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:33.061259  5531 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:33.127384  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.130117  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:33.131752  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:33.132570  2234 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:33.135646  2234 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:33.136050  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:33.138060  2230 pending_rounds.cc:85] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 8
I20260709 13:01:33.138253  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:33.138875  2237 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.138450  2230 pending_rounds.cc:107] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:01:33.141791  2230 raft_consensus.cc:2889] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:33.152457  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 43853.9us
I20260709 13:01:33.157389  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:33.164278  5854 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:33.068279  5885 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:01:33.129182  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:33.179904  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:33.178095  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55870.2us
W20260709 13:01:33.181823  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 113889us
I20260709 13:01:33.123970  5900 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.089370  5912 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 13:01:33.205529  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:33.125675  5913 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.190135  2237 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:33.188184  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:33.202621  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:33.188226  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.094352  5856 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:33.207412  2232 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
W20260709 13:01:33.207584  5910 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:01:33.177868  5915 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:01:33.209614  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:33.210402  5642 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:33.211434  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:33.212213  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:33.248136  1687 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:33.209606  5773 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:01:33.219172  2231 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:33.222245  1689 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.089138  5690 consensus_queue.cc:260] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:33.225567  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 68332.8us
I20260709 13:01:33.257943  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:33.258152  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.225411  1508 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:33.197687  5340 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:33.220882  5854 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:33.268735  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:33.266320  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 51851.3us
I20260709 13:01:33.267769  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:33.251662  5825 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:33.281035  5928 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:33.281381  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:33.285137  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:33.281674  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 72153.8us
I20260709 13:01:33.289412  1691 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:33.296409  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:33.262864  5730 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:33.297266  1691 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:33.302120  1683 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:33.281780  5911 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:01:33.311095  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:33.282621  5914 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:01:33.311770  1692 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:33.319473  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 74607.5us
I20260709 13:01:33.281737  5898 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:33.281723  5733 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:33.327903  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:33.287518  5825 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.290560  5928 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:33.327913  1658 leader_election.cc:400] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11.
W20260709 13:01:33.331681  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 133467us
W20260709 13:01:33.329555  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 64541.9us
I20260709 13:01:33.323048  5922 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:33.331408  1614 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:33.337183  1582 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:33.339743  1658 leader_election.cc:403] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:01:33.338960  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:33.335703  5909 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:33.345036  1584 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:33.343081  1684 raft_consensus.cc:2376] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10. Re-sending same reply.
I20260709 13:01:33.351652  1689 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:33.353996  2201 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:33.357611  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 53126.5us
I20260709 13:01:33.359557  2228 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:33.301455  5340 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:01:33.359835  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:33.336649  5825 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.377471  1540 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:33.281075  5844 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:33.384169  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:33.375864  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:33.350008  5909 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.385543  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:33.411592  1691 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:33.339628  5898 raft_consensus.cc:697] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:33.359630  5922 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:33.387357  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 46995.7us
I20260709 13:01:33.387701  5916 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:33.388404  5903 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.389565  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:33.390702  1541 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:01:33.393826  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 84064us
W20260709 13:01:33.449383  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 224059us
W20260709 13:01:33.396991  5919 log.cc:927] Time spent T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:01:33.397871  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:33.340497  5928 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.427788  5909 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.488188  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:33.488893  1536 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:33.458796  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:33.430724  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:33.490459  1691 raft_consensus.cc:2410] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:33.491235  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:33.325779  5917 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.075s
W20260709 13:01:33.435909  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 73877.8us
I20260709 13:01:33.453548  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:33.455961  1582 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:33.517654  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:33.336143  5854 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.458438  1535 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:33.399979  5923 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:33.471207  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:33.480103  1689 raft_consensus.cc:2410] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:33.430263  5642 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:33.480567  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:33.481036  2230 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:33.482537  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:33.485726  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:33.493605  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:33.493638  1584 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:33.381208  5885 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:33.527343  1609 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:33.528015  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47783.8us
I20260709 13:01:33.528717  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:33.434748  5916 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:33.533393  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:33.555935  1611 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:33.534101  1658 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:33.554692  1613 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:33.572547  1612 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:33.600623  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 54922.9us
I20260709 13:01:33.548843  5844 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:33.601872  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:33.608089  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 152224us
I20260709 13:01:33.620877  1614 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:33.577764  5919 log.cc:927] Time spent T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 13:01:33.677726  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 143698us
W20260709 13:01:33.689568  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.622426  1657 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:33.668929  5837 consensus_meta.cc:287] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.627s	user 0.006s	sys 0.005s
W20260709 13:01:33.701771  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 168081us
I20260709 13:01:33.574276  5885 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:33.601521  5856 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:33.726686  1692 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:33.729007  5773 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11.
W20260709 13:01:33.731783  5578 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.678s	user 0.007s	sys 0.002s
I20260709 13:01:33.732443  5578 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:33.725658  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 394491us
W20260709 13:01:33.738919  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 286212us
I20260709 13:01:33.741902  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:33.748592  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:33.749671  1683 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:33.756889  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:33.557829  5531 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:33.784143  1584 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:33.775418  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:33.792531  2233 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:01:33.793382  1540 raft_consensus.cc:2410] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:01:33.795517  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:33.799593  5842 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.694s	user 0.005s	sys 0.004s
I20260709 13:01:33.800105  5842 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:33.801792  5934 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:33.802522  5937 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.803193  5937 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:33.804723  5763 consensus_meta.cc:287] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.772s	user 0.006s	sys 0.006s
I20260709 13:01:33.707264  5898 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.807078  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:33.802620  5934 consensus_queue.cc:1243] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:01:33.810061  1687 consensus_meta.cc:287] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.543s	user 0.005s	sys 0.004s
I20260709 13:01:33.820566  1687 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:33.759545  5856 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:33.781939  5923 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:33.710070  5837 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.838464  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:33.839506  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:33.815672  5929 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:33.863165  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:33.865783  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:33.850759  5913 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.604s	user 0.009s	sys 0.003s
I20260709 13:01:33.834250  5531 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:33.874821  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.791785  5740 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
W20260709 13:01:33.871371  2200 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.000s	sys 0.006s
I20260709 13:01:33.880035  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:33.881105  2200 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.000s	sys 0.007s
I20260709 13:01:33.859823  5929 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:33.876009  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:33.870249  5917 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:33.890359  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.891455  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:33.722153  5922 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.870501  5903 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:33.857374  5902 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.234s
I20260709 13:01:33.905756  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:33.867269  5934 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 13:01:33.844975  5940 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.211s
I20260709 13:01:33.776081  5916 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:33.913929  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:33.918113  1691 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:33.904526  5935 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.349s	user 0.001s	sys 0.000s
W20260709 13:01:33.923600  5932 log.cc:927] Time spent T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:01:33.944258  5916 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:33.944635  5690 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:33.924093  5921 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 13:01:33.924458  5946 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:33.953392  2228 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:33.925117  5940 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.954634  5940 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.926077  5954 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:33.925122  5947 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.134s
I20260709 13:01:33.957715  5946 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:33.958707  5947 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:33.958758  5946 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:33.959192  5947 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:33.959296  5946 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:33.959520  5928 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:33.960279  5947 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:33.960425  5946 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.930012  5950 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:33.962034  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:33.962757  2233 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:33.957165  5940 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:33.965554  1690 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:01:33.966023  5947 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:33.893863  5842 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:33.969223  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:33.969725  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:33.969590  5940 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:33.932546  5908 consensus_meta.cc:287] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.945s	user 0.007s	sys 0.003s
I20260709 13:01:33.889150  5917 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:33.905318  5740 consensus_queue.cc:1243] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:01:33.938387  5837 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:33.876309  5885 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:33.978127  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:33.980340  5908 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:33.980983  5942 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:01:33.985014  5578 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:33.985864  5842 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:33.980508  5899 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:33.988464  5899 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:33.974920  1508 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:33.989986  5740 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:33.990414  5740 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:33.990769  5740 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:33.990916  5899 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:33.994156  5859 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.880s	user 0.008s	sys 0.004s
I20260709 13:01:33.995204  1686 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:33.939478  5773 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:33.940179  5934 raft_consensus.cc:3055] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:33.997455  5773 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:34.004240  5906 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:34.004815  5906 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:34.005028  5951 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:34.005493  5951 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:34.005888  5906 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.006558  5951 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:34.007253  5908 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:34.007416  5906 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:34.008001  5951 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:34.010110  5929 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.011798  5929 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:33.954070  5937 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:33.972589  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:34.014973  1688 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:34.017702  5773 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:34.019265  5773 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:33.975378  1687 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:01:33.975811  5917 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:33.997514  5934 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:33.874552  5913 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.022212  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:34.026810  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:34.028172  5903 raft_consensus.cc:3055] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:34.029366  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:34.029232  5946 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:34.038230  5934 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.050244  5934 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:34.042621  5903 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:33.974154  2201 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:33.970232  5940 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:34.053368  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 79286.9us
I20260709 13:01:34.039811  5913 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:34.054956  5730 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:34.055649  5730 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:34.059952  1686 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:34.056819  5730 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.052681  5903 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:34.073551  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.074441  5903 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:34.088073  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:34.088974  1686 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:34.092330  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.094729  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:34.139739  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.158303  1614 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:34.178748  1686 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:34.180446  2231 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:34.189787  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 537239us
W20260709 13:01:34.203034  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.203824  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.210304  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:34.219394  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:34.219712  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:34.225673  2230 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:34.235363  1582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.011s	sys 0.011s
W20260709 13:01:34.235890  1582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.011s	sys 0.011s
I20260709 13:01:34.087018  5913 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.126s
I20260709 13:01:34.241840  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:34.237798  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 411900us
I20260709 13:01:34.238592  1614 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:01:34.246630  1507 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.572s (SENT)
W20260709 13:01:34.245782  1657 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.000s	sys 0.024s
W20260709 13:01:34.240409  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 416635us
W20260709 13:01:34.251353  1657 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.000s	sys 0.025s
I20260709 13:01:34.251716  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:34.218998  5763 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:34.259371  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:34.252256  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:34.260264  1612 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:34.262212  1688 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:34.264158  1611 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:01:34.236129  5937 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.243129  5950 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:34.270254  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:34.270618  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:34.271219  2203 leader_election.cc:400] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9.
I20260709 13:01:34.267812  1584 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:34.271656  2203 leader_election.cc:403] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:01:34.266867  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 644506us
I20260709 13:01:34.264964  1611 raft_consensus.cc:606] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:34.267836  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:34.272409  1611 raft_consensus.cc:614] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:34.248907  5928 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:34.275043  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:34.268167  1683 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:34.255013  1691 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:34.280694  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:34.251324  1507 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:34.284138  1655 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.021s	sys 0.055s
W20260709 13:01:34.284678  1655 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.021s	sys 0.055s
W20260709 13:01:34.285266  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.285807  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.286515  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.264225  5907 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:34.295373  1686 pending_rounds.cc:85] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 9
I20260709 13:01:34.288862  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:34.294595  2236 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:34.312415  1686 pending_rounds.cc:107] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 10.10
W20260709 13:01:34.309926  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.289634  1691 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 13:01:34.330367  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 48201.4us
I20260709 13:01:34.313710  1686 raft_consensus.cc:2889] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:34.313789  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:34.315554  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:34.284822  5733 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.001s	sys 0.015s
I20260709 13:01:34.342658  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:34.344645  1691 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:34.284127  5837 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:34.345928  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:34.347239  1690 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:34.351228  1657 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:34.361438  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:34.288231  5763 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:34.365100  2201 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:34.273391  5937 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.329294  5856 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:34.372368  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 101335us
I20260709 13:01:34.376821  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:34.373219  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:34.343315  5733 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.002s	sys 0.015s
I20260709 13:01:34.374315  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:34.313331  5961 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:34.334375  5940 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:34.343246  5844 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.371930  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:34.393224  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 126594us
W20260709 13:01:34.382586  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.386034  1683 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8. Ops: []
W20260709 13:01:34.387005  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 102488us
I20260709 13:01:34.388955  1691 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:34.408689  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:34.367663  5907 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:34.373879  5913 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.418409  1582 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:34.400066  5825 consensus_meta.cc:287] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.950s	user 0.005s	sys 0.009s
I20260709 13:01:34.405830  5854 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
W20260709 13:01:34.410192  5923 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.560s	user 0.000s	sys 0.010s
I20260709 13:01:34.420372  5968 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:34.431309  5923 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:34.420678  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:34.434563  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.420934  1684 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:01:34.422050  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 175447us
W20260709 13:01:34.435681  1507 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.589s (SENT)
I20260709 13:01:34.394233  5730 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:34.411556  5531 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.000s
W20260709 13:01:34.452718  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 46016.4us
I20260709 13:01:34.440130  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:34.368119  5837 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.401225  5940 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:34.371763  5955 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:34.432093  5909 consensus_meta.cc:287] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.855s	user 0.007s	sys 0.006s
I20260709 13:01:34.494500  1582 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:34.497836  1692 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:01:34.500027  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.432307  5899 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.514966  1684 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
W20260709 13:01:34.527406  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 90315.4us
W20260709 13:01:34.529212  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 89235us
I20260709 13:01:34.406030  5844 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:34.482008  5898 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9.
I20260709 13:01:34.528328  5959 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:34.543190  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.491828  5927 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:34.556321  1684 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:34.560314  1507 leader_election.cc:336] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.611s (SENT)
I20260709 13:01:34.425858  5825 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:34.528182  5970 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.344s	user 0.000s	sys 0.004s
W20260709 13:01:34.583279  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 89053.3us
I20260709 13:01:34.565678  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:34.571277  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.510473  5885 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:34.437956  5913 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:34.578428  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 48453.3us
I20260709 13:01:34.592281  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:34.517788  5730 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:34.593930  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 173349us
I20260709 13:01:34.598189  1584 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:34.429869  5763 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.542703  5968 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:34.610411  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:34.610809  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 234054us
I20260709 13:01:34.611764  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:34.491722  5904 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.219s
I20260709 13:01:34.590824  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:34.593859  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:34.554165  5842 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:34.595096  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:34.604329  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:34.605880  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:34.594959  1540 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:34.615531  1539 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:34.564941  5927 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:34.617156  1534 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 7.
W20260709 13:01:34.605901  5740 consensus_meta.cc:287] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.615s	user 0.017s	sys 0.004s
W20260709 13:01:34.621054  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.589854  5923 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:34.595664  5916 consensus_meta.cc:287] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.651s	user 0.007s	sys 0.007s
I20260709 13:01:34.622773  1541 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:34.511191  5909 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.514770  5899 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:34.621479  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:34.620537  1537 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:34.589313  5959 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:34.634825  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:34.626142  5842 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.580300  5907 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:34.621551  5961 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:34.653026  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:34.572363  5966 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 13:01:34.657897  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:34.658742  1684 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:34.627974  5927 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:34.634018  5922 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:34.660866  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:34.657897  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:34.662340  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:34.663044  1684 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:34.653815  5944 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:01:34.664232  5973 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 13:01:34.665014  5940 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:34.677580  5923 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:01:34.634586  5740 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.681890  5740 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:34.682866  5642 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:01:34.683276  5773 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.645s	user 0.013s	sys 0.003s
I20260709 13:01:34.666435  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:34.683511  1689 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
W20260709 13:01:34.671327  5978 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:01:34.686929  1688 raft_consensus.cc:2410] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 13:01:34.671746  5966 consensus_queue.cc:1243] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:34.686121  1684 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:01:34.691385  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 788) took 1307 ms. Trace:
I20260709 13:01:34.691797  1539 rpcz_store.cc:276] 0709 13:01:33.383751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.383911 (+   160us) service_pool.cc:224] Handling call
0709 13:01:34.691152 (+1307241us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540012ded4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:34.691293 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2802220416}
I20260709 13:01:34.693162  5730 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.697952  1687 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:34.671878  5844 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:34.655030  5842 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:34.675161  1540 raft_consensus.cc:2410] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:34.670262  5916 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:34.683297  5642 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:34.684217  5976 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:01:34.684506  5928 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:34.715463  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.715880  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 118880us
I20260709 13:01:34.722239  1687 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 13:01:34.722805  5929 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.711s	user 0.010s	sys 0.002s
I20260709 13:01:34.723639  5642 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.724942  5642 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:34.718407  5928 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.726786  5928 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:34.709887  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 88728.1us
I20260709 13:01:34.729738  5937 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:34.729597  5943 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
W20260709 13:01:34.734030  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 56257.1us
I20260709 13:01:34.659740  5933 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:34.703676  5856 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:34.777951  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:34.779445  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.784116  1617 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:34.783962  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:34.785780  1618 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:34.787842  1611 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: []
W20260709 13:01:34.786957  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 175250us
I20260709 13:01:34.788784  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:34.733707  5956 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:34.789908  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.793658  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.791636  1612 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:34.739173  5859 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:34.798117  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 48218.4us
I20260709 13:01:34.798810  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:34.799082  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 132702us
W20260709 13:01:34.778654  5951 consensus_meta.cc:287] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.770s	user 0.010s	sys 0.005s
I20260709 13:01:34.797400  5885 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.804973  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:34.805153  5900 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:34.805749  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:34.822436  2230 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:34.806748  5984 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:34.748152  5898 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:01:34.808990  1609 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:34.792361  5793 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.810454  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:34.837841  1614 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:34.797169  5906 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.789s	user 0.014s	sys 0.004s
I20260709 13:01:34.718303  5948 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:01:34.802253  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:34.808667  5983 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:01:34.824402  1685 raft_consensus.cc:2410] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:34.826785  5986 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:34.828346  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:34.828713  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 230608us
I20260709 13:01:34.893769  2231 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:34.830358  5933 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:34.925616  5982 log.cc:927] Time spent T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:01:34.926218  2230 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:01:34.832288  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:34.930016  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 97918.8us
I20260709 13:01:34.931844  2203 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:34.788107  5916 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.932425  2231 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:34.833411  1612 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:01:34.933760  1612 raft_consensus.cc:606] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
W20260709 13:01:34.934048  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 131909us
I20260709 13:01:34.934664  5763 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:34.935523  5763 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:34.936477  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:34.937709  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.939556  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:34.932747  5916 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.942349  5916 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:34.942907  5916 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:34.934141  1612 raft_consensus.cc:614] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:34.944330  5933 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:34.944754  5933 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:34.819125  5825 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:34.945081  5933 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:34.945501  5859 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:34.945915  5859 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:34.946302  5859 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:34.820796  5905 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.127s
I20260709 13:01:34.950344  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:34.818689  5856 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:34.953250  5995 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:34.823657  5959 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:34.954067  5995 consensus_queue.cc:1243] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 8a6a5ae1b7f240408b3ddd30d8a18262 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:34.953837  5856 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:34.955073  5986 raft_consensus.cc:3055] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:34.955500  5986 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:34.834362  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 274087us
W20260709 13:01:34.958194  1507 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.714s (SENT)
I20260709 13:01:34.784214  5837 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:34.861770  5904 raft_consensus.cc:3055] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:34.851882  5968 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:34.916109  1584 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:01:34.917191  5980 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 13:01:34.922966  5987 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:01:34.954675  5995 raft_consensus.cc:2764] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
W20260709 13:01:34.904701  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.936508  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 79312.4us
I20260709 13:01:34.958724  1507 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:34.957768  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:34.983697  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.964781  1614 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
W20260709 13:01:34.971918  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:34.991390  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:34.992218  1508 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:34.972798  5842 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:34.852726  5966 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:34.985525  2231 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:34.994674  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:34.894481  5951 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:34.976914  5946 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.923s	user 0.008s	sys 0.009s
I20260709 13:01:34.999470  2228 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 13:01:34.993393  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:35.001396  1507 leader_election.cc:336] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.741s (SENT)
I20260709 13:01:35.004733  1686 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:34.962008  5922 raft_consensus.cc:697] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:34.962919  5904 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:34.824047  5885 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.008883  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:35.022025  2229 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:35.016665  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:34.979488  5913 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:35.023751  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 87382.1us
I20260709 13:01:34.838424  5956 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.162s
I20260709 13:01:34.866755  5793 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.029096  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:34.998158  5837 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:35.036398  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:35.019799  5825 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:35.043671  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:35.047489  1542 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:35.049091  2229 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:35.027637  5340 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:35.052978  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:35.053220  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 51857.3us
I20260709 13:01:34.998643  5928 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.052507  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:35.054458  1507 leader_election.cc:336] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.777s (SENT)
I20260709 13:01:35.054886  1507 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:35.043834  5917 consensus_meta.cc:287] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.004s	user 0.011s	sys 0.011s
I20260709 13:01:35.037595  5909 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:35.057344  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:35.007588  5966 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:35.048632  5947 consensus_meta.cc:287] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.082s	user 0.015s	sys 0.003s
I20260709 13:01:35.064106  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:35.064678  2229 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.065306  2230 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:35.037356  5837 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:35.008241  5951 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:35.067631  5939 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.000s	sys 0.028s
W20260709 13:01:35.068270  5939 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.000s	sys 0.029s
I20260709 13:01:35.073408  1683 raft_consensus.cc:1217] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Deduplicated request from leader. Original: 10.10->[11.11-11.11]   Dedup: 11.11->[]
I20260709 13:01:35.074139  5965 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:34.956413  5986 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.078316  5986 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.067791  5994 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:35.028762  5995 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:01:35.074174  1683 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 13:01:35.089315  2231 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:35.092269  2233 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:35.080857  1507 leader_election.cc:336] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.953s (SENT)
I20260709 13:01:35.094305  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:35.079521  5934 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.029s	user 0.023s	sys 0.000s
I20260709 13:01:35.107173  2230 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:35.062528  5340 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:35.145259  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 87987.5us
I20260709 13:01:35.040935  5991 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.124s
I20260709 13:01:35.160374  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:35.060554  5959 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:35.089444  5837 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:35.155202  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 239160us
W20260709 13:01:35.155879  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.050489  6000 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 13:01:35.094002  5947 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:35.063213  5917 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:35.165549  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.199658  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:35.175822  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:35.097021  5994 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:35.202806  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:35.124117  5922 consensus_queue.cc:237] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.205698  5947 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:35.206382  5947 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:35.206887  5947 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:35.208066  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:35.209038  5960 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:35.162681  5948 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.224098  1690 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:35.197781  5903 consensus_meta.cc:287] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.102s	user 0.014s	sys 0.000s
I20260709 13:01:35.228217  6003 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:35.198421  5934 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:01:35.231278  6004 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:01:35.169112  5916 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:01:35.086279  5968 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.172379  5956 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:35.229862  5989 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:35.169718  5773 consensus_queue.cc:1059] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:01:35.248526  5773 raft_consensus.cc:3055] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:35.249028  5773 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:35.250283  5773 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.251982  5773 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.254292  5793 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:35.260164  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:35.261379  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1734) took 1042 ms. Trace:
I20260709 13:01:35.261803  1685 rpcz_store.cc:276] 0709 13:01:34.218919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.219397 (+   478us) service_pool.cc:224] Handling call
0709 13:01:35.261201 (+1041804us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b54000aafd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:35.261322 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1383053696}
I20260709 13:01:35.263075  5943 raft_consensus.cc:3055] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:35.263491  5943 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:35.237578  5971 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:35.237020  5340 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:35.238428  5985 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 13:01:35.264783  5971 raft_consensus.cc:697] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:35.238549  5994 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.209486  5960 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:35.266006  5960 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:35.266314  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 797) took 1509 ms. Trace:
I20260709 13:01:35.266682  1538 rpcz_store.cc:276] 0709 13:01:33.756469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:33.758328 (+  1859us) service_pool.cc:224] Handling call
0709 13:01:33.795104 (+ 36776us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:34.333564 (+538460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:34.333594 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:34.346630 (+ 13036us) log.cc:844] Serialized 52 byte log entry
0709 13:01:34.570982 (+224352us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:34.571040 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:34.571139 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:35.266065 (+694926us) raft_consensus.cc:1685] finished
0709 13:01:35.266086 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:35.266233 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25480,"raft.queue_time_us":65672,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":6335,"thread_start_us":642345,"threads_started":2,"wal-append.queue_time_us":521865}
I20260709 13:01:35.104235  5904 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.272814  5904 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.281668  2229 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
W20260709 13:01:35.241626  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 65908.3us
W20260709 13:01:35.227273  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 146450us
I20260709 13:01:35.172806  5844 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:01:35.293936  5844 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:35.294417  5844 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:35.295439  5885 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:35.295383  5844 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.296746  5844 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.296921  5951 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:35.245712  2234 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:35.210244  5940 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:01:35.292028  5927 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.004s
W20260709 13:01:35.299860  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.307020  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:35.307832  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:35.308035  1688 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:01:35.305251  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:35.308916  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:35.315723  1538 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:35.315955  1684 raft_consensus.cc:2410] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:01:35.316124  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.310765  1507 leader_election.cc:336] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.120s (SENT)
I20260709 13:01:35.317991  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:35.319868  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:35.323406  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:35.329022  1691 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:35.322219  5740 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:01:35.338719  5966 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:35.302237  5940 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:35.339509  5940 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:35.344591  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:35.293951  5917 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:35.309885  5929 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:35.346807  5929 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.348942  5929 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:35.326570  5927 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.352098  5927 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:35.339550  5953 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:35.298441  5825 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:01:35.358420  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 57206.6us
W20260709 13:01:35.359369  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 48660.8us
I20260709 13:01:35.362836  5971 consensus_queue.cc:237] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:35.365537  1507 leader_election.cc:336] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.206s (SENT)
W20260709 13:01:35.366655  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.367093  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.367472  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.369011  1507 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:35.369370  5960 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.406513  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.407112  5960 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:35.407295  5929 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:35.370186  1689 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:35.407992  5929 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:35.408195  5927 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.371742  5940 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.408874  5927 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.409914  1683 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:35.309741  5903 consensus_queue.cc:1059] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:01:35.412335  6002 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:35.412945  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 51421.3us
I20260709 13:01:35.413789  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:35.414067  5940 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:35.414572  6003 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:35.412977  6002 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.416574  5903 raft_consensus.cc:3055] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:35.416982  5903 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:35.417209  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:35.418629  6002 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:35.376237  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 859) took 1012 ms. Trace:
I20260709 13:01:35.302867  5909 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:01:35.419677  1690 rpcz_store.cc:276] 0709 13:01:34.363568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.386885 (+ 23317us) service_pool.cc:224] Handling call
0709 13:01:34.419486 (+ 32601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.233862 (+814376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:35.376055 (+142193us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b54000f6254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:35.376174 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1850039680}
I20260709 13:01:35.420231  5909 raft_consensus.cc:3055] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:35.420637  5909 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:35.421372  1609 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:35.421473  5909 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.422210  5854 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:35.423003  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:35.423334  5854 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:35.423672  1612 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:35.423774  5854 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:35.423992  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:35.357901  5953 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:35.424970  5953 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:35.235920  5948 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.396272  5916 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.225735  5933 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.465262  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:35.400190  1534 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:01:35.462644  1509 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:35.403448  1537 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
W20260709 13:01:35.405037  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.468596  1541 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:35.411628  5927 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:35.283015  5956 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.418819  6003 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:35.421715  6009 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:35.424562  1614 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.470353  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:35.428398  1538 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:01:35.424710  5854 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.430557  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:35.374914  5859 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.418646  1507 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.395043  5947 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.404824  5837 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:35.470120  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 52976.6us
I20260709 13:01:35.471973  5934 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:35.470942  5959 consensus_queue.cc:1059] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:01:35.462143  5940 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.473657  5948 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:35.477018  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:35.477666  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:35.479246  1689 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:35.466725  1537 pending_rounds.cc:85] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 8
I20260709 13:01:35.479451  1615 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:35.491575  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:35.515034  1535 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:35.494812  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:35.515591  1537 pending_rounds.cc:107] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 11.9
W20260709 13:01:35.502382  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 83805.5us
I20260709 13:01:35.508992  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:35.515884  1537 raft_consensus.cc:2889] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:35.499449  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:35.516535  1538 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:35.516525  1543 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:35.522501  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:35.523375  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:35.526273  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:35.527441  1689 raft_consensus.cc:2410] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:01:35.530675  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.531808  1689 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:35.540547  1657 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:35.542114  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:35.545471  1614 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:35.470558  5916 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.546446  1617 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:35.506613  5929 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:35.506649  5934 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.508904  5854 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:35.506593  5960 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:35.525049  5951 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:35.547465  5898 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 13:01:35.564993  1507 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.554678  5989 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:35.552707  5984 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:35.516999  5940 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.554385  5942 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:35.554011  5933 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:35.557844  5934 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.561784  5773 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:35.581457  2203 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.568887  5951 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.571184  5917 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.584700  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:35.586901  1614 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:35.572474  5960 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.564638  5993 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:01:35.590376  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:35.594297  1611 raft_consensus.cc:2376] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11. Re-sending same reply.
I20260709 13:01:35.538425  5903 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.577548  5989 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:35.579946  5984 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:35.600203  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:35.601363  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:35.585559  5959 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:35.575439  5965 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:01:35.602813  1611 raft_consensus.cc:2410] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 10.
I20260709 13:01:35.603477  1614 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:35.588889  5951 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.590528  5986 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:35.620733  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55819.2us
I20260709 13:01:35.671646  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.601707  5984 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:35.599781  5989 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:35.672367  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:35.678310  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 108640us
I20260709 13:01:35.590445  5917 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:35.674770  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 80867.8us
I20260709 13:01:35.678588  1614 raft_consensus.cc:2410] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:01:35.660396  5859 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:35.600211  5844 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 13:01:35.705904  1614 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:01:35.673262  5900 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:01:35.693280  5956 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:35.707928  6024 log.cc:927] Time spent T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:01:35.681756  5793 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:01:35.711223  6029 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:35.711741  6029 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:35.715965  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:35.701996  5859 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:35.704308  5844 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.742408  5909 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:35.739439  5937 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:35.701306  5885 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:01:35.712826  6029 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.705513  5837 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:35.746121  1507 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:35.749068  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.716101  5916 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:35.714273  5960 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:35.748668  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 66242.3us
I20260709 13:01:35.752763  2203 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:35.739848  5859 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:35.705005  5961 consensus_queue.cc:1059] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:01:35.761909  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:35.765295  6027 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:01:35.765198  5859 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.766774  5859 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.758566  5905 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:01:35.771314  5940 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:35.763597  6029 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.751830  5959 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:35.781154  5959 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.781605  5959 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:35.785985  5961 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:35.786489  5961 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:35.787307  5961 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.788489  5961 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:35.769786  6026 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:01:35.760824  5937 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:01:35.792549  1611 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:01:35.798331  1543 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:35.799577  1507 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.802131  1618 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:35.752336  6030 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:35.803256  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 299036us
I20260709 13:01:35.811486  5825 raft_consensus.cc:3055] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:01:35.811426  6030 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.811998  5825 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:35.804207  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 293889us
I20260709 13:01:35.813323  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:35.813813  6030 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:35.818140  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:35.742722  5844 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.806627  1614 raft_consensus.cc:606] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.811743  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.803288  6000 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:35.824623  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:35.825307  1535 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:35.759239  5837 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:35.826419  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:35.803900  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.822834  1541 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:35.822376  5733 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:35.839502  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:35.836480  1683 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:35.821161  5909 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:35.825702  1684 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:01:35.855362  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.852150  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:35.857931  1683 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:35.861727  1684 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:01:35.865979  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:35.867648  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:35.870419  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:35.852051  5907 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.169s	user 0.009s	sys 0.013s
I20260709 13:01:35.869848  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:35.869153  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:35.812923  5825 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.839653  5942 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:01:35.841970  5837 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.841665  5733 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.859349  5929 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.853680  5937 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:35.884096  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.891729  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:35.854125  5966 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.853545  6010 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:35.885492  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:35.887311  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:35.878712  5907 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.199s	user 0.009s	sys 0.013s
I20260709 13:01:35.881891  1684 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:01:35.880136  5900 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:35.887133  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:35.864041  6009 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:35.833580  6002 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:01:35.893957  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:35.859045  5993 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
W20260709 13:01:35.933948  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:35.936856  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 69340.5us
I20260709 13:01:35.938818  2203 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.896917  1684 pending_rounds.cc:85] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 8
I20260709 13:01:35.940115  1684 pending_rounds.cc:107] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:01:35.940431  1684 raft_consensus.cc:2889] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:35.941857  1613 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 7. Current term is 8. Ops: []
I20260709 13:01:35.943519  1684 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:01:35.888435  5944 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.212s	user 0.021s	sys 0.016s
I20260709 13:01:35.898941  1584 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:35.948472  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:35.949479  2229 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:35.955510  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:35.965183  2230 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:35.937386  5933 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:35.965936  5933 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:35.890143  5733 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:35.887531  5837 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:35.973242  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 97224.8us
I20260709 13:01:35.966389  5933 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:35.945928  5854 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:35.954663  5825 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:01:35.945044  5944 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.268s	user 0.023s	sys 0.018s
I20260709 13:01:35.937125  5966 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.942008  6010 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:35.895424  5929 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.978307  5989 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:35.942798  5900 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:35.982649  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 83795us
I20260709 13:01:35.982287  1683 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:35.980731  6009 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:35.899504  5953 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:35.980180  5944 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:35.988320  6033 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:36.020023  6043 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:36.020741  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:35.998677  6030 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.024698  1692 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:36.033052  6036 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:36.034298  6036 consensus_queue.cc:1059] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:36.036347  6054 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:36.040771  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.044548  6035 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:01:36.047086  5959 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:36.047760  5953 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:36.027038  6038 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:01:36.011765  5844 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:36.009721  6009 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:36.056905  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:36.051100  6042 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
W20260709 13:01:36.197219  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 176577us
I20260709 13:01:36.130908  6000 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:36.204800  1657 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:36.049821  6030 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.179021  5959 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:36.238128  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:36.215368  6038 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:01:36.256402  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:36.233661  5844 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:36.235802  5989 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:36.255000  5984 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.003s
W20260709 13:01:36.306404  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 303341us
I20260709 13:01:36.319881  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:36.235219  6000 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.051225  6010 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:36.321969  5929 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:36.335680  5900 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:36.335685  6055 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 13:01:36.346035  5859 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.003s
I20260709 13:01:36.233397  5944 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.346030  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:36.350093  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.315094  5984 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.318051  6009 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.320178  5966 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:36.343614  6030 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:36.357082  2237 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:36.350772  6033 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:36.367084  5951 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.697s	user 0.011s	sys 0.000s
I20260709 13:01:36.369624  6000 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:36.373936  5899 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:36.379019  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:36.385710  5934 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.787s	user 0.011s	sys 0.002s
I20260709 13:01:36.394713  1509 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:36.410778  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.414870  5933 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.416283  5837 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.396010  5966 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.388644  5951 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:36.418329  6049 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 13:01:36.398515  5929 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:36.434006  5929 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:36.435034  5929 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.439754  5984 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:36.440668  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.443578  5947 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:36.444569  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:36.446578  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.448263  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.457623  1690 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:36.396093  5934 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:36.502993  5773 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:36.503748  5773 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:36.506384  5773 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:36.483175  5837 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:36.484417  1540 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:36.507025  5928 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:36.507571  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 806) took 1108 ms. Trace:
I20260709 13:01:36.508093  1540 rpcz_store.cc:276] 0709 13:01:35.398774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.402545 (+  3771us) service_pool.cc:224] Handling call
0709 13:01:36.483876 (+1081331us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b5800003998. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:01:36.484160 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:36.507223 (+ 23063us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.507496 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2450816}
I20260709 13:01:36.485569  5951 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:36.509598  5961 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.721s	user 0.009s	sys 0.006s
I20260709 13:01:36.489144  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1757) took 1068 ms. Trace:
I20260709 13:01:36.423377  6033 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.510563  5903 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:36.510174  1617 rpcz_store.cc:276] 0709 13:01:35.420361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.423312 (+  2951us) service_pool.cc:224] Handling call
0709 13:01:35.423596 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.554771 (+131175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.488738 (+933967us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54001b8854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:36.489010 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282084736}
I20260709 13:01:36.512176  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.490861  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2646) took 1056 ms. Trace:
W20260709 13:01:36.492388  5733 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.005s
I20260709 13:01:36.493263  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.499262  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:36.518510  1616 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:36.500344  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:36.501515  5825 consensus_meta.cc:287] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.000s
I20260709 13:01:36.449944  5966 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:36.487757  6032 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.739s	user 0.008s	sys 0.004s
I20260709 13:01:36.513005  1617 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:36.515770  1692 rpcz_store.cc:276] 0709 13:01:35.434513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.753088 (+318575us) service_pool.cc:224] Handling call
0709 13:01:35.763928 (+ 10840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:36.025363 (+261435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.490513 (+465150us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54000aaad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:36.490773 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":599106816}
W20260709 13:01:36.524807  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.516000  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.516942  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:36.519394  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:36.529124  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:36.517447  5951 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.538084  1687 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:36.541617  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.546629  1658 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:36.551370  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:36.556627  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:36.582309  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.583223  1609 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:36.584712  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:36.556411  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:36.587827  1683 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:36.523932  6009 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:36.588980  1684 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:36.569103  1614 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:36.589876  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:36.591095  1684 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:36.592839  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.547749  5947 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:36.594130  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:36.595780  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:36.592217  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:36.597795  1692 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:36.548761  5933 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:36.581514  5859 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:36.584311  5989 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.583812  5953 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:36.554376  5934 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:36.522123  5733 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.549181  6030 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:36.545046  5773 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:36.602098  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.602715  1539 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: [11.12-11.12]
I20260709 13:01:36.548383  5951 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:36.646154  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 52084.8us
I20260709 13:01:36.646956  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:36.648586  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:36.649266  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.649786  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:36.650432  1614 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:36.651549  5947 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:36.652132  5947 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:36.653106  5947 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:36.656435  5953 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:36.656960  5953 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.629544  5340 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.597409  6009 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.661631  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.661849  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:36.662434  5340 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:36.679060  5966 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:36.679694  5340 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:36.681023  6009 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:36.683039  5842 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:36.657343  5953 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:36.667084  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 810) took 1171 ms. Trace:
I20260709 13:01:36.672072  5773 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:36.675732  2234 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1212) took 1681 ms (client timeout 1573 ms). Trace:
I20260709 13:01:36.640962  5733 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:36.658885  5859 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:36.680922  1615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 725) took 1243 ms (client timeout 1630 ms). Trace:
I20260709 13:01:36.670768  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1746) took 1925 ms. Trace:
I20260709 13:01:36.690438  1534 rpcz_store.cc:276] 0709 13:01:35.495735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.495878 (+   143us) service_pool.cc:224] Handling call
0709 13:01:36.666804 (+1170926us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b54001e5854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:36.666998 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2155571968}
I20260709 13:01:36.676347  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:36.693672  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:36.693861  1616 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
W20260709 13:01:36.692945  2234 rpcz_store.cc:269] 0709 13:01:34.994600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.994715 (+   115us) service_pool.cc:224] Handling call
0709 13:01:36.675459 (+1680744us) spinlock_profiling.cc:243] Waited 21.4 ms on lock 0x7b5400648a54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:36.675618 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49277696}
W20260709 13:01:36.695494  1615 rpcz_store.cc:269] 0709 13:01:35.437411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.464987 (+ 27576us) service_pool.cc:224] Handling call
0709 13:01:36.680654 (+1215667us) spinlock_profiling.cc:243] Waited 6.58 ms on lock 0x7b5400190854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:36.680807 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15126400}
I20260709 13:01:36.695685  1613 rpcz_store.cc:276] 0709 13:01:34.745776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.775368 (+ 29592us) service_pool.cc:224] Handling call
0709 13:01:34.797072 (+ 21704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.942351 (+1145279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.670415 (+728064us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b540008b6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:36.670682 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2632231168}
I20260709 13:01:36.697777  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:36.698385  1610 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:36.698266  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.613587  5989 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:36.699287  1616 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:36.696676  5773 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.701241  1614 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:01:36.702479  1614 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:36.702631  5773 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:36.703266  5773 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:36.703577  1614 raft_consensus.cc:1217] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 9.5->[11.6-11.6]   Dedup: 11.6->[]
I20260709 13:01:36.705578  1614 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:01:36.707509  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.712492  1614 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:36.706768  5842 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.714073  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:36.715021  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.715978  1613 raft_consensus.cc:2393] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:36.696808  5859 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.728228  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:36.729200  1534 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:36.650933  6030 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.719261  5773 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:36.740257  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2645) took 1302 ms. Trace:
I20260709 13:01:36.731721  5859 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:36.732673  5989 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:36.741211  1689 rpcz_store.cc:276] 0709 13:01:35.433797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.533915 (+100118us) service_pool.cc:224] Handling call
0709 13:01:35.534272 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.534540 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:35.559070 (+ 24530us) log.cc:844] Serialized 14 byte log entry
0709 13:01:36.679044 (+1119974us) spinlock_profiling.cc:243] Waited 2.92 ms on lock 0x7b180038ed20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c979c0a 00007fe0960b9219 00007fe08ca01a40 00007fe08c9fc4e4 00007fe08c9fc455 00007fe0934e43c7 00007fe08faf69a7 00007fe08fb060bf 00007fe08fb06038
0709 13:01:36.679303 (+   259us) log.cc:844] Serialized 14 byte log entry
0709 13:01:36.735406 (+ 56103us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:36.735479 (+    73us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:36.735552 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.735641 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.735805 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1113503,"prepare.queue_time_us":1149637,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":77,"raft.queue_time_us":8759,"spinlock_wait_cycles":6720256,"thread_start_us":52603,"threads_started":1,"wal-append.queue_time_us":61619}
W20260709 13:01:36.750226  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 56653.1us
I20260709 13:01:36.753611  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:36.727790  5842 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:36.757694  1658 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:36.760524  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2638) took 1454 ms. Trace:
I20260709 13:01:36.759850  1609 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:36.767496  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.766283  5773 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:36.767426  1534 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:36.773795  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:36.772547  5842 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:36.769954  1685 rpcz_store.cc:276] 0709 13:01:35.305715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.305957 (+   242us) service_pool.cc:224] Handling call
0709 13:01:35.306363 (+   406us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:35.323567 (+ 17204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:35.323598 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:35.358549 (+ 34951us) log.cc:844] Serialized 52 byte log entry
0709 13:01:35.561802 (+203253us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:36.754536 (+1192734us) log.cc:844] Serialized 14 byte log entry
0709 13:01:36.758912 (+  4376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.759026 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:36.760207 (+  1181us) raft_consensus.cc:1685] finished
0709 13:01:36.760227 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:36.760447 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1121269,"prepare.queue_time_us":1119757,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":10208,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":9612,"thread_start_us":270742,"threads_started":2,"wal-append.queue_time_us":308061}
I20260709 13:01:36.787340  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:36.769243  5844 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:36.786759  6070 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:36.785830  5773 raft_consensus.cc:697] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:36.806427  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:36.807265  1534 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:36.810827  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.798461  5951 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:36.817718  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:36.818447  1542 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:36.819314  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:36.794055  5989 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:01:36.821117  1610 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:36.819913  1542 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:36.823732  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:36.824887  1542 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:36.825335  1534 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:36.825706  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:36.826625  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:36.827659  1534 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:36.826545  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:36.830646  5773 consensus_queue.cc:237] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:36.839947  1543 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:36.840912  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:36.837558  5859 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:36.838294  5909 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:36.838934  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:36.856557  1542 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:36.841696  1543 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
W20260709 13:01:36.827337  6074 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:01:36.861267  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:36.858994  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:36.883311  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2171) took 1444 ms. Trace:
I20260709 13:01:36.883826  1539 rpcz_store.cc:276] 0709 13:01:35.438515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.438660 (+   145us) service_pool.cc:224] Handling call
0709 13:01:35.439100 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:36.645671 (+1206571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.882988 (+237317us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540012ded4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:36.883235 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2672191872}
I20260709 13:01:36.885118  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:36.886823  5917 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:36.888796  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:36.887076  5859 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:36.889991  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:36.856026  5909 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:36.893388  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:36.899457  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:36.900563  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:36.901314  1543 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:36.902938  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:36.902415  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:36.897564  5951 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:36.907277  5951 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:36.907770  5951 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:36.909519  6075 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:36.924738  6075 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.904846  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:36.856671  5953 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:36.898978  6088 log.cc:927] Time spent T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 13:01:36.927126  5953 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:36.928391  6072 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.134s	user 0.001s	sys 0.001s
I20260709 13:01:36.928651  5953 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:36.929272  5953 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.929769  5953 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:36.928436  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:36.874346  5989 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:36.911767  5943 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:36.893651  1542 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
W20260709 13:01:36.913760  6039 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.930s	user 0.004s	sys 0.007s
I20260709 13:01:36.916033  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1221) took 1051 ms. Trace:
I20260709 13:01:36.922951  5909 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:36.892140  5917 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:36.905932  5947 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:36.905817  6076 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:36.925180  6075 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:36.944020  1543 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:36.944125  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:36.943753  2237 rpcz_store.cc:276] 0709 13:01:35.864371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.868651 (+  4280us) service_pool.cc:224] Handling call
0709 13:01:35.883083 (+ 14432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:36.359014 (+475931us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:36.915753 (+556739us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b54007404d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:36.915954 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1089587072}
I20260709 13:01:36.950178  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:36.952606  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:36.940595  5773 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:01:36.973917  1618 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:36.974387  1610 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:36.946197  5917 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:36.984596  1610 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:36.945567  5989 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:36.987790  5953 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:36.989760  5989 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:36.990535  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:36.992921  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:36.993897  5997 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:36.994318  5997 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:36.994972  5953 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:36.995193  5997 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:36.996034  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:36.997977  6083 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:01:36.976886  5909 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:36.977288  5943 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:36.996762  5997 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:37.002890  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:37.010483  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:37.010767  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:37.011513  1688 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:37.014010  1610 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:37.011425  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:36.978789  5773 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:37.011793  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:36.981169  5917 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:37.012516  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:37.014919  1685 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:37.012369  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:37.020999  1686 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:37.021622  1691 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:37.018957  1689 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:37.022622  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:37.025607  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:37.025771  6076 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:37.053637  6009 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:37.028551  1618 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:37.055924  5859 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.030215  5942 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:37.056598  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1071) took 1183 ms. Trace:
I20260709 13:01:37.056751  5942 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:37.031735  6087 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:01:37.023890  5989 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.056488  5859 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.052009  5947 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:37.058007  5991 raft_consensus.cc:3055] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:37.058450  5991 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:37.058161  5947 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.059273  5859 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.059345  5991 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.060299  5947 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.060537  5991 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:37.061220  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 742222us
I20260709 13:01:37.020547  5773 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:01:37.063264  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:37.064913  1691 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:37.070955  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:37.024096  5903 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:37.019078  5842 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:37.057057  2236 rpcz_store.cc:276] 0709 13:01:35.872941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.873090 (+   149us) service_pool.cc:224] Handling call
0709 13:01:35.882865 (+  9775us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:35.883100 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:35.973513 (+ 90413us) log.cc:844] Serialized 14 byte log entry
0709 13:01:36.257073 (+283560us) log.cc:844] Serialized 14 byte log entry
0709 13:01:36.434963 (+177890us) spinlock_profiling.cc:243] Waited 2.56 ms on lock 0x7b18004a1ca0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:01:37.056105 (+621142us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b18004a1ca0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c979c0a 00007fe0960b9219 00007fe08ca01a40 00007fe08c9fc4e4 00007fe08c9fc455 00007fe08fae4060 00007fe08faf6716 00007fe08fb07287 00007fe08fb07208
0709 13:01:37.056258 (+   153us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:37.056309 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:37.056358 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.056421 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.056520 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":792667,"prepare.queue_time_us":43675,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":3079,"raft.queue_time_us":804730,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":9717,"spinlock_wait_cycles":12415104,"thread_start_us":360445,"threads_started":2,"wal-append.queue_time_us":351645}
I20260709 13:01:37.079259  1583 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:37.080482  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.019322  5962 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.057200  5942 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:37.018280  5953 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:37.084789  1583 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:37.084887  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:37.090380  1691 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:37.087285  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:37.092048  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.093621  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:37.074889  5968 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:37.098944  1583 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:37.106705  1538 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:01:37.060088  5989 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.138010  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:37.138970  2234 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:37.139707  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:37.135739  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:37.140962  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:37.146802  1691 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:37.148854  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:37.149277  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:37.149865  2237 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:37.108060  6059 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:01:37.087788  6071 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:37.096342  5966 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.108723  6010 raft_consensus.cc:2764] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:01:37.158915  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:37.159646  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:37.108362  5934 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:37.111374  5928 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:37.162657  1684 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:37.167014  1539 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:37.097090  5578 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:01:37.090108  5962 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.109042  5968 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:37.094959  5953 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.135716  5986 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 13:01:37.134944  5984 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:01:37.228910  2237 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:37.094964  5842 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:37.230297  1541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 809) took 1736 ms (client timeout 2048 ms). Trace:
I20260709 13:01:37.153465  5793 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:37.263656  1541 rpcz_store.cc:269] 0709 13:01:35.494331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.494491 (+   160us) service_pool.cc:224] Handling call
0709 13:01:37.216157 (+1721666us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54003458d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:37.230233 (+ 14076us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706117504}
I20260709 13:01:37.158124  5966 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.159663  5971 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:37.238427  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:37.269214  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:37.162878  5947 consensus_queue.cc:1059] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
W20260709 13:01:37.272614  6091 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:01:37.265744  6036 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:37.273501  6036 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:37.273605  6090 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 13:01:37.273613  5947 raft_consensus.cc:3055] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:37.274235  5947 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:37.274822  1618 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:37.275199  5947 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.276262  5962 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.276731  5947 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:37.262760  6051 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.278287  6051 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.280763  5842 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:37.281406  5793 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:37.272526  6092 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:01:37.283998  5966 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.287400  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:37.254880  5934 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.257273  2232 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:37.289109  1611 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:01:37.259835  2228 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:37.290040  1611 pending_rounds.cc:85] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 7
I20260709 13:01:37.260048  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:37.290316  1611 pending_rounds.cc:107] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:01:37.290586  1611 raft_consensus.cc:2889] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:37.268693  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:37.269994  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:37.270792  2229 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:37.281558  5971 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.293375  5951 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:37.248518  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:37.240056  1538 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:37.303638  2232 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:37.260833  5968 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:37.310511  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:37.310864  5953 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.261407  5900 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:37.311378  2237 raft_consensus.cc:2368] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11.
I20260709 13:01:37.311640  5900 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.313912  5900 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:37.314124  6082 log.cc:927] Time spent T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.422s	user 0.002s	sys 0.000s
I20260709 13:01:37.238123  6076 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:37.314864  6036 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.316059  6051 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:37.319660  5934 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.320448  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:37.321352  1691 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:37.322304  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:37.324019  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.327782  1543 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:37.325358  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:37.330277  1618 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:37.269560  5859 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.331885  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:37.333340  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:37.337673  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:37.348925  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:37.353786  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:37.355010  6075 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:37.357812  6094 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:01:37.322943  5971 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:37.370467  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.369328  1685 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:37.328821  6054 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:37.380271  2203 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:37.336087  6033 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.381886  6075 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:37.335947  5934 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:37.407820  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68509.4us
I20260709 13:01:37.406603  6053 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:37.347031  6095 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:37.405622  5900 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:37.337539  5958 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:37.347290  5943 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:37.404863  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:37.362424  6103 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:37.336370  5859 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:37.407640  5951 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:37.404383  5898 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.373631  5953 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:01:37.487043  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 78250.8us
I20260709 13:01:37.422211  1542 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:37.412968  6053 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:37.409550  6076 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:37.490114  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:37.409603  5934 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:37.488579  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 70083.9us
I20260709 13:01:37.497041  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:37.497270  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:37.491108  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.492841  1543 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:37.497406  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:37.416699  6075 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:37.498338  1610 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:37.493616  2201 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:37.416512  5947 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:37.422775  5971 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:37.487176  5933 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:37.407401  6054 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.410635  6033 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.422042  5958 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.506088  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:37.512091  1617 raft_consensus.cc:3055] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:37.513329  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:37.505913  1543 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
W20260709 13:01:37.514333  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.521281  1617 raft_consensus.cc:740] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:37.528267  1543 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:01:37.528822  1543 pending_rounds.cc:85] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 8
I20260709 13:01:37.498944  5934 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:37.529078  1543 pending_rounds.cc:107] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:01:37.529403  1543 raft_consensus.cc:2889] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:37.487615  5943 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:37.540992  1584 leader_election.cc:336] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.541s (SENT)
I20260709 13:01:37.541611  1584 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:37.542668  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:37.544274  1685 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:37.501606  5966 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:37.552392  1689 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:37.583261  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:37.586880  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45925.1us
I20260709 13:01:37.512006  6109 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.505861  6052 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:37.513062  5900 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:37.597937  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:37.559180  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:37.424496  6095 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.499678  5773 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:37.514642  6030 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.603672  1687 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:37.498142  5953 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.494114  5898 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.489580  6103 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.604817  1617 consensus_queue.cc:260] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:37.614599  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:37.628808  1542 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:37.629024  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:37.507737  5947 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.619899  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:37.597210  5907 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:37.584331  5733 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:37.640348  1685 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:37.640465  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:37.622336  1584 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.623s (SENT) [suppressed 54 similar messages]
I20260709 13:01:37.642273  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:37.618866  5900 raft_consensus.cc:697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:37.645875  1692 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:37.504005  6076 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.618782  6052 raft_consensus.cc:697] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:37.646953  1617 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:37.647673  6102 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 13:01:37.647971  2228 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:01:37.608731  1582 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:37.535578  5922 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 13:01:37.505107  6075 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.704759  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1761) took 1148 ms. Trace:
W20260709 13:01:37.705118  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 158127us
I20260709 13:01:37.610059  6110 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.515467  5971 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.751452  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:37.752942  1686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.005s	sys 0.005s
W20260709 13:01:37.753320  1686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.005s	sys 0.005s
I20260709 13:01:37.709573  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:37.742177  1683 rpcz_store.cc:276] 0709 13:01:36.556555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.589981 (+ 33426us) service_pool.cc:224] Handling call
0709 13:01:36.590398 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:36.786220 (+195822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:36.786257 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:36.791165 (+  4908us) log.cc:844] Serialized 26 byte log entry
0709 13:01:36.937959 (+146794us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:36.954568 (+ 16609us) log.cc:844] Serialized 14 byte log entry
0709 13:01:36.954866 (+   298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.647831 (+692965us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b1800443d40. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9cd2a6 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419
0709 13:01:37.704292 (+ 56461us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b540014a854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:37.704387 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:37.704452 (+    65us) raft_consensus.cc:1685] finished
0709 13:01:37.704474 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:37.704678 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":14841,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":13316,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":6448,"spinlock_wait_cycles":234080768,"thread_start_us":215301,"threads_started":2,"wal-append.queue_time_us":217963}
I20260709 13:01:37.701815  1687 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:37.627653  5951 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:37.624963  6030 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:37.550263  5962 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.115s
I20260709 13:01:37.698817  6010 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:37.763314  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:37.763969  1538 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:01:37.763906  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:37.764652  1541 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:37.765839  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:37.698659  5842 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:37.723837  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 210593us
I20260709 13:01:37.739092  2228 raft_consensus.cc:606] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:01:37.619369  6071 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.108s
W20260709 13:01:37.747926  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 139564us
I20260709 13:01:37.723734  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:37.717588  6099 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.710434  5933 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:37.807984  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:37.809669  2233 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:37.715899  5907 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:37.754412  6009 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:37.610983  6109 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:37.785202  1692 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:37.790261  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:37.791222  2203 leader_election.cc:336] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.630s (SENT)
I20260709 13:01:37.789075  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:37.790030  1657 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:37.741094  5922 consensus_queue.cc:1243] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:01:37.824052  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:37.761035  5951 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.830582  2203 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:37.835150  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 192837us
W20260709 13:01:37.838498  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 216095us
I20260709 13:01:37.840211  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:37.889815  1691 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:01:37.828188  6071 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:37.781005  1538 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:37.825562  6030 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.898944  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:37.838176  5951 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:37.895730  5966 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:37.798601  6010 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.897967  6071 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:37.837805  5859 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:37.815332  6099 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.901181  6120 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:37.936391  6120 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:37.798950  5842 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.937278  6120 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.939164  5842 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.940114  6122 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:37.940655  6122 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:37.941089  6122 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:37.879707  5900 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:37.937134  6099 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.903544  1539 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:37.948071  1541 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
W20260709 13:01:37.894358  6111 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.403s	user 0.000s	sys 0.002s
I20260709 13:01:37.952409  1617 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:01:37.922806  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 131590us
W20260709 13:01:37.957981  1690 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2647) took 2478 ms (client timeout 1965 ms). Trace:
W20260709 13:01:37.958534  1690 rpcz_store.cc:269] 0709 13:01:35.474131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.504950 (+ 30819us) service_pool.cc:224] Handling call
0709 13:01:37.951821 (+2446871us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b54000749d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:37.951979 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2148007168}
I20260709 13:01:37.959882  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 731) took 2442 ms. Trace:
I20260709 13:01:37.960566  1611 rpcz_store.cc:276] 0709 13:01:35.509990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:35.801543 (+291553us) service_pool.cc:224] Handling call
0709 13:01:35.801867 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.291240 (+1489373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:37.952405 (+661165us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54003c96d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:37.952579 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3419470208}
I20260709 13:01:37.961642  1541 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:37.768187  6110 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:37.918498  5909 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.916s	user 0.006s	sys 0.006s
I20260709 13:01:37.929677  6002 raft_consensus.cc:993] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 984c9089d9804652846f3caeb609d56e to start an election
I20260709 13:01:37.934571  5859 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:37.912300  5958 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:37.907341  5947 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.012567  1689 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:37.930760  5793 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:37.908381  6109 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:37.958105  5991 consensus_meta.cc:287] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.897s	user 0.005s	sys 0.009s
I20260709 13:01:37.914084  5844 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:37.911140  5953 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:37.930681  5917 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.886s	user 0.009s	sys 0.001s
W20260709 13:01:37.959880  6041 log.cc:927] Time spent T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.957s	user 0.000s	sys 0.016s
I20260709 13:01:37.915110  6076 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.933051  6033 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:37.909958  6070 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:37.926735  6103 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.996546  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:37.983948  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:37.933543  6095 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.935163  6054 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:37.931763  6010 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.922254  5773 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:37.908003  5898 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:37.911960  5942 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.816s	user 0.008s	sys 0.003s
I20260709 13:01:37.931646  6051 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:37.933858  5943 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:38.019726  5968 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.688s	user 0.007s	sys 0.002s
I20260709 13:01:37.995746  5859 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.021653  1689 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:38.024148  6053 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.032310  6036 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:38.035760  6002 raft_consensus.cc:1081] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Signalling peer 984c9089d9804652846f3caeb609d56e to start an election
I20260709 13:01:38.038869  6075 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.045847  5909 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:37.933286  6030 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.068333  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:38.060916  6107 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.450s	user 0.002s	sys 0.000s
I20260709 13:01:38.142942  5909 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:38.068843  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 278752us
I20260709 13:01:38.146880  1508 leader_election.cc:400] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11.
I20260709 13:01:38.147217  1508 leader_election.cc:403] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:38.069162  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:38.148059  1540 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:38.147795  5903 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:38.148414  1508 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:38.148980  5903 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11.
I20260709 13:01:38.149806  5903 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:38.150195  5903 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:38.151190  5903 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.152784  5903 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:38.153612  5340 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:37.908557  6052 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.155905  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:38.157584  5642 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:38.062793  5951 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.104161  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:38.158416  1541 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:38.159648  5951 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:37.924646  5966 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.159840  5928 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:01:38.160867  1536 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:38.161413  6010 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:38.162596  5842 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:38.107992  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2224) took 1603 ms. Trace:
I20260709 13:01:38.162953  5842 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:38.163233  1609 rpcz_store.cc:276] 0709 13:01:36.504730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:36.707150 (+202420us) service_pool.cc:224] Handling call
0709 13:01:38.107796 (+1400646us) spinlock_profiling.cc:243] Waited 51.1 ms on lock 0x7b54003499d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:38.107935 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117484800}
I20260709 13:01:38.108346  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:38.108633  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 268452us
I20260709 13:01:38.163825  5842 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.067236  5844 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:38.164933  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:38.176743  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:38.071679  6070 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:38.128293  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:37.933094  6071 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.064322  5917 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.131716  5859 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.100231  6116 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.108067  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:38.213461  1538 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:38.164815  1535 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:38.165568  5844 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.068722  5989 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:38.167449  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.138084  1534 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:38.215739  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:38.216190  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:38.198026  6071 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:38.198148  5943 consensus_queue.cc:1175] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "83f6f5810950417195f707ff308a8125" responder_term: 9 status { last_received { term: 7 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: [8.6-8.6]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:01:38.126541  5942 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.200896  5933 raft_consensus.cc:3055] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:38.206231  1540 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:38.208427  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:38.227519  1690 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:38.228730  1690 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:38.222234  5642 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:38.131562  5968 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.174913  6053 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:38.220906  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:38.234829  1684 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:38.200232  5917 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:38.219947  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:38.187990  5951 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:38.226049  1683 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:38.212503  6103 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:38.211090  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.215409  6075 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.214995  5909 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.190107  6118 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.064s
W20260709 13:01:38.235526  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 438061us
I20260709 13:01:38.227278  5907 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:38.215198  5773 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:01:38.235481  1542 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:38.218745  6095 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 13:01:38.226805  5933 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:38.231586  6030 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.280525  5933 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.282678  5933 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.222189  5953 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:01:38.295051  1582 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:38.282788  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.276887  6125 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:38.298132  6110 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.298516  6125 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.244480  5898 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.299096  6125 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:38.220046  6116 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.308112  2235 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:38.309075  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:38.310034  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:38.310211  1539 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:38.314057  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.315111  2231 raft_consensus.cc:2410] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:38.318168  2201 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:38.290568  5951 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:38.323031  2234 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:01:38.319767  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.290463  6129 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:38.330142  5986 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:38.330631  5986 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.330984  5986 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.322084  2233 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:01:38.330195  6129 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.307533  5898 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.325310  1658 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:38.329468  2230 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:38.343832  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:38.346635  1686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.010s	sys 0.014s
W20260709 13:01:38.348551  1686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.011s	sys 0.015s
I20260709 13:01:38.340699  2201 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:38.352069  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:38.290031  6075 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.312031  5943 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
W20260709 13:01:38.363629  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 225156us
W20260709 13:01:38.370313  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 413562us
I20260709 13:01:38.374914  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:38.362725  6009 consensus_meta.cc:287] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.000s
I20260709 13:01:38.383348  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.381659  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.351414  6110 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.352022  6030 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:38.362085  6103 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:38.387153  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.387292  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.340508  5773 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:01:38.388664  2230 pending_rounds.cc:85] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 8
I20260709 13:01:38.397342  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.397753  2230 pending_rounds.cc:107] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:01:38.398219  2230 raft_consensus.cc:2889] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:38.370365  5900 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:38.366015  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:38.372687  6002 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:01:38.393590  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 83665.8us
I20260709 13:01:38.365026  6128 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:38.396646  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.396752  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.376636  5917 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:38.403463  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:38.395239  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.406980  1582 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:38.378715  5968 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.409623  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.378826  6057 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:38.422818  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:38.414444  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.416333  1539 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:38.417820  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.425755  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.418342  2232 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:38.419322  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.422108  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.427603  1611 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:38.422778  1612 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:38.422773  6132 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:38.405300  6009 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:38.428177  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:38.430581  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.431726  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.390395  6129 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.433364  1685 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:38.433596  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:38.440884  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.394320  6030 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.405723  6002 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.405687  5922 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.439466  1618 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:38.451112  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.432701  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.405185  5900 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:38.451982  1614 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:38.415170  5953 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:38.447975  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.456408  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:38.458969  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:38.459661  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:38.466472  1615 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.466809  1535 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:38.460225  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:38.467975  1538 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:38.469161  1538 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:38.462951  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:38.470320  1541 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:38.471434  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:38.472556  1539 raft_consensus.cc:2393] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:38.473734  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:38.474545  1613 raft_consensus.cc:2393] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 8: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:38.474431  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.475216  1612 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:38.475971  1615 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:38.480154  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.480620  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:38.480890  1614 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:38.482167  6120 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.482710  6120 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.484845  6120 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.464150  5989 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:38.399843  5951 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.450975  6002 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:38.473829  5922 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.492707  1610 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:38.493736  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:38.494035  5922 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:38.494825  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.465536  5900 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:38.497743  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:38.498488  1537 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:38.437677  6132 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.436755  5942 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:38.503225  1610 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:38.434991  5917 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:38.531456  1615 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:01:38.501929  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:38.446456  5968 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.443573  6030 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.513780  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.406177  6110 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.423710  6122 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.477240  6009 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:38.516479  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.517472  1583 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:38.537941  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.533658  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:38.451013  5909 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:38.539144  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:38.542078  1613 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:38.467298  6129 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:38.552625  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:38.553252  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.553994  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:38.555398  5991 raft_consensus.cc:993] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: : Instructing follower 984c9089d9804652846f3caeb609d56e to start an election
W20260709 13:01:38.555622  1643 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:36375 (1 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:36375 timed out after 15.000s (SENT)
I20260709 13:01:38.556002  5991 raft_consensus.cc:1081] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Signalling peer 984c9089d9804652846f3caeb609d56e to start an election
I20260709 13:01:38.556434  5793 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:38.557320  5793 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:38.557610  6095 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:38.557693  5793 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:38.558115  6095 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:38.558526  6095 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.561671  6140 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:38.562207  6140 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:38.562640  6140 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:38.563864  5947 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:38.564306  6109 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:38.564836  6109 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:38.565187  6109 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.568316  5642 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:38.569083  1617 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:38.569893  5898 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:38.509639  5953 raft_consensus.cc:697] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:38.570958  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.571069  5953 consensus_queue.cc:237] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.568919  5909 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:38.573964  5934 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.990s	user 0.010s	sys 0.001s
I20260709 13:01:38.574441  5934 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.575773  5909 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.576151  5793 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.576479  1540 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:38.577328  5917 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.474225  5963 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.270s
I20260709 13:01:38.578321  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:38.576570  5934 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.578626  1613 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:38.579537  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:38.588095  5844 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:38.588608  5844 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:38.588950  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2653) took 1030 ms. Trace:
I20260709 13:01:38.589026  5844 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:38.589599  1689 rpcz_store.cc:276] 0709 13:01:37.558546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.558826 (+   280us) service_pool.cc:224] Handling call
0709 13:01:38.588727 (+1029901us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b5400058f54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:38.588892 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":940185472}
I20260709 13:01:38.590844  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:38.591776  1690 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 13:01:38.592581  6134 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 13:01:38.593713  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:38.594877  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:38.595963  6135 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 13:01:38.596024  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:38.597332  5922 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.597059  5986 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.597899  5922 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.598433  2234 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:38.600244  5922 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:38.602209  6129 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.604368  6129 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:38.606664  6075 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.607200  6133 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:38.607581  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2744) took 1465 ms. Trace:
I20260709 13:01:38.607995  1536 rpcz_store.cc:276] 0709 13:01:37.142187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.142293 (+   106us) service_pool.cc:224] Handling call
0709 13:01:38.607403 (+1465110us) spinlock_profiling.cc:243] Waited 829 ms on lock 0x7b54001e4454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:38.607520 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1907814144}
I20260709 13:01:38.608788  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:38.439950  6128 consensus_queue.cc:1059] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.110s
I20260709 13:01:38.610255  6128 consensus_queue.cc:1243] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:01:38.610785  6128 raft_consensus.cc:3055] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:38.611126  6128 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:38.611338  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2650) took 1068 ms. Trace:
I20260709 13:01:38.611938  6127 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:38.612417  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.611935  6128 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.614578  6128 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.618669  1683 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:38.619333  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:38.580681  6122 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:38.583390  5842 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:38.583837  6141 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:01:38.510345  6132 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.627442  1543 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:38.611725  1691 rpcz_store.cc:276] 0709 13:01:37.542591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.556311 (+ 13720us) service_pool.cc:224] Handling call
0709 13:01:37.556703 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:37.890311 (+333608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.611106 (+720795us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b540003dc54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:38.611278 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":765001472}
I20260709 13:01:38.744670  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:38.622628  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:38.638236  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:38.745632  1610 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:38.674117  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 156798us
I20260709 13:01:38.744318  1657 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:38.792775  2232 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:38.804405  5793 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.550602  6009 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:38.564296  5971 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:38.743793  5859 consensus_meta.cc:287] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.000s
I20260709 13:01:38.612435  6127 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:38.543877  5773 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.266s
I20260709 13:01:38.744772  5947 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.789898  6076 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:38.801553  6120 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:38.801820  5642 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:38.833174  6002 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.802752  5837 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.855821  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:38.550194  5968 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.843376  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:38.815142  6009 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.857887  1615 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:38.859243  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:38.815619  6116 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:38.859835  1615 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.821696  6129 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.860612  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:38.829980  6127 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:38.861289  1540 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:38.867103  1684 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
W20260709 13:01:38.862390  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 224239us
I20260709 13:01:38.870503  1615 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:38.871833  1615 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 13:01:38.875141  1610 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:38.875376  1615 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:38.888247  1691 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:38.891404  5951 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:38.893087  5951 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.865597  5971 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.896330  6149 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:38.896543  5971 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.897909  6131 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:38.898372  6131 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:38.898730  6131 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:38.899063  5971 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.585130  6070 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.910578  6052 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:38.854398  5929 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:38.912770  5929 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.915105  5929 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:38.847781  5642 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:38.915915  5642 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:38.921808  6110 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.865958  5837 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.924535  5837 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:38.925571  6129 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:38.927922  5933 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.645s	user 0.008s	sys 0.001s
I20260709 13:01:38.928422  6129 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.928774  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:38.848512  5947 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.932180  5947 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.828290  5859 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.936968  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 892) took 1189 ms. Trace:
I20260709 13:01:38.937449  1686 rpcz_store.cc:276] 0709 13:01:37.747290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.754201 (+  6911us) service_pool.cc:224] Handling call
0709 13:01:37.754583 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.350156 (+595573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.350204 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.353634 (+  3430us) log.cc:844] Serialized 26 byte log entry
0709 13:01:38.456238 (+102604us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:38.456273 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.456343 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:38.936745 (+480402us) raft_consensus.cc:1685] finished
0709 13:01:38.936766 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:38.936904 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":179090,"raft.run_cpu_time_us":64,"raft.run_wall_time_us":64,"thread_start_us":681174,"threads_started":2,"wal-append.queue_time_us":549856}
I20260709 13:01:38.939405  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:38.942556  1684 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40014
W20260709 13:01:38.943463  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:38.945034  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.946421  1615 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:38.947551  5986 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:38.896934  6149 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:38.954277  6149 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:38.956035  5968 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:38.868000  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:38.957592  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 89742.2us
I20260709 13:01:38.958321  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:38.960129  1658 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:38.960908  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:38.961666  6075 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:38.962155  6075 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:38.963035  6075 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.966073  5773 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:01:38.966735  6110 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:38.967344  5773 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:38.967730  5773 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:38.968089  5773 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:38.968248  6110 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:38.968776  6110 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:38.975432  5844 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:38.925298  5837 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:38.977450  5837 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:38.978773  1689 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:38.979414  1685 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:38.980103  1535 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:38.986006  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.987890  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.988791  1610 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:38.990530  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.991559  1610 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:38.993389  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.979602  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:38.995167  1611 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:38.996292  6002 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:38.860263  6076 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:39.173023  1610 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:39.008953  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:39.173822  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 164953us
I20260709 13:01:39.174042  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.174723  1610 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:01:39.011731  1535 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
W20260709 13:01:38.900072  6071 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.675s	user 0.008s	sys 0.000s
W20260709 13:01:39.010102  6036 consensus_meta.cc:287] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.874s	user 0.011s	sys 0.000s
I20260709 13:01:39.176641  1616 raft_consensus.cc:2410] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 13:01:39.010613  5900 consensus_meta.cc:287] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.514s	user 0.003s	sys 0.006s
I20260709 13:01:39.177232  5900 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:39.178481  6125 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.852s	user 0.010s	sys 0.000s
I20260709 13:01:39.179006  6125 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:39.180398  1690 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:39.175614  6144 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:01:39.181906  6128 consensus_meta.cc:287] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.001s
I20260709 13:01:39.182359  6144 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:39.183393  6144 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.185485  6129 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:39.186353  6129 consensus_queue.cc:1243] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:39.186949  6129 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:39.186791  6009 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.151729  5844 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:39.189633  1686 raft_consensus.cc:2410] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:01:39.131982  5859 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:38.941179  6095 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.010964  5929 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:39.065953  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:38.942449  6140 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.014569  5947 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:39.073256  6053 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.797s	user 0.010s	sys 0.000s
W20260709 13:01:39.190747  6030 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.577s	user 0.007s	sys 0.002s
I20260709 13:01:39.148756  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:39.214200  1540 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:39.215420  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:39.216279  1540 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:39.148941  1541 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:39.149610  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:39.151110  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.218411  1538 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:39.155076  5951 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:39.156291  6109 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.591s	user 0.003s	sys 0.004s
I20260709 13:01:39.062183  6110 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:39.035084  5837 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.171370  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.175593  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.179489  5900 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:39.055658  2233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.052s	sys 0.112s
I20260709 13:01:39.187419  6129 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:39.061030  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.236398  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.236836  1685 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:39.191224  1617 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:39.192708  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 894) took 1390 ms. Trace:
I20260709 13:01:39.144897  5642 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.190982  2228 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:39.241474  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.200021  1534 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:39.242255  1685 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:39.202378  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:39.136031  6132 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.172878  6131 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.226400  1684 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:39.229951  1688 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:39.230331  1610 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:39.233052  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:39.239697  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:39.231796  2233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.052s	sys 0.112s
I20260709 13:01:39.199681  6128 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.247098  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.237720  1687 rpcz_store.cc:276] 0709 13:01:37.801533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.801813 (+   280us) service_pool.cc:224] Handling call
0709 13:01:37.802384 (+   571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.184995 (+382611us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:38.388085 (+203090us) log.cc:844] Serialized 14 byte log entry
0709 13:01:39.137209 (+749124us) log.cc:844] Serialized 14 byte log entry
0709 13:01:39.137855 (+   646us) log.cc:844] Serialized 14 byte log entry
0709 13:01:39.137990 (+   135us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:39.138431 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.138543 (+   112us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:39.138573 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.138641 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.191789 (+ 53148us) raft_consensus.cc:1685] finished
0709 13:01:39.191807 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.191936 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":124875,"prepare.queue_time_us":117896,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":108,"raft.queue_time_us":415448,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":6378,"thread_start_us":1135709,"threads_started":4,"wal-append.queue_time_us":760958}
I20260709 13:01:39.294701  1685 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:39.295019  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.295892  1687 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:39.291769  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:39.292651  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.296998  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.229363  6036 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.301065  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.301795  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.293136  1684 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:39.302995  1691 raft_consensus.cc:2410] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:01:39.228761  6125 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:39.305003  6099 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.012s	sys 0.014s
I20260709 13:01:39.204758  5929 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:39.339584  5962 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:39.348877  5962 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:39.349009  6140 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:39.213328  5947 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:39.349759  6099 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.012s	sys 0.015s
I20260709 13:01:39.343379  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.346128  1687 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40114
I20260709 13:01:39.350504  1687 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:39.350859  1687 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:39.352036  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.347994  6131 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:39.315306  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.353804  1610 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:39.357883  1618 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:39.273741  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 529454us
I20260709 13:01:39.359634  1657 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:39.364871  1615 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:39.255785  5733 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:39.374701  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:39.373616  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:39.374183  1614 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.002s	sys 0.034s
I20260709 13:01:39.228327  6030 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.380267  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.378978  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.232084  6109 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.381094  1615 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:39.397921  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:39.386276  1542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.017s	sys 0.020s
W20260709 13:01:39.399143  1613 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.027s	sys 0.010s
W20260709 13:01:39.399408  1542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.017s	sys 0.020s
I20260709 13:01:39.305563  5578 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.256064  6071 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:01:39.383087  1582 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:39.413573  1615 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:39.415441  2203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.015s	sys 0.025s
W20260709 13:01:39.417613  2203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.016s	sys 0.025s
I20260709 13:01:39.421660  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.422107  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:39.399644  1613 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.028s	sys 0.010s
W20260709 13:01:39.377059  1614 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.002s	sys 0.034s
I20260709 13:01:39.373792  6125 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:39.425583  1615 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:57992
W20260709 13:01:39.397368  1536 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.006s	sys 0.031s
I20260709 13:01:39.426851  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.430675  1615 raft_consensus.cc:606] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:01:39.426851  1687 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.395560  6002 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:39.395421  6131 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:01:39.423768  6057 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.034s	sys 0.008s
W20260709 13:01:39.430692  1536 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.006s	sys 0.031s
W20260709 13:01:39.413017  5793 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.607s	user 0.011s	sys 0.001s
I20260709 13:01:39.356362  6036 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.431667  1689 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:39.438845  1612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.832s	user 0.033s	sys 0.043s
I20260709 13:01:39.308470  6128 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.364650  5773 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.440708  1615 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:39.443300  1612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.836s	user 0.033s	sys 0.043s
I20260709 13:01:39.421116  5733 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:39.397567  5928 consensus_queue.cc:1059] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:01:39.428778  6010 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:39.427739  6151 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:39.442046  6057 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.034s	sys 0.008s
I20260709 13:01:39.521848  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.436700  6009 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:39.425622  5578 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.526921  1615 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:39.530550  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:39.380635  6129 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:39.532832  1615 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:39.524852  6010 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:39.560169  6140 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.566679  6140 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:39.568135  6070 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:39.570045  5642 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:39.567092  5773 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:39.571580  1509 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.022s	sys 0.025s
I20260709 13:01:39.569135  6140 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:39.522724  6125 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:01:39.561347  6129 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:39.558866  6151 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.563462  6010 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:39.565374  6095 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:39.572469  1509 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.022s	sys 0.025s
I20260709 13:01:39.565109  5929 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:39.576525  2201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.732s	user 0.000s	sys 0.017s
W20260709 13:01:39.595330  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 589736us
W20260709 13:01:39.598688  2201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.001s	sys 0.017s
W20260709 13:01:39.603893  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 760290us
W20260709 13:01:39.607394  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.581295  5947 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:39.622579  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.628024  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:39.628916  2232 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:39.636750  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:39.641903  2237 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:39.645408  1684 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.647377  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:39.595768  1658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.017s	sys 0.015s
I20260709 13:01:39.619040  6123 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.008s
W20260709 13:01:39.652452  1658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.018s	sys 0.015s
I20260709 13:01:39.654119  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:39.659530  2231 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:39.595445  6151 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:39.581705  5733 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.676826  6110 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.680168  5947 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.686067  6095 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:39.646322  6053 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.706000  5947 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:39.706907  6163 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:01:39.706990  6162 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
W20260709 13:01:39.702096  6127 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.732s	user 0.012s	sys 0.000s
I20260709 13:01:39.716375  6095 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.727540  6095 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:39.729658  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2210) took 1099 ms. Trace:
I20260709 13:01:39.730187  1542 rpcz_store.cc:276] 0709 13:01:38.629892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.630131 (+   239us) service_pool.cc:224] Handling call
0709 13:01:38.630454 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.742740 (+112286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.742772 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.743303 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.400198 (+656895us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:39.400256 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.400567 (+   311us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.729413 (+328846us) raft_consensus.cc:1685] finished
0709 13:01:39.729453 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.729596 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":160246,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":10313,"thread_start_us":656518,"threads_started":1,"wal-append.queue_time_us":882791}
I20260709 13:01:39.731817  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2254) took 1230 ms. Trace:
I20260709 13:01:39.732311  1612 rpcz_store.cc:276] 0709 13:01:38.501007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.501140 (+   133us) service_pool.cc:224] Handling call
0709 13:01:38.501491 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.501836 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.501869 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.502330 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:01:38.606862 (+104532us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b18003cdde0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:01:39.520289 (+913427us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:39.520327 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.520400 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.731534 (+211134us) raft_consensus.cc:1685] finished
0709 13:01:39.731553 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.731752 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10461,"raft.queue_time_us":839,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":28200,"spinlock_wait_cycles":99264896,"thread_start_us":913237,"threads_started":1,"wal-append.queue_time_us":1197510}
W20260709 13:01:39.733989  6165 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 13:01:39.719112  6053 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.735355  5837 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:39.739001  6151 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:39.742525  1534 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:39.743752  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:39.744295  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2211) took 1114 ms. Trace:
I20260709 13:01:39.744884  1687 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:39.743757  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:39.746218  1538 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:39.749408   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:01:39.749794  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.744678  1536 rpcz_store.cc:276] 0709 13:01:38.630199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.631173 (+   974us) service_pool.cc:224] Handling call
0709 13:01:38.631494 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.812884 (+181390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.812916 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.816451 (+  3535us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.441367 (+624916us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:39.441406 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.441822 (+   416us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.744084 (+302262us) raft_consensus.cc:1685] finished
0709 13:01:39.744104 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.744234 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":189971,"raft.queue_time_us":183153,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9451,"thread_start_us":597488,"threads_started":1,"wal-append.queue_time_us":761751}
I20260709 13:01:39.752667  5578 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:39.753221  6128 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:39.756961  6030 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:39.758379  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1091) took 1386 ms. Trace:
I20260709 13:01:39.759419  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:39.760218  1687 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:39.760385  5947 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:39.737115  6053 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:39.758826  2230 rpcz_store.cc:276] 0709 13:01:38.374110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.374438 (+   328us) service_pool.cc:224] Handling call
0709 13:01:38.388347 (+ 13909us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:38.412811 (+ 24464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.412859 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:38.413683 (+   824us) log.cc:844] Serialized 52 byte log entry
0709 13:01:38.802138 (+388455us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b18003df840. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:01:39.205478 (+403340us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:39.205537 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.205635 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.758054 (+552419us) raft_consensus.cc:1685] finished
0709 13:01:39.758075 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.758312 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2446,"raft.queue_time_us":136,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":9575,"spinlock_wait_cycles":8167296,"thread_start_us":403079,"threads_started":1,"wal-append.queue_time_us":1019529}
I20260709 13:01:39.720049  6127 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.764168  2234 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:39.718189  6154 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.198s	user 0.001s	sys 0.001s
I20260709 13:01:39.767402  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1773) took 1278 ms. Trace:
I20260709 13:01:39.767958  1683 rpcz_store.cc:276] 0709 13:01:38.490046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.511651 (+ 21605us) service_pool.cc:224] Handling call
0709 13:01:38.548807 (+ 37156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.619146 (+ 70339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.767168 (+1148022us) spinlock_profiling.cc:243] Waited 69.6 ms on lock 0x7b5400374e54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:39.767340 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160191360}
I20260709 13:01:39.761056  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:39.769658  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2255) took 1264 ms. Trace:
I20260709 13:01:39.770249  1614 rpcz_store.cc:276] 0709 13:01:38.505504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.505765 (+   261us) service_pool.cc:224] Handling call
0709 13:01:38.544063 (+ 38298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.544537 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.544577 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.545151 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.426382 (+881231us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:39.426507 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.426582 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.769310 (+342728us) raft_consensus.cc:1685] finished
0709 13:01:39.769338 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.769579 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25194,"raft.queue_time_us":5251,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":193074,"thread_start_us":602587,"threads_started":1,"wal-append.queue_time_us":805362}
I20260709 13:01:39.771584  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:39.773944  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 318105us
W20260709 13:01:39.776474  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 157447us
I20260709 13:01:39.779299  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:39.782998  2231 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:39.780126  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.765359  6127 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:39.790122  1691 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:39.790853  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:39.796386  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 895) took 1963 ms. Trace:
I20260709 13:01:39.716297  6110 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.796928  1692 rpcz_store.cc:276] 0709 13:01:37.888879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.889176 (+   297us) service_pool.cc:224] Handling call
0709 13:01:37.935082 (+ 45906us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:38.145958 (+210876us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.145988 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:38.146552 (+   564us) log.cc:844] Serialized 52 byte log entry
0709 13:01:38.340375 (+193823us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:38.372182 (+ 31807us) log.cc:844] Serialized 14 byte log entry
0709 13:01:38.372405 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:38.591937 (+219532us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b180033c700. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff557 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:01:39.796058 (+1204121us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b5400355054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:39.796114 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.796156 (+    42us) raft_consensus.cc:1685] finished
0709 13:01:39.796175 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.796328 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29668,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":8163,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":10921,"spinlock_wait_cycles":524971264,"thread_start_us":193446,"threads_started":1,"wal-append.queue_time_us":253491}
I20260709 13:01:39.806875  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:39.812834  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1779) took 1126 ms. Trace:
I20260709 13:01:39.799765  6167 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:01:39.816437  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:39.817103  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:39.789166  1538 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:39.819366  1683 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:39.672683  6033 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:39.834174  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 144577us
I20260709 13:01:39.819388  1613 rpcz_store.cc:276] 0709 13:01:38.686986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.687141 (+   155us) service_pool.cc:224] Handling call
0709 13:01:38.687544 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.687818 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:39.427626 (+739808us) log.cc:844] Serialized 14 byte log entry
0709 13:01:39.681105 (+253479us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:39.683889 (+  2784us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:39.687390 (+  3501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.812544 (+125154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.812740 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":229872,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":1955,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":122,"thread_start_us":859339,"threads_started":2}
I20260709 13:01:39.827704  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1092) took 1452 ms. Trace:
I20260709 13:01:39.831394  2203 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:39.789614  6123 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:39.838014  5947 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:39.839048  6171 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:39.839454  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:39.840821  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2209) took 1377 ms. Trace:
I20260709 13:01:39.842335  6167 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:39.873013  1610 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:39.873672  5991 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:39.822533  6125 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.874006  5947 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:39.874119  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.874611  5947 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:39.874790  1613 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:39.874536  6125 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:39.876766  6125 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:39.843505  2228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1244) took 1270 ms. Trace:
I20260709 13:01:39.847398  6057 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:39.848417  1692 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:39.877398  2228 rpcz_store.cc:276] 0709 13:01:38.575742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.578045 (+  2303us) service_pool.cc:224] Handling call
0709 13:01:39.843318 (+1265273us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54005eddd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:39.843437 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407494400}
W20260709 13:01:39.850642  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:39.878443  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.879016  1692 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 8.
I20260709 13:01:39.879840  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.880930  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.874161  5991 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:39.882097  5991 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:39.883000  1613 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:39.837178  2233 rpcz_store.cc:276] 0709 13:01:38.374846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.375036 (+   190us) service_pool.cc:224] Handling call
0709 13:01:38.375374 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:38.375750 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:38.375781 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:38.393720 (+ 17939us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.307131 (+913411us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:39.307188 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.307282 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:39.827415 (+520133us) raft_consensus.cc:1685] finished
0709 13:01:39.827443 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:39.827622 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":4237,"thread_start_us":856999,"threads_started":1,"wal-append.queue_time_us":993613}
I20260709 13:01:39.888461  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:39.849592  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:39.889243  1613 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:39.889212  1610 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:01:39.889768  1610 pending_rounds.cc:85] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 5
I20260709 13:01:39.866894  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:39.889973  1685 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:39.891086  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.891676  1685 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:39.892087  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:39.892508  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.889995  1610 pending_rounds.cc:107] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 13:01:39.894073  1610 raft_consensus.cc:2889] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:39.866153  1535 rpcz_store.cc:276] 0709 13:01:38.463348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.480374 (+ 17026us) service_pool.cc:224] Handling call
0709 13:01:39.840630 (+1360256us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54000eddd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:39.840755 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147369856}
W20260709 13:01:39.894623  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 264859us
I20260709 13:01:39.896525  1615 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:01:39.896858  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:39.895699  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:39.822630  6127 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:39.902222  1613 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:39.902614  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:39.908433  1610 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:39.909710  1610 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:39.909897  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:39.910751  1615 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:39.886991  5947 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.902835  1685 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:39.919277  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.931221  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.823254  5968 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
I20260709 13:01:39.911067  1610 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.934170  1610 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:39.934372  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:39.935863  1685 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:39.939270  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:39.944137  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:39.944423  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:39.945240  1538 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:39.945132  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:39.912542  5578 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:39.942708  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:39.951522  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:39.953758  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.955381  1685 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:39.956669  1615 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:39.957067  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:39.958174  1685 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 14. Ops: []
I20260709 13:01:39.961277  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:39.961520  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:39.963210  1538 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:39.967212  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:39.967798  1538 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:39.967885  1534 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:39.968225  1685 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:39.977810  1614 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:39.853549  6033 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:39.991473  1610 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
W20260709 13:01:39.996294  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 104268us
I20260709 13:01:39.999369  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:39.999969  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:40.001470  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:40.002852  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:39.999329  1539 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:40.005534  1542 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:39.959221  6171 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:40.007493  1538 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:01:39.960822  5578 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:39.995870  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 202981us
I20260709 13:01:39.959110  5951 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:39.998294  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:40.012878  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:40.028932  1536 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:40.014842  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:39.983352  6186 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:01:39.981844  6178 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:01:39.970909  6130 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:39.977931  6095 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:40.033754  1509 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:40.038744  1534 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:40.016054  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 834) took 1006 ms. Trace:
I20260709 13:01:40.045684  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:40.047581  1536 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:40.043728  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 147837us
I20260709 13:01:40.018887  6171 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.025579  5578 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:39.979437  6010 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:40.046627  1537 rpcz_store.cc:276] 0709 13:01:39.009329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.009482 (+   153us) service_pool.cc:224] Handling call
0709 13:01:39.016544 (+  7062us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.016750 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:39.450419 (+433669us) log.cc:844] Serialized 14 byte log entry
0709 13:01:40.014156 (+563737us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:40.014238 (+    82us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:40.015718 (+  1480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.015812 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.015981 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":624487,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":100770,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":16044,"thread_start_us":948404,"threads_started":3,"wal-append.queue_time_us":376152}
W20260709 13:01:40.088770  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64752.7us
I20260709 13:01:39.997495  6059 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:01:40.039117  6070 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:40.105084  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 135813us
I20260709 13:01:40.018663  6131 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 13:01:40.101819  1509 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:40.044556  6095 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.135087  6095 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.136101  6149 raft_consensus.cc:3055] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:40.136504  6149 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:40.137372  6149 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:40.138419  6149 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:40.138612  6122 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.139320  6010 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:40.139993  6010 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:40.140446  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 910) took 1507 ms. Trace:
I20260709 13:01:40.140856  1689 rpcz_store.cc:276] 0709 13:01:38.633142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.159784 (+526642us) service_pool.cc:224] Handling call
0709 13:01:39.160108 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.443290 (+283182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.140205 (+696915us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b54000ab9d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:40.140383 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624039680}
I20260709 13:01:40.142727  1689 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:01:40.143334  6144 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:40.144975  5947 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:40.098138  6168 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:40.146965  6168 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:40.044315  6130 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.148034  6130 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:40.150144  6183 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 13:01:40.151063  6190 log.cc:927] Time spent T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:01:40.151816  6160 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.070775  6128 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:40.152410  6160 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:40.134454  6052 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:40.139109  6122 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:40.153187  6128 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:40.153780  6052 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.154233  6052 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.098745  6198 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 13:01:40.103803  6030 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:01:40.155391  6128 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:40.155287  6198 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:40.156284  6198 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:40.133952  6189 log.cc:927] Time spent T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:01:40.157138  6198 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:40.158353  6198 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:40.158982  5991 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:40.039348  6193 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:01:40.161748  6193 consensus_queue.cc:1243] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 11.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:01:40.162276  6193 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:40.162608  6193 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:40.163470  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:40.163383  6193 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:40.164616  6193 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.164858  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:40.155290  6122 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:40.103664  6166 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:01:40.085903  6179 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 13:01:40.165895  6144 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:40.166328  6144 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:40.166651  6144 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:40.149250  6168 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:40.140479  6010 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:40.168339  6167 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:40.169500  6167 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:40.170645  6109 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:40.169875  6167 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:40.171265  6167 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:40.088770  6171 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:40.169852  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 67991.2us
W20260709 13:01:40.174934  6181 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.352s	user 0.001s	sys 0.000s
I20260709 13:01:40.175097  1509 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:40.179352  6033 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:40.183434  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2217) took 1034 ms. Trace:
W20260709 13:01:40.185257  6165 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:01:40.187387  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:40.188548  1539 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:40.188792  6036 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:40.044752  6127 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:40.190871  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:40.185382  1618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2251) took 1724 ms (client timeout 2009 ms). Trace:
I20260709 13:01:40.190505  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:40.191852  1609 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:40.190681  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:40.186086  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 908) took 1554 ms. Trace:
I20260709 13:01:40.147493  6149 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:40.183943  1540 rpcz_store.cc:276] 0709 13:01:39.148931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.217125 (+ 68194us) service_pool.cc:224] Handling call
0709 13:01:39.219515 (+  2390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:39.219954 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:39.219984 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:39.248355 (+ 28371us) log.cc:844] Serialized 26 byte log entry
0709 13:01:39.666741 (+418386us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:39.666813 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:39.666907 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:40.183193 (+516286us) raft_consensus.cc:1685] finished
0709 13:01:40.183218 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:40.183370 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":6278,"thread_start_us":358667,"threads_started":1,"wal-append.queue_time_us":529917}
I20260709 13:01:40.186968  1689 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.192950  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:40.194523  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.196199  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.102918  6070 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:40.039336  6075 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
W20260709 13:01:40.269557  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 74744.4us
W20260709 13:01:40.230388  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.041029  5951 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:40.199838  1618 rpcz_store.cc:269] 0709 13:01:38.460781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.494578 (+ 33797us) service_pool.cc:224] Handling call
0709 13:01:40.185136 (+1690558us) spinlock_profiling.cc:243] Waited 862 ms on lock 0x7b5400389d54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:40.185290 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1982613376}
I20260709 13:01:40.261200  1686 rpcz_store.cc:276] 0709 13:01:38.631260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.939134 (+307874us) service_pool.cc:224] Handling call
0709 13:01:40.185850 (+1246716us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54000744d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:40.186008 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":573459840}
I20260709 13:01:40.232223  6193 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:40.197068  1536 raft_consensus.cc:2410] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:40.275609  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:40.276579  1538 raft_consensus.cc:2376] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8. Re-sending same reply.
I20260709 13:01:40.276187  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:40.280812  1536 raft_consensus.cc:2376] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14. Re-sending same reply.
W20260709 13:01:40.295212  1584 leader_election.cc:336] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.623s (SENT)
I20260709 13:01:40.295897  1584 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:40.427130  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.267498  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 195477us
I20260709 13:01:40.442830  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2757) took 2664 ms. Trace:
W20260709 13:01:40.442737  1616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1767) took 2020 ms (client timeout 2020 ms). Trace:
I20260709 13:01:40.443230  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:40.444378  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 174064us
I20260709 13:01:40.446352  1540 raft_consensus.cc:2376] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10. Re-sending same reply.
I20260709 13:01:40.446478  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:40.447304  1535 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:40.447377  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:40.449319  2201 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:40.179332  6160 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:40.445957  1616 rpcz_store.cc:269] 0709 13:01:38.422428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.422575 (+   147us) service_pool.cc:224] Handling call
0709 13:01:40.276640 (+1854065us) spinlock_profiling.cc:243] Waited 752 ms on lock 0x7b54003c87d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:40.442652 (+166012us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1729629952}
I20260709 13:01:40.177253  6010 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:40.268432  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 269070us
W20260709 13:01:40.451418  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 156268us
I20260709 13:01:40.451776  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:40.478199  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.459055  1689 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:01:40.479557  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:40.480113  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:40.480964  1689 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:01:40.451287  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:40.482172  1536 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:40.444940  1543 rpcz_store.cc:276] 0709 13:01:37.599342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:37.599662 (+   320us) service_pool.cc:224] Handling call
0709 13:01:38.610719 (+1011057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:38.743896 (+133177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.190087 (+1446191us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b54001e4454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:40.263824 (+ 73737us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001e4450. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:40.263978 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2354404608}
I20260709 13:01:40.530148  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:40.530965  1615 raft_consensus.cc:2376] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11. Re-sending same reply.
I20260709 13:01:40.531903  1612 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:40.532505  1615 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:01:40.242625  6184 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 13:01:40.531948  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:40.534164  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:40.537384  1534 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:40.535142  1618 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:40.535815  1538 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:40.527264  5733 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:40.259558  5991 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:40.543326  1692 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:40.541999  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:40.270099  6070 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:40.547394  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:40.555485  1614 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:40.560180  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:40.563200  6194 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:40.557574   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:01:40.535425  6099 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.540817  6118 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.551945  5956 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.256978  6130 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:40.442703  6149 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:40.595260  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:40.610384  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 130315us
I20260709 13:01:40.611198  1535 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:40.526779  6123 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 13:01:40.588567  6099 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:40.613811  6205 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:40.614758  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 165532us
I20260709 13:01:40.645058  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 739) took 1818 ms. Trace:
I20260709 13:01:40.645524  1609 rpcz_store.cc:276] 0709 13:01:38.826443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.826704 (+   261us) service_pool.cc:224] Handling call
0709 13:01:40.191622 (+1364918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.448445 (+256823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.644882 (+196437us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400389d50. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:40.645002 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3138225536}
I20260709 13:01:40.532932  6206 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.168s
I20260709 13:01:40.551627  6195 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:40.563966  6103 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.003s	sys 0.030s
W20260709 13:01:40.628930  1658 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.048s	sys 0.078s
W20260709 13:01:40.658726  1658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.049s	sys 0.078s
W20260709 13:01:40.659193  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 612445us
I20260709 13:01:40.661216  1658 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:40.632182  1534 raft_consensus.cc:2410] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:01:40.578111  6194 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:40.241376  6171 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:40.533517  6095 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:40.629691  6118 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:40.621323  6127 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:40.664703  5989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.015s	sys 0.012s
I20260709 13:01:40.716995  6177 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:01:40.759397  5962 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:40.723429  6193 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:01:40.761284  6129 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.177s	user 0.011s	sys 0.002s
I20260709 13:01:40.723917  6206 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:01:40.763729  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:40.765661  6118 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:40.765695  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:40.723421  6052 consensus_meta.cc:287] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.000s
I20260709 13:01:40.717303  6122 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:40.767752  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:40.723716  6205 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:01:40.770556  6132 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:40.771221  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:40.771948  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:40.772197  1684 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:40.772608  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:40.773828  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:40.758023  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.767805  1685 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:01:40.686401  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.739703  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:40.700341  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:40.762632  6110 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:40.830864  1541 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:01:40.829387  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:40.869766  1692 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:40.731290  5578 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.605s	user 0.008s	sys 0.000s
I20260709 13:01:40.807337  1686 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:40.876230  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:40014 (request call id 907) took 2307 ms. Trace:
W20260709 13:01:40.662243  6103 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.003s	sys 0.031s
I20260709 13:01:40.755246  6123 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:40.732208  6200 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:01:40.866084  2236 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:01:40.866679  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 232783us
W20260709 13:01:40.867354  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 167107us
I20260709 13:01:40.887631  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:40.888659  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:40.747758  6099 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:40.630903  5956 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:40.932717  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:40.808817  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 243753us
I20260709 13:01:40.876089  5962 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:40.872848  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:40.779560  6144 consensus_meta.cc:287] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.570s	user 0.008s	sys 0.000s
I20260709 13:01:40.878855  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:40.876585  1684 rpcz_store.cc:276] 0709 13:01:38.558466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.942384 (+383918us) service_pool.cc:224] Handling call
0709 13:01:40.770795 (+1828411us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b5400374954. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fae3a3d 00007fe094ab3cd0 00007fe08e038254 00007fe08e0381e2 00007fe08e038111 00007fe08e03808c 00007fe08e037342 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:40.866013 (+ 95218us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":195,"spinlock_wait_cycles":650936064}
W20260709 13:01:40.754225  6198 consensus_meta.cc:287] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.595s	user 0.010s	sys 0.000s
I20260709 13:01:40.786839  6194 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:40.968918  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 90180.6us
I20260709 13:01:40.806576  6177 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:40.812767  5989 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:40.891745  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:40.969832  1613 raft_consensus.cc:2410] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 13:01:40.969861  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:40.970878  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:40.727165  6167 consensus_meta.cc:287] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.555s	user 0.009s	sys 0.000s
I20260709 13:01:40.902035  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:40.896296  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:40.827234  6128 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.807761  6199 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.787763  6193 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:40.931468  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:40.867780  6052 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:40.875233  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:40.721894  6210 consensus_queue.cc:1059] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.208s
I20260709 13:01:40.964349  6036 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:40.959909  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:40.698534  6160 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.232s
I20260709 13:01:40.878863  6075 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:40.959956  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:41.005714  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2670) took 1749 ms. Trace:
I20260709 13:01:41.005894  2231 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:40.875612  6118 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.927866  6123 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:40.925069  6059 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.096s	sys 0.068s
I20260709 13:01:40.897403  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:40.961360  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:40.967728  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:40.968034  1685 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:40.969664  6177 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:40.971869  1584 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:41.017675  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:40.884157  6122 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:40.973520  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:41.021883  1534 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:01:40.974907  1535 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:41.003407  6110 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.665939  6195 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:40.972381  6144 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:40.977255  6198 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:40.969866  5989 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:40.977856  6132 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:41.014463  6059 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.097s	sys 0.069s
I20260709 13:01:40.981484  6167 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:40.978488  1543 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:01:41.029950  1543 pending_rounds.cc:85] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 7
I20260709 13:01:41.030280  1543 pending_rounds.cc:107] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:01:41.030579  1543 raft_consensus.cc:2889] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:40.966936  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:40.979648  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:40.877242  5837 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.086s
W20260709 13:01:41.033567  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 66764us
I20260709 13:01:41.009379  6036 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:40.716840  6109 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.164s
I20260709 13:01:41.042595  1534 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:40.991225  1692 raft_consensus.cc:2410] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:01:40.890908  5733 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 13:01:40.933399  2235 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:41.015164  6075 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:40.998752  2229 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:40.883689  5578 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:40.973503  5956 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:40.868402  6129 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:40.984825  6219 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.006261  1683 rpcz_store.cc:276] 0709 13:01:39.256673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.828376 (+571703us) service_pool.cc:224] Handling call
0709 13:01:39.829105 (+   729us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:39.833843 (+  4738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:39.833872 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:39.838500 (+  4628us) log.cc:844] Serialized 26 byte log entry
0709 13:01:40.007397 (+168897us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:40.007433 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:40.007505 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.005406 (+997901us) raft_consensus.cc:1685] finished
0709 13:01:41.005452 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.005635 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4105,"raft.queue_time_us":4457,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":32561,"thread_start_us":168532,"threads_started":1,"wal-append.queue_time_us":715552}
I20260709 13:01:41.013928  2234 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:41.015451  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:41.065976  2232 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:41.018409  1685 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:41.069361  1685 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:40.976507  5922 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:40.978228  1657 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.014s	sys 0.018s
I20260709 13:01:40.984826  6128 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.034847  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:40.985329  1686 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:41.043587  2231 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:41.044626  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:41.045192  6215 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:41.045207  6216 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:01:41.050307  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:40.997300  2201 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:41.053468  1539 raft_consensus.cc:2368] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10.
I20260709 13:01:41.053658  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:40.993721  6199 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:01:41.088918  1657 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.014s	sys 0.019s
I20260709 13:01:41.093084  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:41.107196  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 853224us
I20260709 13:01:41.050935  5773 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:41.090174  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:41.090480  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:41.034698  6110 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:41.091563  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56780.8us
W20260709 13:01:41.102630  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 130824us
W20260709 13:01:41.104121  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 106941us
I20260709 13:01:41.017046  6118 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.085433  5956 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:01:41.065609  6103 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:41.092690  1683 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:41.120605  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:41.091219  5922 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.150669  2236 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:41.145612  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:41.022879  6193 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.146656  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:41.101339  6149 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:41.121639  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 162365us
I20260709 13:01:41.089798  6218 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:41.154476  2237 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:41.154589  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:41.155606  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 901) took 2937 ms. Trace:
I20260709 13:01:41.161705  1685 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:41.163609  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:41.113196  6212 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:41.087117  6129 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:41.048877  6177 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.165990  1688 rpcz_store.cc:276] 0709 13:01:38.208229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:38.208339 (+   110us) service_pool.cc:224] Handling call
0709 13:01:38.793832 (+585493us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b5800010e98. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7eee 0000000000375939 00007fe094a3b1e0 00007fe094b040c4 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:01:38.793996 (+   164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.290998 (+497002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.145209 (+1854211us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b540003e654. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:41.145470 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1004810752}
I20260709 13:01:41.173213  2231 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:41.173381  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:41.146891  6059 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:41.151716  6130 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:41.156785  5922 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.189877  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:41.170660  6010 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:41.194872  2236 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:41.191075  1537 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:41.165541  6218 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:41.165086  5956 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:41.090703  6219 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:41.197094  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:41.199055  2231 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:41.199714  6123 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:41.199965  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:41.177896  6109 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:41.205983  1688 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 14. Ops: []
I20260709 13:01:41.182571  6193 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.202195  6052 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:41.218021  1686 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:41.217672  5956 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:41.218623  6195 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:41.222229  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:41.222888  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:41.223781  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:41.224377  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:41.227322  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:41.228024  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:41.248932  1611 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:41.161216  6103 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.230556  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:41.231419  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:41.250000  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:41.250303  1612 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:41.250592  1611 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:41.235598  1657 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:41.236482  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:41.252599  1615 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:41.197680  6177 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:01:41.151266  6122 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.188989  5733 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:01:41.249096  1614 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:41.228628  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:41.182372  6149 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:01:41.255201  1657 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:41.249791  1618 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:41.238593  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:41.170305  6036 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.239959  6130 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:41.213913  6010 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:41.277297  1657 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:41.165530  6075 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.282886  1611 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:01:41.264294  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:41.289541  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:41.289623  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 51108us
I20260709 13:01:41.245247  6123 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.284790  5793 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 13:01:41.326826  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 177461us
I20260709 13:01:41.328441  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:41.325989  2235 raft_consensus.cc:2376] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10. Re-sending same reply.
I20260709 13:01:41.329985  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:41.190867  6059 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.333547  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:41.245080  5837 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 13:01:41.254595  5995 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
W20260709 13:01:41.323851  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 46649.5us
W20260709 13:01:41.322312  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 150215us
I20260709 13:01:41.337811  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:41.349771  1657 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:41.334899  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:41.320960  6130 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:41.275424  6232 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:41.321404  6210 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:41.320385  5989 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:41.360124  2229 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.347714  5793 consensus_queue.cc:1243] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 8a6a5ae1b7f240408b3ddd30d8a18262 log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:41.364217  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:41.314415  6144 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:41.363408  1618 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [12.11-15.12]
I20260709 13:01:41.374450  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:41.383181  1615 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:41.378060  1617 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:41.379184  1614 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:41.379767  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:41.389523  1535 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:41.323160  6109 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:41.399384  1615 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:41.336711  6160 consensus_queue.cc:1059] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:01:41.319763  6218 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.415496  5793 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:41.416148  6218 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:41.416266  5793 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.416944  5793 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:41.417572  6010 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:41.360188  6210 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:41.418962  6210 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.420563  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:41.377373  6215 raft_consensus.cc:697] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:41.422248  6215 consensus_queue.cc:237] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:41.325731  6123 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.353513  6167 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:41.400189  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 223230us
I20260709 13:01:41.441727  6109 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:41.355525  6130 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:41.388793  6128 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:41.432296  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 77357.3us
I20260709 13:01:41.351822  6195 consensus_queue.cc:1059] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:01:41.390415  5989 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.461555  1543 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:41.431583  2229 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
W20260709 13:01:41.453800  1584 leader_election.cc:336] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.647s (SENT)
W20260709 13:01:41.462723  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 127893us
I20260709 13:01:41.462991  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:41.466544  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 116867us
W20260709 13:01:41.456517  6070 consensus_meta.cc:287] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.876s	user 0.012s	sys 0.000s
I20260709 13:01:41.467722  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:41.429791  6110 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:41.429370  6144 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:41.466235  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:41.474356  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 140226us
I20260709 13:01:41.474532  1618 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:41.475068  1618 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:41.462086  6129 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:41.467988  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:41.466392  6199 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:41.481729  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:41.484392  1612 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:41.464398  6109 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 13:01:41.475997  1618 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:41.488214  1618 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:41.481056  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:41.470907  6128 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:41.493149  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:41.494391  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:41.494843  6123 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:41.501319  6219 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:41.502295  6118 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:41.479466  6160 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:41.502966  6160 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:41.503599  6160 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.505885  1657 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:41.507548  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:41.515827  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:41.527752  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46778.8us
I20260709 13:01:41.528829  2203 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:41.531258  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:41.532058  1686 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:41.472906  6070 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.557991  1618 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:01:41.475891  6144 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.561802  1618 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:01:41.548537  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 91675.6us
W20260709 13:01:41.570708  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 132072us
I20260709 13:01:41.561571  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:41.510300  5989 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:41.526976  6103 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:41.557791  5962 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.002s
I20260709 13:01:41.533877  5837 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.546857  6195 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:41.558127  6110 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:41.563333  6226 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
I20260709 13:01:41.575151  1689 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:41.578200  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:41.647898  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:41.648170  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 70014.9us
I20260709 13:01:41.563045  6238 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:41.622277  1584 leader_election.cc:336] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.807s (SENT)
I20260709 13:01:41.651119  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:41.659678  6238 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.565045  5793 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:41.665997  6233 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.266s	user 0.001s	sys 0.001s
W20260709 13:01:41.674146  6236 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
I20260709 13:01:41.680615  6238 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:41.681787  6204 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:41.682389  6204 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:41.682615  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:41.683308  1543 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.684314  5922 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:41.685503  1614 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 13:01:41.683764  6204 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:41.686815  6241 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.209s	user 0.000s	sys 0.000s
I20260709 13:01:41.687045  6204 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:41.689594  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2663) took 2452 ms. Trace:
I20260709 13:01:41.613988  1508 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:41.649246  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:41.574329  6221 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
W20260709 13:01:41.574277  6229 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 13:01:41.737691  6245 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:41.738297  6245 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:41.738618  1616 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:01:41.738691  6245 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:41.739452  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1791) took 1211 ms. Trace:
I20260709 13:01:41.739816  1616 rpcz_store.cc:276] 0709 13:01:40.528483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.530887 (+  2404us) service_pool.cc:224] Handling call
0709 13:01:41.738225 (+1207338us) spinlock_profiling.cc:243] Waited 4.06 ms on lock 0x7b54002dbd28. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a90ebc 00007fe094ac608c 00007fe094ab093b 00007fe094ab03d2 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c
0709 13:01:41.738359 (+   134us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:41.739142 (+   783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.739370 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9347456}
I20260709 13:01:41.745604  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:41.746237  6249 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:01:41.747267  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:41.749003  1538 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:01:41.750293  2232 raft_consensus.cc:2368] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14.
W20260709 13:01:41.751850  1539 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2776) took 1564 ms (client timeout 1647 ms). Trace:
I20260709 13:01:41.752369  6130 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:41.575289  6234 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:41.690016  1691 rpcz_store.cc:276] 0709 13:01:39.237202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.303775 (+ 66573us) service_pool.cc:224] Handling call
0709 13:01:39.338234 (+ 34459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:39.796025 (+457791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.689335 (+1893310us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b540028e6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:41.689532 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038456576}
I20260709 13:01:41.574291  6240 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:41.574272  6235 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 13:01:41.575378  6231 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:41.575300  6228 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:41.576257  6207 raft_consensus.cc:3055] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:41.719384  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:41.730563  6144 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:41.731804  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2784) took 1542 ms. Trace:
I20260709 13:01:41.893903  1536 rpcz_store.cc:276] 0709 13:01:40.196374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.534056 (+337682us) service_pool.cc:224] Handling call
0709 13:01:40.534393 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.563771 (+ 29378us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:40.585781 (+ 22010us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.062669 (+476888us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.062847 (+   178us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:41.063386 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 13:01:41.063544 (+   158us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:41.063748 (+   204us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.174477 (+110729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.174599 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:41.731548 (+556949us) raft_consensus.cc:1685] finished
0709 13:01:41.731571 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:41.731732 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80740,"prepare.queue_time_us":69996,"prepare.run_cpu_time_us":146,"prepare.run_wall_time_us":3046,"raft.queue_time_us":415617,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":50286,"spinlock_wait_cycles":17519360,"thread_start_us":526877,"threads_started":2,"wal-append.queue_time_us":532213}
I20260709 13:01:41.647054  6059 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:41.897006  6251 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:41.889914  6228 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:41.898830  5578 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:41.644538  6123 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:41.898684  6228 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.899636  6123 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:41.899984  6228 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:41.574319  6237 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:41.901831  6123 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:41.902355  6160 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.765617  5922 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:41.906037  6171 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:41.766683  6070 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:41.906888  6160 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:41.644910  5837 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.574307  6248 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.274s
I20260709 13:01:41.690510  6128 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:41.575266  6244 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:41.644538  6219 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.125s
I20260709 13:01:41.771929  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1794) took 1242 ms. Trace:
W20260709 13:01:41.772323  1583 connection.cc:256] RPC call timeout handler was delayed by -1.1603061600000046s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
W20260709 13:01:41.925885  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.776541  1507 connection.cc:256] RPC call timeout handler was delayed by -1.6620797749999952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1676s for a response
W20260709 13:01:41.927137  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.927564  1507 connection.cc:256] RPC call timeout handler was delayed by -1.4591591210000274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1666s for a response
W20260709 13:01:41.928143  1507 connection.cc:256] RPC call timeout handler was delayed by -1.2023130110000011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1768s for a response
W20260709 13:01:41.928571  1507 connection.cc:256] RPC call timeout handler was delayed by -1.1780130110000187s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
W20260709 13:01:41.752365  1539 rpcz_store.cc:269] 0709 13:01:40.187528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.189246 (+  1718us) service_pool.cc:224] Handling call
0709 13:01:41.751623 (+1562377us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b54003458d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:41.751779 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1861766912}
I20260709 13:01:41.828742  1543 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
W20260709 13:01:41.837112  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 223252us
I20260709 13:01:41.575392  6227 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:41.578048  6075 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:41.719043  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 69855.9us
I20260709 13:01:41.935608  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:41.936319  1508 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:41.937064  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:41.649734  6195 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.579702  6099 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:41.872638  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 311167us
I20260709 13:01:41.939786  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:41.760041  6132 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 13:01:41.908528  6245 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:41.911407  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 192143us
I20260709 13:01:41.892505  6207 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:41.965723  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:41.788131  6240 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.924816  1613 rpcz_store.cc:276] 0709 13:01:40.529409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.541510 (+ 12101us) service_pool.cc:224] Handling call
0709 13:01:41.771664 (+1230154us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b54003c87d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:41.771847 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":963099008}
I20260709 13:01:41.932569  1538 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:41.929353  1507 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:41.947037  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:41.947439  1509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.013s	sys 0.020s
I20260709 13:01:41.955725  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:41.956256  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:41.998759  1509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.013s	sys 0.026s
W20260709 13:01:41.914403  6185 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.783s	user 0.003s	sys 0.001s
I20260709 13:01:41.782958  5900 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:01:41.917654  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 408860us
W20260709 13:01:42.002907  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:41.916954  6059 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:41.965459  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:41.999253  1614 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:41.996840  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:42.014139  1610 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.015548  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:41.973371  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:42.016407  2234 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:41.931213  5929 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:42.001305  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:41.935040  6075 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:41.939139  6194 consensus_meta.cc:287] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.970s	user 0.013s	sys 0.000s
I20260709 13:01:41.962375  6070 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:41.964876  6193 consensus_meta.cc:287] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.732s	user 0.007s	sys 0.004s
I20260709 13:01:41.995991  6144 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:41.968444  6171 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:41.946951  6195 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:41.959604  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:42.013756  6059 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:41.873543  6234 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:41.962666  6123 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:01:41.968623  6099 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.061651  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:41.968185  6109 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:01:41.998931  6240 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:42.027475  6210 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.003s
I20260709 13:01:42.021126  5900 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:42.027892  6075 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.052351  6194 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.075616  5951 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:42.077057  2233 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:42.078133  6167 raft_consensus.cc:3055] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:42.077979  5900 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.078711  6167 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:42.082088  6130 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:42.082836  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 925) took 1083 ms. Trace:
I20260709 13:01:42.083374  1685 rpcz_store.cc:276] 0709 13:01:40.999777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.071503 (+ 71726us) service_pool.cc:224] Handling call
0709 13:01:41.071807 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.166092 (+ 94285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.082491 (+916399us) spinlock_profiling.cc:243] Waited 89.7 ms on lock 0x7b5400355054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:42.082756 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":206281088}
I20260709 13:01:42.085664  2229 raft_consensus.cc:2410] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 13:01:42.066179  6171 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:42.087162  6171 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.066650  6246 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:01:42.069119  6256 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:01:42.091604  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:42.092182  6234 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:42.092504  2234 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:42.094023  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:42.094795  2234 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:42.065924  6252 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:41.964053  6227 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.085449  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:42.104291  2228 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:42.057432  6059 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.071986  6263 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:42.062635  6123 consensus_queue.cc:1243] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:42.060565  5793 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:41.958277  6244 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.103253  5837 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:42.092229  6194 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:42.095170  6210 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.210783  6231 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:42.216727  6227 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:42.210109  6245 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:42.216646  6231 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.238930  6244 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:42.240706  6210 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:42.317289  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 390554us
W20260709 13:01:42.318846  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.319067  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:42.321934  6132 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.323308  6132 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:42.324841  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:41.966353  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.020421  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.017688  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:42.001442  1609 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:42.029932  1610 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:42.030093  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:42.477945  6244 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.478803  6244 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:42.059154  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 105708us
W20260709 13:01:42.310802  1618 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.010s	sys 0.041s
W20260709 13:01:42.490041  1618 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.011s	sys 0.041s
I20260709 13:01:42.490823  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:42.491880  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2286) took 1002 ms. Trace:
W20260709 13:01:42.000377  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 71158.8us
I20260709 13:01:42.325922  5956 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.332314  6275 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.340229  6245 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.320927  2234 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:01:42.451031  6206 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:01:42.451413  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:42.461272  1617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.000s	sys 0.007s
I20260709 13:01:42.467841  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.468005  6132 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:42.467626  6210 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.471705  1538 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
W20260709 13:01:42.471609  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 150575us
I20260709 13:01:42.472463  6240 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:42.499078  6275 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:42.499593  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:42.497222  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.512964  1543 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:42.492601  1618 rpcz_store.cc:276] 0709 13:01:41.489589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.563470 (+ 73881us) service_pool.cc:224] Handling call
0709 13:01:41.563915 (+   445us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:41.564353 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:41.564399 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:41.573800 (+  9401us) log.cc:844] Serialized 52 byte log entry
0709 13:01:42.491112 (+917312us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:42.491269 (+   157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.491432 (+   163us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:42.491546 (+   114us) raft_consensus.cc:1685] finished
0709 13:01:42.491564 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.491774 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5471,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3357,"thread_start_us":761423,"threads_started":1,"wal-append.queue_time_us":571266}
I20260709 13:01:42.514043  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:42.508625  1583 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:42.515406  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:42.516176  1583 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:42.509466  6240 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:42.514066  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:42.511284  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.511684  6275 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:42.523073  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.524047  1543 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 8.
I20260709 13:01:42.500185  6244 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:42.527381  6122 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:42.533766  1507 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:42.536000  6252 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:42.536406  1507 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:42.519050  6132 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:42.537670  6122 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:42.518718  5956 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:42.537911  6275 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:42.506393  6245 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.539161  6275 consensus_queue.cc:1243] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 8a6a5ae1b7f240408b3ddd30d8a18262 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:01:42.540633  5956 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:42.540887  6245 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:42.518540  6128 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:42.541642  6128 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:42.542234  6128 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.542466  6240 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:42.525333  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.500790  6206 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:42.546578  1543 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:42.546977  6240 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:01:42.547202  6206 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:42.547924  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.548754  6206 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.549105  1543 raft_consensus.cc:2410] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:01:42.549290  6263 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:42.549925  6263 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:42.517967  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.510390  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2278) took 1041 ms. Trace:
I20260709 13:01:42.538308  6122 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.536926  6252 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:42.577767  6252 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.578295  6252 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:42.584156  6244 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:42.585777  6227 raft_consensus.cc:3055] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:42.516676  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:42.551517  6263 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:42.586455  6227 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:42.556744  1617 rpcz_store.cc:276] 0709 13:01:41.468790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.468948 (+   158us) service_pool.cc:224] Handling call
0709 13:01:41.469313 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.469639 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:41.935127 (+465488us) log.cc:844] Serialized 14 byte log entry
0709 13:01:42.509755 (+574628us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:42.509849 (+    94us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:42.509954 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.510086 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.510312 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":217124,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":31,"raft.queue_time_us":1426,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":1028142,"threads_started":2,"wal-append.queue_time_us":525182}
I20260709 13:01:42.559530  6210 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.561164  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:42.561378  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:42.562088  6144 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:42.563577  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:42.596638  1543 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:42.596304  6227 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.598193  6227 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:01:42.562832  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 137966us
I20260709 13:01:42.565932  1539 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:42.570036  6059 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.592142  6263 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.606004  5929 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.606786  5929 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:42.624399  6210 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:42.627732  2236 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:42.628146  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 67289.5us
I20260709 13:01:42.628816  6059 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:42.635676  6144 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:42.637238  1609 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:42.639943  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:42.640901  1539 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:42.641584  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 75299.3us
I20260709 13:01:42.642567  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:42.632625  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:42.647579  2232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1254) took 1665 ms (client timeout 1918 ms). Trace:
I20260709 13:01:42.647919  1687 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 13:01:42.648214  2232 rpcz_store.cc:269] 0709 13:01:40.982360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.090262 (+107902us) service_pool.cc:224] Handling call
0709 13:01:42.647053 (+1556791us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54004bb254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:42.647345 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1457666944}
I20260709 13:01:42.650672  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:42.651973  1691 raft_consensus.cc:2410] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
W20260709 13:01:42.655153  2228 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.550s	user 0.007s	sys 0.007s
I20260709 13:01:42.655238  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:42.655843  2228 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:42.656422  1685 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:42.609396  5929 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:42.677485  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 45254.5us
I20260709 13:01:42.679208  1507 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:42.679867  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.681042  6283 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:42.682405  6203 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:42.682816  1537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2236) took 1771 ms (client timeout 1818 ms). Trace:
I20260709 13:01:42.682996  6203 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:42.683525  6203 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.683683  6283 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:42.684204  6283 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.684614  6283 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:42.683359  1537 rpcz_store.cc:269] 0709 13:01:40.911068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.960096 (+ 49028us) service_pool.cc:224] Handling call
0709 13:01:42.645944 (+1685848us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b5400419bd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:42.682582 (+ 36638us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365967616}
I20260709 13:01:42.687129  6095 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:42.687300  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:42.688486  1537 raft_consensus.cc:2368] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10.
I20260709 13:01:42.689644  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:42.690511  1537 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.682405  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:42.676362  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:42.694483  2234 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
W20260709 13:01:42.695521  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.696123  2234 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:01:42.699673  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:42.693822  1543 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:42.702874  2232 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:42.701355  6128 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.710733  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:42.711771  1685 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:42.712189  6128 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:42.714737  2231 pending_rounds.cc:85] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 9
I20260709 13:01:42.715137  2231 pending_rounds.cc:107] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:01:42.715423  2231 raft_consensus.cc:2889] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:42.718060  2234 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:01:42.718982  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2228) took 2720 ms. Trace:
I20260709 13:01:42.719528  1541 rpcz_store.cc:276] 0709 13:01:39.998029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.998177 (+   148us) service_pool.cc:224] Handling call
0709 13:01:39.998466 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.868803 (+870337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.718404 (+1849601us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54000ec9d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:42.718747 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1913455872}
I20260709 13:01:42.720477  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:42.721560  1686 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
W20260709 13:01:42.727397  1614 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.687s	user 0.008s	sys 0.008s
I20260709 13:01:42.721159  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.727986  1614 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:42.728509  2228 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:42.730005  1541 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:42.731670  6207 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.734654  1535 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
W20260709 13:01:42.736027  1535 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 853) took 1494 ms (client timeout 1938 ms). Trace:
W20260709 13:01:42.736440  1535 rpcz_store.cc:269] 0709 13:01:41.242032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.404426 (+162394us) service_pool.cc:224] Handling call
0709 13:01:42.735822 (+1331396us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54000ed3d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:42.735970 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3019764608}
I20260709 13:01:42.733793  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:42.743386  2233 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 882) took 1980 ms (client timeout 2069 ms). Trace:
W20260709 13:01:42.744474  6195 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.682s	user 0.014s	sys 0.004s
W20260709 13:01:42.720767  6075 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.620s	user 0.016s	sys 0.002s
I20260709 13:01:42.728166  6167 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:42.743999  2233 rpcz_store.cc:269] 0709 13:01:40.763139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.763452 (+   313us) service_pool.cc:224] Handling call
0709 13:01:42.742988 (+1979536us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400484ed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:42.743175 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2873453824}
I20260709 13:01:42.734184  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.747138  1543 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:42.733330  6207 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:42.751344  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:42.752343  2234 raft_consensus.cc:2368] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14.
I20260709 13:01:42.744972  6195 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.757611  2228 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:42.758671  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.759997  1541 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:42.763142  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:42.764036  1541 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:42.764389  6122 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:42.765751  6033 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:42.767661  6075 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:41.745764   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:42.797078  1687 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:01:42.797785  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:42.717202  6099 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:42.772997  1541 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:42.778288  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:42.779507  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:42.781541  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 921) took 1951 ms. Trace:
I20260709 13:01:42.785964  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:42.786484  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:42.807554  1691 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.788719  1539 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:42.789361  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:42.820856  2233 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:42.792687  6033 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:42.803913  1657 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:42.820401  1440 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:42.827073  1537 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:42.799762  6128 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.804914  6167 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:42.838486  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:42.838922  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:42.839375  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 318566us
I20260709 13:01:42.841063  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:42.842742  1692 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.846941  1537 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:42.848500  1686 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:42.849815  1687 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:42.824926  6195 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:42.851663  1535 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:42.818713  6227 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:01:42.857617  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.807512  1684 rpcz_store.cc:276] 0709 13:01:40.830113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.963267 (+133154us) service_pool.cc:224] Handling call
0709 13:01:40.963665 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.061506 (+ 97841us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:01:41.093982 (+ 32476us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.175967 (+ 81985us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.176028 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:41.176448 (+   420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.781005 (+1604557us) spinlock_profiling.cc:243] Waited 97.5 ms on lock 0x7b54001dbfd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:42.781067 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.781346 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33834,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":55213,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":14471,"spinlock_wait_cycles":224245120,"thread_start_us":81629,"threads_started":1,"wal-append.queue_time_us":231318,"wal-append.run_cpu_time_us":2609,"wal-append.run_wall_time_us":1329655}
I20260709 13:01:42.829773  6244 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.863656  6198 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.868538  6198 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.871714  6198 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:42.873639  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:42.873957  6212 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:42.874310  1541 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:42.874498  6212 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.877316  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.878425  1686 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:01:42.880378  1686 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:42.881441  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.881637  6205 raft_consensus.cc:3055] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:42.882161  6232 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:42.882416  6205 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:42.883162  6033 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.884845  6033 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.884071  6205 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.886415  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 855) took 1629 ms. Trace:
I20260709 13:01:42.865540  2228 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:42.881815  6244 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:42.874727  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2669) took 3625 ms. Trace:
I20260709 13:01:42.868073  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:42.886994  1536 rpcz_store.cc:276] 0709 13:01:41.256681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.896053 (+639372us) service_pool.cc:224] Handling call
0709 13:01:41.896343 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.313050 (+416707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:42.313130 (+    80us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:42.313220 (+    90us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:42.313272 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.885783 (+572511us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54003e1f54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:42.885870 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.886152 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":956586752}
W20260709 13:01:42.863785  2203 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.602s (SENT) [suppressed 23 similar messages]
I20260709 13:01:42.893123  1685 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:42.896755  1541 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:42.897393  6215 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:42.899842  1691 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:42.897665  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:42.903700  1583 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:01:42.906013  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 60829.8us
I20260709 13:01:42.866506  5929 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:42.908188  1509 leader_election.cc:336] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.020s (SENT)
I20260709 13:01:42.910126  1541 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:42.917207  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 917) took 2142 ms. Trace:
I20260709 13:01:42.909461  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.864135  6128 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.913774  6070 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:42.920127  1689 rpcz_store.cc:276] 0709 13:01:40.774338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.774458 (+   120us) service_pool.cc:224] Handling call
0709 13:01:40.774731 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.648244 (+873513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.916579 (+1268335us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b540028f0d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:42.916978 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1840322432}
I20260709 13:01:42.922055  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:42.927333  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 133958us
I20260709 13:01:42.890986  1690 rpcz_store.cc:276] 0709 13:01:39.248761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:39.351642 (+102881us) service_pool.cc:224] Handling call
0709 13:01:40.190512 (+838870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:40.190737 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:40.643579 (+452842us) log.cc:844] Serialized 14 byte log entry
0709 13:01:41.029727 (+386148us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:41.029780 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:41.029833 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:41.029903 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:42.874259 (+1844356us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b54000744d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:42.874493 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":429354,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":4199,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":18210,"spinlock_wait_cycles":1926144384,"thread_start_us":837633,"threads_started":2,"wal-append.queue_time_us":748140,"wal-append.run_cpu_time_us":1785,"wal-append.run_wall_time_us":1117675}
I20260709 13:01:42.882902  6232 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:42.939566  1535 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:42.940654  1539 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:42.939674  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:42.947600  6129 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:42.948135  6129 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:42.948546  6129 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:42.950812  6128 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:42.951575  6128 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:42.952035  6128 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:42.926647  6070 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:42.904908  6252 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.955556  1536 raft_consensus.cc:2410] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:01:42.934566  5956 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:42.966320  1692 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:42.966601  6232 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:42.957198  2228 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:42.968516  6203 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:42.969614  5968 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:42.970476  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:42.937635  5929 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.966503  1534 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 13:01:42.974833  6051 raft_consensus.cc:3055] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:42.975296  6051 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:42.976292  6051 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.953066  6128 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:42.960125  6122 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:42.987752  1538 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:42.986135  1657 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:42.990514  1537 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:42.982853  6051 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:42.994035  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 94765.2us
I20260709 13:01:42.995790  1543 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:43.003355  1537 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 13 index: 13. (term mismatch)
I20260709 13:01:42.996542  6205 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:43.003949  1537 pending_rounds.cc:85] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 12
I20260709 13:01:42.995400  6144 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:01:43.003476  6279 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:01:43.011685  6127 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:01:43.014402  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 109134us
I20260709 13:01:43.010520  1537 pending_rounds.cc:107] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 12.13
I20260709 13:01:43.016320  1537 raft_consensus.cc:2889] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:42.971925  6123 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.147s
W20260709 13:01:43.018616  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 90195.2us
I20260709 13:01:43.008280  6122 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:43.030216  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 82975.9us
I20260709 13:01:43.021507  6127 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.037874  6127 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:43.018631  6123 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:43.038651  6123 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:43.039913  6123 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:43.041677  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.048166  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 194284us
I20260709 13:01:43.049747  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:43.050652  1537 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:43.054736  1440 catalog_manager.cc:5697] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 7 to 11, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:01:43.056227  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:43.057093  1540 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
W20260709 13:01:43.057605  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.058627  1540 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 13:01:43.058992  1540 pending_rounds.cc:85] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 11
I20260709 13:01:43.059252  1540 pending_rounds.cc:107] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:01:43.060290  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 887) took 2290 ms. Trace:
I20260709 13:01:43.056625  1440 catalog_manager.cc:5697] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:43.060950  2237 rpcz_store.cc:276] 0709 13:01:40.769248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.772369 (+  3121us) service_pool.cc:224] Handling call
0709 13:01:40.772840 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:41.165585 (+392745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.059778 (+1894193us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b5400472a54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:43.060091 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":877491200}
I20260709 13:01:43.060369  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 933) took 1049 ms. Trace:
I20260709 13:01:43.062686  1688 rpcz_store.cc:276] 0709 13:01:42.010926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.011211 (+   285us) service_pool.cc:224] Handling call
0709 13:01:42.011581 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.036693 (+ 25112us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:42.561617 (+524924us) log.cc:844] Serialized 14 byte log entry
0709 13:01:42.853531 (+291914us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:42.861751 (+  8220us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:42.896522 (+ 34771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.059931 (+163409us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b540003f054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:43.059997 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.060194 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5708,"prepare.queue_time_us":212348,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":11455,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4603,"spinlock_wait_cycles":57013632,"thread_start_us":699253,"threads_started":2,"wal-append.queue_time_us":433768}
I20260709 13:01:43.063504  1440 catalog_manager.cc:5697] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 6 to 11, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 11 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:01:43.059530  1540 raft_consensus.cc:2889] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:43.069227  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 390211us
W20260709 13:01:43.070698  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 66220.8us
W20260709 13:01:43.077641  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.047041  6123 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.080261  6252 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.047243  6244 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:43.084641  1540 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:01:43.085492  6051 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.098547  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:43.099493  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:43.099833  1688 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:43.101284  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:43.102288  1584 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:43.102288  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:43.102648  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:43.103996  1509 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:43.107550  1690 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:01:43.107396  6051 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.107934  6286 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:43.107550  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2768) took 3082 ms. Trace:
I20260709 13:01:43.102607  6103 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.008s
I20260709 13:01:43.110057  6286 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:43.110526  6103 consensus_queue.cc:1243] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 13.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:01:43.110913  1583 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:43.106237  6203 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:43.111558  6051 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:43.110077  1542 rpcz_store.cc:276] 0709 13:01:40.025273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:40.026434 (+  1161us) service_pool.cc:224] Handling call
0709 13:01:40.027816 (+  1382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:40.028193 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:40.028235 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:42.641101 (+2612866us) spinlock_profiling.cc:243] Waited 9.06 ms on lock 0x7b2c00056c38. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:01:42.641803 (+   702us) log.cc:844] Serialized 26 byte log entry
0709 13:01:42.891427 (+249624us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:42.898915 (+  7488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:42.922516 (+ 23601us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:43.083314 (+160798us) raft_consensus.cc:1685] finished
0709 13:01:43.083343 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.107331 (+ 23988us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20830592,"thread_start_us":249069,"threads_started":1,"wal-append.queue_time_us":353232}
I20260709 13:01:43.108023  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:43.112675  1657 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:43.114080  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:43.123226  5956 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:43.123775  1688 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:43.130081  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.130795  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.131166  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.132452  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 44454.8us
I20260709 13:01:43.134267  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:43.110607  6286 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:43.078820  6129 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.109474  6283 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.139411  6275 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:43.139953  6275 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:43.140359  6275 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:43.141793  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:43.144721  1542 raft_consensus.cc:2410] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 16 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:01:43.139460  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:43.146615  1540 raft_consensus.cc:2368] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13.
I20260709 13:01:43.123749  5956 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:43.148047  1657 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:43.141358  6283 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:43.147089  6263 consensus_meta.cc:287] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.539s	user 0.016s	sys 0.000s
I20260709 13:01:43.152774  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:43.154501  1542 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:01:43.108369  6244 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
W20260709 13:01:43.157557  1582 connection.cc:256] RPC call timeout handler was delayed by -1.3155112740000163s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
W20260709 13:01:43.158102  1582 connection.cc:256] RPC call timeout handler was delayed by -1.1558880409999688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19640000000000002s for a response
W20260709 13:01:43.158625  1582 connection.cc:256] RPC call timeout handler was delayed by -1.05971792400004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
W20260709 13:01:43.158972  1582 connection.cc:256] RPC call timeout handler was delayed by -1.0534816080000269s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3186s for a response
W20260709 13:01:43.159276  1582 connection.cc:256] RPC call timeout handler was delayed by -1.04425340399996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25579999999999997s for a response
I20260709 13:01:43.159346  6129 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.147876  6171 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:43.160378  6171 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:43.160950  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:43.161549  6224 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:43.161960  6288 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:01:43.162566  6291 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:43.168612  6291 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.169121  6291 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:43.143563  6051 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.170811  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 1111) took 1205 ms. Trace:
I20260709 13:01:43.171520  6099 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:43.172238  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2240) took 1172 ms. Trace:
I20260709 13:01:43.173178  1692 raft_consensus.cc:2410] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:01:43.162626  6171 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:43.133249  6203 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:43.166891  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.171212  2230 rpcz_store.cc:276] 0709 13:01:41.965265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.965507 (+   242us) service_pool.cc:224] Handling call
0709 13:01:43.170425 (+1204918us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b54006e78d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:43.170619 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":787031808}
I20260709 13:01:43.172652  1538 rpcz_store.cc:276] 0709 13:01:41.999751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.560967 (+561216us) service_pool.cc:224] Handling call
0709 13:01:43.172043 (+611076us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54001fccd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:43.172176 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":896615680}
I20260709 13:01:43.181012  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:43.185617  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:43.157605  1584 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:43.188612  6215 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:43.146508  6252 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.149866  5956 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.197170  2232 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [12.11-15.12]
W20260709 13:01:43.195274  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 50624.9us
I20260709 13:01:43.197324  5773 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:43.185510  6128 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:43.199169  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:43.202515  1617 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:43.199932  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:43.201843  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:43.201776  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:43.202670  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:43.214080  1657 leader_election.cc:400] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14.
I20260709 13:01:43.214476  1657 leader_election.cc:403] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:01:43.206111  6263 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:43.220613  1618 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:43.215893  5773 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:43.222942  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 62100.3us
I20260709 13:01:43.223955  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:43.223819  5773 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.208003  6051 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.207515  6224 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:43.225412  1507 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:43.216293  6206 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:43.225464  6244 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:43.218364  6198 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:43.262418  6198 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.263451  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:43.264092  6193 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:43.264611  6193 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.264217  1614 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:43.264968  6193 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:43.265533  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:43.266168  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 148488us
I20260709 13:01:43.267760  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:43.268837  5995 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:43.205570  6203 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:43.271430  6203 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:43.227408  6051 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:43.229555  6252 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.278764  5733 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:43.279304  5733 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.279671  5733 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.281340  6263 raft_consensus.cc:697] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:43.219277  6215 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:43.282595  6215 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:43.282523  6263 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.230775  6075 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.285569  6128 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.236160  6212 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:43.242715  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:43.287619  1618 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:43.288065  6128 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.246320  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:43.246623  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.254078  1617 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:43.252835  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1108) took 1288 ms. Trace:
I20260709 13:01:43.239331  1609 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:43.244439  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:43.291839  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 98335.6us
I20260709 13:01:43.287891  1440 catalog_manager.cc:5697] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 4 to 8, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 8 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:43.246711  2201 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:43.290576  1542 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:01:43.294137  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 47606.9us
I20260709 13:01:43.290798  2236 rpcz_store.cc:276] 0709 13:01:41.964447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:41.964579 (+   132us) service_pool.cc:224] Handling call
0709 13:01:41.964855 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.628428 (+663573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.252550 (+624122us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b5400700154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:43.252755 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1523401472}
I20260709 13:01:43.292816  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:43.299417  1612 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:43.247615  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:43.247185  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:43.301967  6103 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:01:43.303463  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:43.303689  6103 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.304270  6103 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:43.304353  1618 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.247452  6224 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.288292  6122 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:43.302254  6212 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:43.307400  6212 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.307804  6212 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:43.254125  5968 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:43.309178  5968 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:43.310807  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:43.311295  6232 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:43.311685  6122 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:43.311920  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:43.312309  6122 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:43.312170  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.317360  1611 raft_consensus.cc:2410] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 13:01:43.302683  1613 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:43.317711  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:43.319578  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 111775us
I20260709 13:01:43.320772  1610 raft_consensus.cc:2410] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:01:43.326406  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:43.328243  1614 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:43.220623  6288 raft_consensus.cc:3055] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:43.298830  6198 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.337010  1618 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:43.249792  6052 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 13:01:43.298247  6128 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.298398  6075 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Leader election lost for term 13. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14.
I20260709 13:01:43.339046  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:43.346163  6099 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:43.323788  6122 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:43.348327  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.311729  6232 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:43.336810  6288 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:43.351398  5989 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:01:43.405583  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 259120us
I20260709 13:01:43.411306  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:43.384431  6244 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:43.427364  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 47797.3us
I20260709 13:01:43.393844  6167 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:43.437903  6203 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.402806  6275 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.438638  6203 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.395391  5989 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:43.441681  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:43.456149  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 166997us
I20260709 13:01:43.453408  2203 leader_election.cc:400] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10.
I20260709 13:01:43.461447  2203 leader_election.cc:403] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:01:43.462595  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:43.380105  6128 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:43.453976  6299 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
W20260709 13:01:43.473950  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 138067us
I20260709 13:01:43.454147  6051 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:43.400779  6075 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:01:43.441361  6275 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.476994  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2247) took 1143 ms. Trace:
I20260709 13:01:43.477407  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:43.477561  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 152411us
I20260709 13:01:43.441414  6203 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.446231  6171 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:01:43.413483  6070 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.459473  5989 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:43.472970  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:43.480682  1539 rpcz_store.cc:276] 0709 13:01:42.333165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.974928 (+641763us) service_pool.cc:224] Handling call
0709 13:01:42.975282 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:42.975576 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:01:43.106227 (+130651us) log.cc:844] Serialized 14 byte log entry
0709 13:01:43.476607 (+370380us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:43.476669 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:43.476721 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.476791 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.476922 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":202599,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":66,"raft.queue_time_us":9180,"thread_start_us":459307,"threads_started":2}
I20260709 13:01:43.452744  5885 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:01:43.474089  6167 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:43.478441  6099 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:01:43.488310  1440 catalog_manager.cc:5697] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 1 to 9, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 9 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:01:43.483098  6205 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:01:43.441862  6288 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.447521  5956 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:43.515118  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 52661.1us
I20260709 13:01:43.485546  6051 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:43.518299  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.503068  5995 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:43.461452  6232 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.505542  6224 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:43.522862  5773 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:43.545122  5773 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.516054  6052 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:43.526046  6232 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:43.526790  6127 raft_consensus.cc:3055] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:43.597983  6193 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:43.600276  6193 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:43.601547  6128 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.603469  5956 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.592819  6311 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.604552  6311 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.527185  6224 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.607304  6224 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.607712  5956 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.608314  5956 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.526624  6275 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:01:43.610363  5956 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:43.611276  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 106458us
I20260709 13:01:43.612231  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:43.613193  6205 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:43.587352  6052 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:43.521099  6306 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:01:43.626729  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 66727.6us
I20260709 13:01:43.613392  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:43.630681  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 369300us
I20260709 13:01:43.632956  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2248) took 1299 ms. Trace:
W20260709 13:01:43.635740  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 156952us
I20260709 13:01:43.634461  1534 rpcz_store.cc:276] 0709 13:01:42.384918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.987197 (+602279us) service_pool.cc:224] Handling call
0709 13:01:42.987653 (+   456us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:42.994241 (+  6588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:42.994289 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:43.013555 (+ 19266us) log.cc:844] Serialized 52 byte log entry
0709 13:01:43.134186 (+120631us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:43.134229 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.134292 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:43.632628 (+498336us) raft_consensus.cc:1685] finished
0709 13:01:43.632659 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.632864 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":389,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5138,"thread_start_us":86957,"threads_started":1,"wal-append.queue_time_us":185045}
I20260709 13:01:43.613768  6205 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.593817  6127 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:43.629792  6206 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:43.630795  6275 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.522691  6167 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.651494  6288 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:43.668963  1582 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:43.670202  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 242240us
W20260709 13:01:43.669356  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.616820  6309 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:01:43.660627  6205 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:43.681697  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:43.632093  5968 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:43.668112  6033 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:43.662777  6132 raft_consensus.cc:3055] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:43.671422  6306 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:43.675901  5733 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:43.701223  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:43.682574  6309 consensus_queue.cc:1243] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:01:43.694624  6051 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.707425  5773 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.707826  1582 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:01:43.709589  6286 consensus_meta.cc:287] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
I20260709 13:01:43.711315  6127 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.715104  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2802) took 1706 ms. Trace:
I20260709 13:01:43.730506  1541 rpcz_store.cc:276] 0709 13:01:42.008848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.935712 (+926864us) service_pool.cc:224] Handling call
0709 13:01:42.947889 (+ 12177us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:42.966265 (+ 18376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:42.966297 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:42.966860 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 13:01:43.166233 (+199373us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b180047d100. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:01:43.353912 (+187679us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:43.353962 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:43.354036 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:43.714775 (+360739us) raft_consensus.cc:1685] finished
0709 13:01:43.714798 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:43.715034 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13495,"raft.queue_time_us":16708,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":23504,"spinlock_wait_cycles":11294464,"thread_start_us":187467,"threads_started":1,"wal-append.queue_time_us":571146}
I20260709 13:01:43.715397  6103 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:43.716270  5995 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10.
I20260709 13:01:43.614815  6052 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.717923  6033 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:43.720767  6308 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:01:43.720413  6128 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:43.720420  1440 catalog_manager.cc:5697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 10 to 11, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 11 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:01:43.722476  6291 consensus_meta.cc:287] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.553s	user 0.002s	sys 0.009s
I20260709 13:01:43.667973  6275 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:43.729342  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56923.6us
W20260709 13:01:43.728276  1438 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55368 (request call id 327) took 21406 ms (client timeout 15000 ms). Trace:
W20260709 13:01:43.733515  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 70875.8us
I20260709 13:01:43.736037  6322 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:43.737125  6103 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:43.717073  6132 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:43.742406  6127 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:43.744073  1683 raft_consensus.cc:2393] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:43.728888  6252 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.745148  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:43.725293  6263 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:43.738056  5733 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:43.744925  1438 rpcz_store.cc:269] 0709 13:01:22.316604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:22.316939 (+   335us) service_pool.cc:224] Handling call
0709 13:01:22.569061 (+252122us) spinlock_profiling.cc:243] Waited 97.2 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:01:43.722951 (+21153890us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:34.013636 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:34.014462 (+   826us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:34.014592 (+   130us) write_op.cc:435] Acquired schema lock
0709 13:01:34.014624 (+    32us) tablet.cc:662] Decoding operations
0709 13:01:34.016705 (+  2081us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:34.016831 (+   126us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:34.016874 (+    43us) tablet.cc:685] Acquiring locks for 17 operations
0709 13:01:34.429739 (+412865us) tablet.cc:701] Row locks acquired
0709 13:01:34.429792 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:01:34.430013 (+   221us) write_op.cc:270] Start()
0709 13:01:34.430237 (+   224us) write_op.cc:276] Timestamp: P: 1783602094429931 usec, L: 0
0709 13:01:34.430274 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:01:34.431215 (+   941us) log.cc:844] Serialized 6730 byte log entry
0709 13:01:35.151143 (+719928us) op_driver.cc:464] REPLICATION: finished
0709 13:01:35.343226 (+192083us) write_op.cc:301] APPLY: starting
0709 13:01:35.343322 (+    96us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:35.475364 (+132042us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:35.475486 (+   122us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:35.805554 (+330068us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:35.805956 (+   402us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=34,mrs_lookups=0
0709 13:01:35.806030 (+    74us) write_op.cc:312] APPLY: finished
0709 13:01:35.806491 (+   461us) log.cc:844] Serialized 156 byte log entry
0709 13:01:35.806677 (+   186us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:35.806853 (+   176us) write_op.cc:454] Released schema lock
0709 13:01:35.806935 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:35.807338 (+   403us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:35.807371 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":543796,"reported_tablets":46,"spinlock_wait_cycles":223613696,"child_traces":[["op",{"apply.queue_time_us":191783,"apply.run_cpu_time_us":26302,"apply.run_wall_time_us":464126,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75289,"num_ops":17,"prepare.queue_time_us":743275,"prepare.run_cpu_time_us":38075,"prepare.run_wall_time_us":1055531,"raft.queue_time_us":79422,"raft.run_cpu_time_us":64531,"raft.run_wall_time_us":96427,"replication_time_us":720664,"spinlock_wait_cycles":27648,"thread_start_us":1107331,"threads_started":4,"wal-append.queue_time_us":629032,"wal-append.run_cpu_time_us":21063,"wal-append.run_wall_time_us":1739403}]]}
I20260709 13:01:43.782142  6275 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.748216  6075 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.763617  5968 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:43.769472  6238 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:43.794981  6238 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:43.794668  6132 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.796403  6132 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:43.769706  6128 consensus_queue.cc:1243] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:43.738122  6033 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.772764  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:43.770824  6317 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:43.797633  6317 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.778340  6205 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.779716  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:43.742991  6291 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.800273  6291 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.800987  6291 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:43.780802  5956 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:43.777887  6263 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.734081  6286 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:43.779361  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 71620.7us
I20260709 13:01:43.798009  6317 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:43.795979  6238 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.801888  1615 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:43.798092  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:43.751824  1440 catalog_manager.cc:5697] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 5 to 9, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:01:43.801215  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56151.8us
I20260709 13:01:43.780347  6099 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:43.827682  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:43.801194  5733 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:01:43.800920  6306 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:01:43.855171  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.801470  6324 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:43.847409  5773 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:43.862772  1582 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:43.826453  6275 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:43.847506  5885 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
W20260709 13:01:43.864929  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.864568  1440 catalog_manager.cc:5697] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:43.855279  5989 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:43.867172  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:43.867525  5989 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:43.868223  6304 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.868570  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:43.868783  6304 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:43.869369  2232 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:43.870867  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:43.871217  2228 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:43.871683  2236 raft_consensus.cc:2368] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13.
I20260709 13:01:43.866377  1440 catalog_manager.cc:5697] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 8 to 11, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 11 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:43.783597  6075 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.851401  6238 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.856063  6033 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:43.888305  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:43.889060  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:43.870746  6263 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.893168  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:43.859292  6306 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:43.893498  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:43.894269  1692 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:43.894825  1539 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:43.863008  5773 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.900368  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:43.901600  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:43.904780  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:43.897012  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:43.875406  6275 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.904953  1543 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:43.862694  6099 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:43.905582  2228 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:43.896311  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:43.899353  1540 pending_rounds.cc:85] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 11
I20260709 13:01:43.908613  1540 pending_rounds.cc:107] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:01:43.886117  6324 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:43.909013  1540 raft_consensus.cc:2889] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:43.908682  1683 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1793) took 1256 ms (client timeout 1547 ms). Trace:
I20260709 13:01:43.891110  6033 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:43.891270  6291 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:43.913372  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:43.916198  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:43.916873  1538 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:01:43.914783  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:43.916491  2236 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
W20260709 13:01:43.914561  1683 rpcz_store.cc:269] 0709 13:01:42.643601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:42.649920 (+  6319us) service_pool.cc:224] Handling call
0709 13:01:43.202625 (+552705us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b5800010e98. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:01:43.900062 (+697437us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b54000f6754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:43.900208 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1097703680}
I20260709 13:01:43.898914  6309 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.902940  5956 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:43.915832  1541 pending_rounds.cc:85] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 5
I20260709 13:01:43.925973  1541 pending_rounds.cc:107] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 13.6
I20260709 13:01:43.909240  6275 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:43.926297  1541 raft_consensus.cc:2889] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:43.922987  2228 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:43.924036  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:43.924770  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:43.908056  5773 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:43.920869  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:43.927748  1609 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:43.930979  1539 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:43.931244  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:43.924415  1440 catalog_manager.cc:5697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 7 to 8, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 8 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:43.927755  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:43.928072  1691 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
W20260709 13:01:43.929509  6330 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.077s	user 0.001s	sys 0.003s
I20260709 13:01:43.926119  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:43.888795  6052 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:01:43.933663  2228 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:43.919263  6033 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:43.930117  2230 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:43.940037  1440 catalog_manager.cc:5697] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 7 to 8, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 8 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:43.941598  1618 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:01:43.941579  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 91574.2us
I20260709 13:01:43.928524  6128 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.945083  1609 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 13:01:43.945791  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:43.950984  1538 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:43.951851  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:43.946261  1537 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:43.924484  6263 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:43.938635  6291 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.945295  6052 consensus_queue.cc:1243] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:43.952324  1440 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:43.960300  1543 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:43.942729  5929 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:43.946308  6286 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:43.965112  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:43.963351  6311 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:43.973745  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 125548us
I20260709 13:01:43.976128  2228 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:43.968580  1440 catalog_manager.cc:5697] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:01:43.982251  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:43.947324  5956 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:43.984802  2230 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:43.981402  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:43.966496  6215 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:43.992162  2228 raft_consensus.cc:2410] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:43.951328  6309 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.002032  1440 catalog_manager.cc:5697] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 5 to 12, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 12 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:44.003422  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:43.963830  6263 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.004073  2228 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:43.984633  6205 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:44.035948  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.006634  6324 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:44.040447  6333 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:01:44.041994  1615 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:44.042254  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:44.010514  1609 pending_rounds.cc:85] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 4
I20260709 13:01:44.077167  1609 pending_rounds.cc:107] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:01:44.077582  1609 raft_consensus.cc:2889] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:43.974577  6291 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.085319  1440 catalog_manager.cc:5697] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 10, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 10 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:44.093034  2203 leader_election.cc:400] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13.
I20260709 13:01:44.093505  2203 leader_election.cc:403] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:01:43.959034  6306 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.098749  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:44.104027  2230 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:43.990768  6128 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.106165  1440 catalog_manager.cc:5697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 5 to 10, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 10 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:44.109668  6325 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:44.112564  6335 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:44.114466  5956 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:44.115020  1611 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:44.115191  1613 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:44.116309  6286 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:44.116575  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:44.117177  1617 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:44.118799  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:44.119547  1612 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:44.120724  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:44.006395  6215 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.121820  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:44.123263  6215 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:44.098263  6301 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:01:44.126432  6304 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:44.123328  1616 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:44.116333  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:44.133908  1617 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:44.131544  2232 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:44.142117  2228 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:44.144042  6205 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13.
I20260709 13:01:43.982306  6099 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:44.150061  2230 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 13:01:44.128985  6331 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.231s	user 0.000s	sys 0.001s
I20260709 13:01:44.148000  1440 catalog_manager.cc:5697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:01:44.202489  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.206799  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.209158  2236 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:44.210204  1615 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 13:01:44.210317  1440 catalog_manager.cc:5697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:01:44.214763  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 138921us
I20260709 13:01:44.210954  1615 pending_rounds.cc:85] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 4
I20260709 13:01:44.216586  1615 pending_rounds.cc:107] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:01:44.216907  1615 raft_consensus.cc:2889] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:44.220669  1440 catalog_manager.cc:5697] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 2 to 12, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 12 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:44.220661  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:44.228103  6212 consensus_meta.cc:287] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.920s	user 0.012s	sys 0.004s
I20260709 13:01:44.228763  6212 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:44.228626  6319 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.504s	user 0.013s	sys 0.005s
W20260709 13:01:44.238215  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.233915  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.251431  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.253608  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:44.254598  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 126083us
I20260709 13:01:44.257117  1534 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:01:44.262163  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.264951  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.274387  6337 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.275115  6337 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.275722  6212 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:44.276477  6324 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.277014  6324 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.279114  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.279241  6324 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:44.279796  2230 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:44.290542  2236 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:44.290124  6033 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.293195  6033 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:44.295782  6317 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:44.305037  2236 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:44.301388  6343 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:01:44.331274  6328 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:44.331522  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:44.332451  6313 log.cc:927] Time spent T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.630s	user 0.002s	sys 0.009s
I20260709 13:01:44.332772  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.301973  6337 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:44.310369  2233 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:44.334841  6309 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:44.335398  6309 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:44.335958  6309 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:44.313951  1614 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:44.308730  1687 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:44.331912  6328 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.306022  1583 leader_election.cc:400] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13.
I20260709 13:01:44.310995  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:44.302380  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:44.340426  1583 leader_election.cc:403] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:44.340641  1683 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:44.312408  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:44.309108  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:44.317466  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.271975  6310 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:44.343096  2235 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:44.339701  2236 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:44.337473  6321 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.347522  6212 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.348044  6321 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:44.350459  6033 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.351114  6033 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.351174  6212 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.314320  6275 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:44.354390  6275 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:44.355563  2228 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:44.350572  6321 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:44.361634  2230 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 13:01:44.318142  6288 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:44.365839  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.366252  2234 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:01:44.358562  2201 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.314702  6291 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:44.369709  2230 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:44.324169  6317 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:44.370163  1683 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:44.365669  6288 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:44.371843  6343 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:44.374053  6288 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:44.375159  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 75846.7us
W20260709 13:01:44.379606  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 75393.9us
I20260709 13:01:44.380692  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:44.321803  6167 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.033s	sys 0.037s
W20260709 13:01:44.381613  6167 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.033s	sys 0.037s
I20260709 13:01:44.382109  6212 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:44.382601  2237 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:44.384655  6070 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:44.385073  6291 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13.
I20260709 13:01:44.385361  2234 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:44.386858  1583 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.385915  6309 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:44.390559  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.381026  6075 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:44.391918  2236 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:01:44.393561  6327 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:44.388149  2201 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.388527  2228 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:44.344569  6310 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:44.297410  5773 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:44.399000  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.401736  2228 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:44.403996  1582 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:44.391649  6304 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:44.401192  2201 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.407461  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:44.420332  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:44.400816  6343 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:44.401072  6167 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:44.400013  6317 raft_consensus.cc:2764] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:01:44.407102  5885 consensus_queue.cc:1059] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:01:44.398523  6070 consensus_queue.cc:1059] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 13:01:44.426781  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.399906  6337 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:01:44.391908  6321 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:01:44.420846  6304 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:44.431248  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.422955  6171 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:44.442454  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:44.457403  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:44.426777  6318 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:44.461949  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.462626  1691 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:44.463735  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:44.467005  1657 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.468097  1657 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.413738  6275 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:01:44.445142  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.469556  6075 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:44.470038  2236 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:44.470541  6075 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:44.470971  6075 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.471314  6075 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:44.396826  6310 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.429636  6291 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:44.426332  6167 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:44.446475  6238 consensus_meta.cc:287] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.552s	user 0.014s	sys 0.000s
I20260709 13:01:44.428140  1658 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:44.447883  6316 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.738s	user 0.000s	sys 0.015s
I20260709 13:01:44.443981  6325 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:44.491237  6325 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:44.460373  6171 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.492120  6171 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.429352  2231 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:44.458153  6345 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:44.494733  6345 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.495353  6345 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:44.458242  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.484547  6309 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:01:44.455132  6127 raft_consensus.cc:3055] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:44.498644  6127 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:44.499868  6127 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.458750  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.503512  6311 raft_consensus.cc:3055] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:44.504179  6311 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:44.486992  6167 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.457293  6304 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:44.519137  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.467154  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:44.525516  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.526492  1683 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:44.424448  6343 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.445888  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.537880  2228 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
W20260709 13:01:44.474421  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:44.538538  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 65161.1us
I20260709 13:01:44.517104  6075 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.475615  6352 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:44.477819  6318 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:44.474952  6340 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:44.520586  6193 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.557787  6318 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:44.558308  6193 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.558733  6318 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.560174  6318 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:44.560515  6193 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:44.548753  6075 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:44.505280  6311 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:44.561118  6206 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:44.561975  6311 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.563628  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:44.563931  6167 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.564952  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:44.565603  6129 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.566141  6129 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.568959  6129 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:44.539081  6343 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:44.549521  5989 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:44.561697  6206 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.552378  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:44.573329  1541 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:01:44.589123  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 122023us
I20260709 13:01:44.499887  5773 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:44.587739  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.543033  6127 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:44.594437  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 166487us
I20260709 13:01:44.595837  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:44.596644  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:44.591692  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 50166.3us
I20260709 13:01:44.598026  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:44.590124  1543 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:44.596019  2201 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.599085  1535 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:44.492625  6325 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.597062  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.598330  1657 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.602684  1690 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:44.606544  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.607298  2228 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:01:44.571494  6291 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:44.598126  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:44.612005  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.606465  1657 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.585740  6238 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.614694  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:44.616142  1541 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:44.619105  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.620114  6122 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:44.620674  6122 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.621130  6122 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:44.633821  6311 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.634637  6311 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.638000  6311 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:44.591647  6224 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.589437  6309 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.651486  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:44.653561  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.653905  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:44.652383  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:44.656876  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.665163  1541 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:01:44.571957  6337 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:01:44.605146  5885 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:44.700644  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:44.705307  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:44.706598  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:44.707392  1508 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.708642  1508 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:44.708945  1613 raft_consensus.cc:2368] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13.
I20260709 13:01:44.706051  1542 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
W20260709 13:01:44.710124  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 114340us
I20260709 13:01:44.654477  6238 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.652093  6129 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:44.610891  6103 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:44.611263  6275 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:44.712473  1543 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:44.713222  1536 raft_consensus.cc:2410] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:44.716475  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:44.718369  1538 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:44.719175  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:44.720194  1541 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:44.606184  6070 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 13:01:44.725323  1535 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:44.605314  6232 consensus_queue.cc:1059] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 13:01:44.768096  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.730922  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:44.770927  1536 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:44.772909  1542 pending_rounds.cc:85] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 10
I20260709 13:01:44.773571  1542 pending_rounds.cc:107] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:01:44.774128  1542 raft_consensus.cc:2889] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:44.694154  6033 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:44.650784  5968 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:44.715312  6193 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:44.731317  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.708146  6167 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:44.707288  6099 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:44.730316  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.777957  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.778705  1541 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 13:01:44.780092  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.708588  5885 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.759788  1539 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:01:44.716535  6311 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
W20260709 13:01:44.730618  6338 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:01:44.702644  6224 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.798421  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.802316  1613 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:44.768096  6275 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:44.805683  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.783012  5968 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:44.793080  6340 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:44.820545  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.822005  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.821758  6371 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:44.822350  1686 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:44.786722  6193 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:44.819792  6305 raft_consensus.cc:3055] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:44.820035  6224 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:44.808632  5968 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:44.853929  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.854045  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:44.821817  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.795032  6317 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.785265  6075 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:01:44.777406  6122 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.844538  6193 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:44.852061  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:44.801146  5885 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:44.855038  1683 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:44.855042  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:44.808588  6320 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:44.732046  6309 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:44.860850  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.816370  6368 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.873123  1692 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:44.862540  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:44.864579  1614 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:44.867084  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:44.867365  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:44.876400  1609 raft_consensus.cc:2368] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10.
W20260709 13:01:44.876364  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 167772us
I20260709 13:01:44.877794  1687 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:44.821347  6340 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.878217  1689 raft_consensus.cc:2368] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
I20260709 13:01:44.849377  6305 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:44.878801  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.876605  1685 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:44.878919  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:44.802119  6099 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.862038  6224 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:01:44.874887  6351 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:01:44.881372  2236 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:44.884878  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:44.891593  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:44.892679  1609 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:44.893590  1618 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:44.894465  1691 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:44.893718  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.897902  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.898121  1615 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:44.874915  6320 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.911204  1613 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:44.911561  6129 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:44.912864  6129 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.918473  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:44.920897  6375 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.921612  6375 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:44.924688  6375 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:44.886179  6206 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:44.929844  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:44.919358  2235 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:44.887713  6224 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:44.868122  6317 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.932816  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:44.930809  1611 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:44.936092  1689 raft_consensus.cc:2393] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 9: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:44.911100  6320 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:44.940050  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:44.940649  1685 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:44.941154  1613 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:44.942884  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:44.952859  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 70062.9us
I20260709 13:01:44.935662  6224 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:44.955724  1614 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:44.932505  6212 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:44.968957  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:44.956684  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:44.889200  6275 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:44.975657  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:44.894192  6368 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:44.984997  6364 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.130s	user 0.002s	sys 0.000s
I20260709 13:01:44.985929  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1117) took 1117 ms. Trace:
I20260709 13:01:44.986364  2237 rpcz_store.cc:276] 0709 13:01:43.868912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.869049 (+   137us) service_pool.cc:224] Handling call
0709 13:01:43.869330 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:44.383030 (+513700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.985645 (+602615us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b54006586d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:44.985865 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1179931392}
I20260709 13:01:44.980955  5773 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:01:44.990025  6368 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:44.992502  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:44.993175  6215 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:44.993592  6215 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:44.993762  6327 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:44.994225  6327 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:44.995755  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 1119) took 1125 ms. Trace:
I20260709 13:01:44.996152  2232 rpcz_store.cc:276] 0709 13:01:43.870117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:43.872309 (+  2192us) service_pool.cc:224] Handling call
0709 13:01:44.995571 (+1123262us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b5400530054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:44.995697 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521402752}
I20260709 13:01:45.002403  6309 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:45.004199  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:45.006942  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.008580  1614 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
W20260709 13:01:45.009301  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.010821  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45132.1us
I20260709 13:01:45.011736  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:45.011051  1539 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:45.013392  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:45.014156  1539 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:45.014125  1584 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:45.017136  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:45.020455  1536 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:45.023358  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:45.024035  1538 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:45.025832  1535 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:45.028373  6286 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:45.010553  6309 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:45.032267  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:45.011535  6103 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:01:45.032698  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.008893  6238 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:45.035487  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:45.036836  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:45.041052  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:45.045883  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:45.046967  1539 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:45.025008  5773 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:45.046664  1536 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:44.972801  6212 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.048535  6376 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.051584  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:45.018303  6335 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
W20260709 13:01:45.051049  6369 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:01:45.036422  6103 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:45.058573  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:45.059309  1535 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:45.035329  6291 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:45.060063  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:45.057276  5773 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:44.979379  6305 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:45.077860  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.078783  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:45.082110  1689 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:45.055663  6122 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:45.072805  6103 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:45.085958  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:45.086668  1614 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:01:45.088393  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:45.089458  1535 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 9: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:45.095381  6381 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 13:01:45.096252  6212 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:45.096882  6381 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:45.097203  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:45.097721  6381 consensus_queue.cc:1059] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:45.098740  6388 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:45.099586  6388 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:45.099947  6388 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:45.097931  1614 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:45.077096  6291 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:45.105535  6380 consensus_queue.cc:1059] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:01:45.106606  6380 raft_consensus.cc:3055] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:45.106961  6380 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:45.107532  5885 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.107939  6380 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:45.091372  6099 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:45.097361  6320 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.110679  1687 raft_consensus.cc:2368] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
W20260709 13:01:45.114059  6366 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 13:01:45.091073  6391 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:45.092324  6305 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:45.119827  5885 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:45.119150  6320 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:45.126183  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:45.127786  1535 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:45.130432  5773 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:45.135367  6371 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:45.136020  6371 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:45.142993  6372 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
W20260709 13:01:45.143512  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.108168  6291 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:45.157482  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:45.164516  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 877) took 1034 ms. Trace:
I20260709 13:01:45.165040  1534 rpcz_store.cc:276] 0709 13:01:44.129823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.266183 (+136360us) service_pool.cc:224] Handling call
0709 13:01:44.278222 (+ 12039us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:44.571421 (+293199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:44.571482 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:44.572071 (+   589us) log.cc:844] Serialized 26 byte log entry
0709 13:01:44.708597 (+136526us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:44.708683 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.708850 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:45.164241 (+455391us) raft_consensus.cc:1685] finished
0709 13:01:45.164263 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:45.164430 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":292765,"raft.queue_time_us":293363,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":9904,"thread_start_us":92908,"threads_started":1,"wal-append.queue_time_us":175741}
I20260709 13:01:45.109320  6380 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:45.100793  6388 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:45.197853  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:45.200525  1614 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:45.204137  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:45.170666  6127 consensus_meta.cc:287] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.573s	user 0.013s	sys 0.000s
I20260709 13:01:45.206426  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:45.207054  1609 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:45.056808  6376 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.163786  6317 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:45.215770  1612 raft_consensus.cc:2368] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13.
I20260709 13:01:45.217497  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:45.218643  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 214516us
I20260709 13:01:45.227782  1542 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:45.221410  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 168726us
I20260709 13:01:45.224094  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:45.228538  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.228605  1612 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:45.117271  6193 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.229405  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:45.216387  6310 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:45.234220  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 436527us
I20260709 13:01:45.236604  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:45.291636  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:45.292428  1617 raft_consensus.cc:2368] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10.
W20260709 13:01:45.291249  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 54707.4us
I20260709 13:01:45.294847  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:45.295835  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:45.297508  1538 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:45.232880  6288 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:45.305059  1609 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:45.137233  6371 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.306926  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:45.308249  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.307699  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:45.242223  6317 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:45.312176  1689 raft_consensus.cc:2368] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10.
I20260709 13:01:45.310362  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:45.310716  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:45.306602  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:45.313838  1686 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:45.315474  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:45.313978  1609 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:45.315970  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:45.322296  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:45.316037  2237 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:45.323571  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:45.296952  6309 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:45.315491  2233 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:45.321053  1535 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:45.329674  1657 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:45.306710  5989 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:45.338881  1617 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:45.307703  6311 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:45.377247  1685 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:45.290410  6320 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 13:01:45.297116  6387 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:01:45.389398  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:45.322619  6328 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:45.436633  1535 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:01:45.388670   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:01:45.380884  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.397756  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.380013  6193 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:45.332715  6317 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:45.378446  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:45.442219  6317 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.378986  6371 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:45.423092  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:45.453684  1617 raft_consensus.cc:2376] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11. Re-sending same reply.
I20260709 13:01:45.389022  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:45.315608  6390 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.490136  6390 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.492844  6390 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:45.494165  1691 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:45.497011  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.344766  6304 consensus_meta.cc:287] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.826s	user 0.011s	sys 0.002s
I20260709 13:01:45.522665  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:45.522504  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:45.524786  1535 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
W20260709 13:01:45.386082  6384 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:01:45.318859  6327 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.581718  1689 raft_consensus.cc:2410] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:45.566983  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:45.582612  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:45.334146  6309 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.422137  6122 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:45.566774  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:45.586910  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:45.397286  5989 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:45.394613  6388 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:45.378296  6311 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.601092  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:45.602628  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:45.565209  6345 consensus_meta.cc:287] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.069s	user 0.006s	sys 0.009s
I20260709 13:01:45.622351  1686 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:45.598140  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:45.583702  5968 consensus_meta.cc:287] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.716s	user 0.008s	sys 0.004s
W20260709 13:01:45.624425  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 269256us
I20260709 13:01:45.629755  1691 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:45.629824  1657 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:45.628216  1535 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:45.632782  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:45.565339  6171 consensus_meta.cc:287] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.072s	user 0.016s	sys 0.000s
W20260709 13:01:45.541227  6318 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.981s	user 0.014s	sys 0.002s
I20260709 13:01:45.642606  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:45.639586  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:45.653944  1535 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:45.629043  6132 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:45.594074  6304 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.664366  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:45.668195  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:45.668931  1535 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:45.635984  6122 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:45.634404  6309 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:45.647241  6311 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:45.691231  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:45.693876  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:45.695547  1582 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:45.631537  6345 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:45.697713  1610 raft_consensus.cc:2368] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13.
I20260709 13:01:45.654850  6376 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:45.701107  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:45.703393  1582 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:45.664714  6291 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:45.634521  5968 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:45.693131  6286 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.658s	user 0.011s	sys 0.000s
I20260709 13:01:45.705073  6393 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:45.713675  6342 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:45.706887  6379 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:45.714998  1609 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:45.707119  6397 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:01:45.711536  6345 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:45.718791  6381 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:45.742527  6311 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:45.747376  6327 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:45.749181  6304 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:45.751096  6103 consensus_meta.cc:287] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.665s	user 0.007s	sys 0.004s
I20260709 13:01:45.751552  6103 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:45.753684  6406 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:01:45.687587  6122 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:45.730118  6407 log.cc:927] Time spent T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:01:45.727324  5989 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:45.733973  1535 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:45.779637  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:45.779873  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:45.657157  6171 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.732749  6342 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:01:45.766302  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 327822us
I20260709 13:01:45.745168  6311 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:45.684494  6132 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:45.739925  6286 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:01:45.790223  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.813328  1534 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:01:45.814355  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:45.777412  6224 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.800s	user 0.011s	sys 0.002s
I20260709 13:01:45.777683  6122 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:45.818363  1683 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:45.818636  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:45.819846  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:45.803251  6335 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.820650  2232 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:45.821071  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:45.822173  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:45.822476  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:45.801560  1542 consensus_meta.cc:287] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.000s
I20260709 13:01:45.815366  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:45.823526  2236 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:45.788817  6343 consensus_meta.cc:287] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.216s	user 0.009s	sys 0.007s
I20260709 13:01:45.825930  6343 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.827227  1615 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:45.828189  6343 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:45.811789  6379 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.819900  6325 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:45.820842  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:45.791241  6342 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:45.738960  6291 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.830942  1542 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:45.851271  2228 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:45.782088  6306 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:45.851555  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:45.812235  6324 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:45.859925  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 156617us
I20260709 13:01:45.858917  2232 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:45.816701  6052 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:45.863140  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:45.863402  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 400353us
I20260709 13:01:45.820793  6335 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.870101  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:45.816213  6381 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:45.829900  6304 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:45.879694  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:45.879813  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:45.875946  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 282684us
I20260709 13:01:45.879659  2228 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:45.880730  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:45.881048  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:45.882326  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:45.883874  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:45.892511  2232 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:01:45.912238  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:45.913151  2231 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:45.914844  2230 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:45.919934  1618 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:45.829268  6379 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.883767  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:45.928651  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 44939us
I20260709 13:01:45.931347  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:45.858929  6103 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:45.829751  6224 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.829193  6309 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.859390  6325 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:45.893318  6310 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.599s	user 0.012s	sys 0.000s
I20260709 13:01:45.942837  2236 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:45.943984  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:45.944685  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 122630us
W20260709 13:01:45.916749  6380 consensus_meta.cc:287] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.687s	user 0.008s	sys 0.003s
I20260709 13:01:45.892462  6311 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:45.889758  6327 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:45.933897  6386 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:45.957513  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:45.962182  2229 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:45.920948  6393 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:45.976209  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:45.977459  1615 raft_consensus.cc:2410] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:01:45.893142  6401 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:45.875914  6324 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:45.962000  6327 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:45.944515  6404 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:45.883281  6381 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:45.881315  6052 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:45.990837  6396 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:01:45.957828  6128 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.010700  1542 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:45.916436  6304 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:46.056389  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 68840.8us
I20260709 13:01:46.057808  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:45.992763  6408 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:46.061599  1539 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:46.039448  6404 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.010349  6311 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:46.043869  6401 raft_consensus.cc:697] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:45.977080  6393 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.062892  2233 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
W20260709 13:01:46.067127  6403 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 13:01:46.044497  5968 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.067672  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.070945  1542 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:45.960525  6405 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 13:01:46.071846  6132 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:46.074088  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:46.052528  6414 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:46.079452  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 121874us
I20260709 13:01:46.050150  6171 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.073237  1615 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:46.015538  6327 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:46.080569  2203 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:45.966771  6317 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.108s
I20260709 13:01:45.978360  6127 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:01:45.991945  6310 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:46.063910  6337 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:01:46.105361  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.098593  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:46.063587  5995 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:01:46.107659  6401 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.111284  6379 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.113483  6324 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:46.068498  6404 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:46.069533  6393 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:46.070071  6408 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:46.101600  2203 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:46.137642  6335 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:46.131044  2229 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:46.024648  6386 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:46.129037  6310 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:46.151568  6412 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:01:46.191802  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 880) took 2061 ms. Trace:
I20260709 13:01:46.072844  6325 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.153378  6335 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:46.093448  6414 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:46.138089  6224 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.143834  6324 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:46.158538  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 782) took 1134 ms. Trace:
I20260709 13:01:46.206154  1618 rpcz_store.cc:276] 0709 13:01:45.023992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.025051 (+  1059us) service_pool.cc:224] Handling call
0709 13:01:46.158286 (+1133235us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b54003c91d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:46.158448 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1890576256}
I20260709 13:01:46.158357  6408 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.159873  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 769) took 1442 ms. Trace:
I20260709 13:01:46.161626  6386 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:46.214923  1610 rpcz_store.cc:276] 0709 13:01:44.717325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.717559 (+   234us) service_pool.cc:224] Handling call
0709 13:01:46.159656 (+1442097us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b54000befd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:46.159812 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2127742720}
I20260709 13:01:46.159919  1616 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:01:46.162352  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 771) took 1352 ms. Trace:
I20260709 13:01:46.165293  6304 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:46.167704  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 876) took 2038 ms. Trace:
I20260709 13:01:46.171293  6052 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:46.172843  6263 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:46.175158  6288 consensus_meta.cc:287] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.865s	user 0.009s	sys 0.001s
I20260709 13:01:46.176952  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:46.177995  1689 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
W20260709 13:01:46.178409  6305 consensus_meta.cc:287] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.059s	user 0.013s	sys 0.003s
I20260709 13:01:46.179593  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:46.180190  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2278) took 1180 ms. Trace:
I20260709 13:01:46.194197  1540 rpcz_store.cc:276] 0709 13:01:44.130555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.283063 (+152508us) service_pool.cc:224] Handling call
0709 13:01:44.283472 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:44.582112 (+298640us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:45.002931 (+420819us) spinlock_profiling.cc:243] Waited 5.83 ms on lock 0x7b180040d380. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fc455 00007fe0934e43c7 00007fe08faf69a7 00007fe08fb060bf 00007fe08fb06038
0709 13:01:45.043065 (+ 40134us) log.cc:844] Serialized 14 byte log entry
0709 13:01:46.159349 (+1116284us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b480030c858. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:01:46.182918 (+ 23569us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:46.183439 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 13:01:46.183612 (+   173us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:46.183646 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.183706 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.188095 (+  4389us) raft_consensus.cc:1685] finished
0709 13:01:46.188115 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.191735 (+  3620us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1252947,"prepare.queue_time_us":239380,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":1219955,"raft.run_cpu_time_us":5381,"raft.run_wall_time_us":1580588,"spinlock_wait_cycles":19507968,"thread_start_us":202823,"threads_started":2,"wal-append.queue_time_us":273279}
I20260709 13:01:46.243605  1538 rpcz_store.cc:276] 0709 13:01:45.000084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.031907 (+ 31823us) service_pool.cc:224] Handling call
0709 13:01:46.180020 (+1148113us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54000ed8d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:46.180134 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":607442176}
I20260709 13:01:46.181675  6238 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.182379  6227 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:46.183804  6394 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:46.251363  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:46.251089  6394 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:46.252982  6129 raft_consensus.cc:2764] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:46.183863  6206 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:46.253770  6206 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:46.256450  6206 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:46.185437  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2277) took 1185 ms. Trace:
I20260709 13:01:46.258169  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:46.258983  1686 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:46.257270  1535 rpcz_store.cc:276] 0709 13:01:44.999846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.781559 (+781713us) service_pool.cc:224] Handling call
0709 13:01:45.781947 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:45.782532 (+   585us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:45.957012 (+174480us) log.cc:844] Serialized 14 byte log entry
0709 13:01:46.184899 (+227887us) log.cc:844] Serialized 14 byte log entry
0709 13:01:46.185075 (+   176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:46.185122 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:46.185164 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.185233 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.185356 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":174206,"prepare.queue_time_us":262240,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":6603,"raft.queue_time_us":175883,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":138,"thread_start_us":218118,"threads_started":2,"wal-append.queue_time_us":281902}
I20260709 13:01:46.259992  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.260778  1535 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:46.186066  1542 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:46.262637  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.263067  6393 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.264097  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.264447  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.265239  1610 raft_consensus.cc:2368] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12.
I20260709 13:01:46.187026  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 959) took 1128 ms. Trace:
W20260709 13:01:46.182390  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 1.02494e+06us
W20260709 13:01:46.170256  6418 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:01:46.228513  6335 raft_consensus.cc:697] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:46.192737  2231 raft_consensus.cc:2410] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:01:46.201540  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.179497  6381 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:46.208664  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.278177  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.278892  1540 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:46.279830  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:46.149613  6291 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.228248  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.222723  1613 rpcz_store.cc:276] 0709 13:01:44.810040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.818466 (+  8426us) service_pool.cc:224] Handling call
0709 13:01:46.162182 (+1343716us) spinlock_profiling.cc:243] Waited 40.3 ms on lock 0x7b54002bd754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:46.162293 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92644224}
I20260709 13:01:46.111729  6311 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:46.229763  1537 rpcz_store.cc:276] 0709 13:01:44.129550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.264192 (+134642us) service_pool.cc:224] Handling call
0709 13:01:44.264533 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:44.569671 (+305138us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:44.569704 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:44.570200 (+   496us) log.cc:844] Serialized 26 byte log entry
0709 13:01:44.853905 (+283705us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:44.870697 (+ 16792us) log.cc:844] Serialized 14 byte log entry
0709 13:01:44.871436 (+   739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:44.871522 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.137521 (+1265999us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b180033f2e0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:01:46.167374 (+ 29853us) raft_consensus.cc:1685] finished
0709 13:01:46.167478 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.167620 (+   142us) inbound_call.cc:177] Queueing success response
0709 13:01:46.179623 (+ 12003us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b180033f2e0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c979c0a 00007fe0960b9219 00007fe0939c058f 00007fe08c9ff571 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd
Metrics: {"mutex_wait_us":304267,"prepare.queue_time_us":18400,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":352024,"raft.run_cpu_time_us":1648,"raft.run_wall_time_us":23955,"spinlock_wait_cycles":27263744,"thread_start_us":283317,"threads_started":1,"wal-append.queue_time_us":363580}
W20260709 13:01:46.234839  6371 consensus_meta.cc:287] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.715s	user 0.012s	sys 0.000s
W20260709 13:01:46.246526  1688 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1803) took 1427 ms (client timeout 1824 ms). Trace:
I20260709 13:01:46.248168  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.249548  6304 consensus_queue.cc:1059] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:46.249804  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:46.250476  1612 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:01:46.187155  1539 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: [9.9-11.10]
I20260709 13:01:46.266830  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.314805  1584 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:46.267916  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.267935  1683 rpcz_store.cc:276] 0709 13:01:45.058924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.059823 (+   899us) service_pool.cc:224] Handling call
0709 13:01:46.186834 (+1127011us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b5400355054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:46.186966 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1742102272}
I20260709 13:01:46.190554  1684 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:01:46.270365  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:46.237317  6309 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.325747  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:46.326526  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.281849  6305 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:46.328213  6305 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:46.329576  1611 raft_consensus.cc:2410] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:01:46.294667  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.330539  1613 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:46.295383  6414 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:46.266216  6238 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:46.296944  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:46.311695  6388 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.683s	user 0.011s	sys 0.000s
I20260709 13:01:46.337823  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:46.338464  1613 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:46.338522  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.316957  1582 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:46.323089  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.339203  1540 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:46.323851  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 792) took 1072 ms. Trace:
I20260709 13:01:46.339469  1610 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:01:46.339740  1616 rpcz_store.cc:276] 0709 13:01:45.251161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.293004 (+ 41843us) service_pool.cc:224] Handling call
0709 13:01:46.159573 (+866569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.222237 (+ 62664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.323634 (+101397us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b54003c91d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:46.323789 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1961586048}
I20260709 13:01:46.312829  6103 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:46.324316  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55802.5us
I20260709 13:01:46.330598  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:46.342572  1537 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:01:46.306000  6381 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:46.335700  1541 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:46.337172  1609 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:46.303812  6394 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:46.310976  1688 rpcz_store.cc:269] 0709 13:01:44.819083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.821527 (+  2444us) service_pool.cc:224] Handling call
0709 13:01:46.179426 (+1357899us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b5800010e98. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:01:46.246435 (+ 67009us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2959616}
I20260709 13:01:46.336445  6070 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:46.338279  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 155460us
I20260709 13:01:46.323374  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:46.349128  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.349447  1539 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:46.343338  1584 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:46.266968  6393 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.343811  1617 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
W20260709 13:01:46.351433  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.353310  1535 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:46.357188  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.335374  6309 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.364795  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.373701  1609 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:46.378137  1613 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:46.379555  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.360584  1584 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:46.362371  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:46.365268  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:46.383826  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.383659  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.337036  6224 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:46.345046  6304 consensus_queue.cc:1243] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 13:01:46.389189  1687 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1812) took 1510 ms (client timeout 1971 ms). Trace:
I20260709 13:01:46.396157  1537 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:46.384788  1616 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:46.353118  6414 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:46.386222  1584 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:01:46.391696  1687 rpcz_store.cc:269] 0709 13:01:44.878990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.879136 (+   146us) service_pool.cc:224] Handling call
0709 13:01:46.388999 (+1509863us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54000ab9d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:46.389133 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493871872}
I20260709 13:01:46.353741  6310 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.359267  6390 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:46.391103  1507 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:46.402338  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:46.402855  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:46.391170  6371 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:46.406167  6132 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:46.400748  5885 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:46.385150  6340 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:01:46.388288  6227 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.353165  6103 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:46.363916  6070 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.371390  6305 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:46.409017  5929 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.415208  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.403586  1540 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
W20260709 13:01:46.426893  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.365412  6416 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:46.430377  1537 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:46.430917  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 329389us
I20260709 13:01:46.478897  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.481731  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:46.456664  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.400875  6381 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:46.461072  1610 raft_consensus.cc:2410] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:01:46.463549  1615 raft_consensus.cc:2368] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13.
I20260709 13:01:46.432730  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:46.485325  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:46.486146  2231 raft_consensus.cc:2368] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
I20260709 13:01:46.423676  6414 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:46.383358  6309 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.483139  1616 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:46.431823  6132 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:46.498816  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:46.460634  6375 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:46.502425  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.499465  1543 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:01:46.502713  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:46.432471  6304 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:46.458463  6423 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.386668  6432 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.112s
I20260709 13:01:46.497073  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 796) took 1169 ms. Trace:
I20260709 13:01:46.503816  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:46.503634  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:46.504649  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:46.404137  6394 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:01:46.511196  1615 rpcz_store.cc:276] 0709 13:01:45.327740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.414919 (+1087179us) service_pool.cc:224] Handling call
0709 13:01:46.496990 (+ 82071us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.505990  1609 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:46.508916  1508 leader_election.cc:400] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
W20260709 13:01:46.513727  2203 connection.cc:256] RPC call timeout handler was delayed by -1.09786165700001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29340000000000005s for a response
I20260709 13:01:46.479200  6275 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.478713  6390 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.487999  5929 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.518162  1616 raft_consensus.cc:2376] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12. Re-sending same reply.
I20260709 13:01:46.384809  6335 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.520301  1508 leader_election.cc:403] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:01:46.519954  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.522222  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 797) took 1182 ms. Trace:
I20260709 13:01:46.495129  6371 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:46.523151  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 90670.4us
I20260709 13:01:46.520526  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 795) took 1207 ms. Trace:
I20260709 13:01:46.460510  6227 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:46.460695  5885 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:46.520496  2203 connection.cc:256] RPC call timeout handler was delayed by -1.0097132780000493s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1515s for a response
I20260709 13:01:46.534102  1616 rpcz_store.cc:276] 0709 13:01:45.340433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.503422 (+1162989us) service_pool.cc:224] Handling call
0709 13:01:46.522160 (+ 18738us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.526865  1509 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:46.510210  6304 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.577185  1507 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:46.572849  1537 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:46.450254  6404 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:46.576491  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.579573  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.580562  1537 raft_consensus.cc:2410] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:01:46.541538  1609 rpcz_store.cc:276] 0709 13:01:45.312732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.386442 (+1073710us) service_pool.cc:224] Handling call
0709 13:01:46.520459 (+134017us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.519140  6436 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.535595  6414 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:46.581749  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:46.534090  6375 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:46.534232  6224 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:46.586349  1686 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:46.585618  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 58929.2us
I20260709 13:01:46.595502  1507 leader_election.cc:400] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
I20260709 13:01:46.575060  6318 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.591245  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:46.580917  6304 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:46.596287  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:46.591224  1543 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:46.582484  6376 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:46.573573  6324 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:46.658047  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.596236  1507 leader_election.cc:403] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:01:46.546741  6275 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.601648  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:46.582356  6442 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:46.653893  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 62715.6us
I20260709 13:01:46.572057  6390 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.658699  6132 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.671830  6132 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:46.672264  6390 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.656101  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:46.573558  6371 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.663779  6224 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:46.673791  6224 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:46.674170  6224 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:46.674537  6371 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.678550  5929 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.678798  6442 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.679452  1539 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:46.679586  6070 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.680718  1539 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:46.683398  6275 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.686220  6306 raft_consensus.cc:3055] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:01:46.686674  6306 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:46.687493  6306 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:46.688731  6306 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:01:46.689724  6342 consensus_meta.cc:287] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.816s	user 0.012s	sys 0.000s
I20260709 13:01:46.690945  6337 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:46.691530  6337 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:46.692368  6337 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:46.694096  6392 log.cc:927] Time spent T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.317s	user 0.001s	sys 0.006s
I20260709 13:01:46.693717  6215 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:46.695194  6215 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:01:46.696879  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1801) took 1893 ms. Trace:
I20260709 13:01:46.697268  1690 rpcz_store.cc:276] 0709 13:01:44.803959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.804098 (+   139us) service_pool.cc:224] Handling call
0709 13:01:44.816611 (+ 12513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:45.009226 (+192615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:45.009259 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:45.015973 (+  6714us) log.cc:844] Serialized 26 byte log entry
0709 13:01:45.186981 (+171008us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:45.187033 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.163859 (+976826us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540028f5d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:46.163911 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.693351 (+529440us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:46.696654 (+  3303us) raft_consensus.cc:1685] finished
0709 13:01:46.696676 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.696821 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1602,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":3795,"spinlock_wait_cycles":488165120,"thread_start_us":143201,"threads_started":1,"wal-append.queue_time_us":260024}
I20260709 13:01:46.699282  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2317) took 1462 ms. Trace:
I20260709 13:01:46.699909  6442 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:46.601224  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:46.702719  6327 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.605s	user 0.011s	sys 0.007s
W20260709 13:01:46.660174  6440 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
W20260709 13:01:46.662892  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 66705.3us
I20260709 13:01:46.533725  6423 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.648908  6408 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:46.709302  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:46.709841  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:46.710767  2232 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:46.657859  6324 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.699680  1612 rpcz_store.cc:276] 0709 13:01:45.236766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.236888 (+   122us) service_pool.cc:224] Handling call
0709 13:01:45.288039 (+ 51151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.313931 (+1025892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.698999 (+385068us) spinlock_profiling.cc:243] Waited 962 ms on lock 0x7b5400398ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:46.699190 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2212412160}
I20260709 13:01:46.664780  1537 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:46.657308  5968 raft_consensus.cc:2764] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:01:46.666273  6227 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:46.667351  6435 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:01:46.667989  6445 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.067s	user 0.001s	sys 0.001s
I20260709 13:01:46.668937  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2288) took 1370 ms. Trace:
I20260709 13:01:46.703471  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1828) took 1100 ms. Trace:
I20260709 13:01:46.730321  1609 rpcz_store.cc:276] 0709 13:01:45.602965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.583298 (+980333us) service_pool.cc:224] Handling call
0709 13:01:46.703393 (+120095us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:46.704016  1509 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:46.706705  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:46.707228  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 120660us
I20260709 13:01:46.730173  1534 rpcz_store.cc:276] 0709 13:01:45.298694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.814779 (+516085us) service_pool.cc:224] Handling call
0709 13:01:45.860202 (+ 45423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:46.186095 (+325893us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:46.186132 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:46.186593 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:01:46.345409 (+158816us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:46.345888 (+   479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.345961 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:46.668703 (+322742us) raft_consensus.cc:1685] finished
0709 13:01:46.668725 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:46.668877 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":325444,"raft.queue_time_us":325593,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"thread_start_us":158474,"threads_started":1,"wal-append.queue_time_us":335297}
I20260709 13:01:46.732610  1507 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:46.708563  6238 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:46.649488  6318 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:46.722728  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:46.724080  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:46.724197  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:46.725262  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.592742  6436 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.656888  6325 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:46.733294  1584 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:46.735984  1612 raft_consensus.cc:2368] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12.
I20260709 13:01:46.739828  1690 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:01:46.741755  2232 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:01:46.743023  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:46.718451  6371 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
I20260709 13:01:46.744831  2203 leader_election.cc:400] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13.
I20260709 13:01:46.746240  2203 leader_election.cc:400] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 9 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 9. Current term is 10.
I20260709 13:01:46.747217  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:46.728639  6394 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.742301  6052 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:46.753312  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.736351  6305 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:46.755970  1507 leader_election.cc:400] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10.
I20260709 13:01:46.703209  6327 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.753130  6252 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:46.757131  6378 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:46.690243  6342 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.758921  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:46.755204  5929 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.783349  1507 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:46.757306  6227 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:46.759441  5885 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:46.762957  6224 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.751345  6291 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:46.788015  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:46.766513  6441 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 13:01:46.770412  6443 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 13:01:46.770278  6404 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:46.756042  1609 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:46.773370  6318 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:46.755187  6288 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 13:01:46.774369  6398 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:46.776575  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.721752  6442 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:01:46.889154  6310 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:46.889711  6310 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:46.890548  6310 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.751691  6416 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:46.786394  1534 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:01:46.786970  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 83038.4us
I20260709 13:01:46.789952  1507 leader_election.cc:400] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
I20260709 13:01:46.923688  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:46.923805  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 134028us
I20260709 13:01:46.793045  2203 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:46.758014  6394 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.785558  5929 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:46.785723  6238 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:46.786011  6227 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.796136  6252 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:46.800506  6405 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.801591  5885 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:46.802053  6052 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:46.802353  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2322) took 1489 ms. Trace:
I20260709 13:01:46.802968  1615 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:46.803834  6342 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.783991  6305 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.785646  6371 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:46.802359  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1833) took 1198 ms. Trace:
I20260709 13:01:46.988488  1610 rpcz_store.cc:276] 0709 13:01:45.604164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.340452 (+736288us) service_pool.cc:224] Handling call
0709 13:01:46.802173 (+461721us) spinlock_profiling.cc:243] Waited 99.6 ms on lock 0x7b54002dbed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:46.802312 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":229112064}
I20260709 13:01:46.784307  6378 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:46.884914  1618 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:46.788081  6309 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:46.972111  6291 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:46.925271  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:46.929258  1611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 779) took 1978 ms (client timeout 1644 ms). Trace:
I20260709 13:01:46.968132  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:46.932734  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:46.970418  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46814.5us
I20260709 13:01:46.835110  6327 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:46.967512  1614 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:01:47.055315  1614 pending_rounds.cc:85] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 10
I20260709 13:01:47.055207  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:47.055714  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:46.976629  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.056000  2231 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:46.976615  1617 rpcz_store.cc:276] 0709 13:01:45.313201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.568935 (+255734us) service_pool.cc:224] Handling call
0709 13:01:46.802191 (+1233256us) spinlock_profiling.cc:243] Waited 715 ms on lock 0x7b540008b6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:46.802292 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1645610368}
I20260709 13:01:46.977660  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:46.965298  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 172305us
I20260709 13:01:46.979220  6310 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:46.967875  6423 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:46.991019  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:46.930887  6232 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:46.993554  1508 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:46.998018  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:46.999399  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:46.973806  6227 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:47.003786  1583 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:01:47.073985  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 70259.3us
I20260709 13:01:47.074915  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:46.720517  6205 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:01:47.055786  1614 pending_rounds.cc:107] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:01:47.099123  1614 raft_consensus.cc:2889] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:46.934266  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:46.934741  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:46.974129  6306 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.059141  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:47.055445  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:47.057515  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:47.056663  1692 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:47.056504  2235 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:47.060461  1690 raft_consensus.cc:2368] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
W20260709 13:01:47.055178  1611 rpcz_store.cc:269] 0709 13:01:44.950938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:44.951121 (+   183us) service_pool.cc:224] Handling call
0709 13:01:46.928924 (+1977803us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540008c5d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:46.929152 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3126343040}
W20260709 13:01:46.983136  6122 consensus_meta.cc:287] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.112s	user 0.012s	sys 0.002s
I20260709 13:01:47.064873  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:46.974748  6442 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:46.975689  6375 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:47.071028  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.005288  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:47.006162  6381 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.000s
I20260709 13:01:46.967639  6238 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:46.943439  6408 raft_consensus.cc:697] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:47.057927  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2324) took 1744 ms. Trace:
I20260709 13:01:47.143213  1609 rpcz_store.cc:276] 0709 13:01:45.316613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.884588 (+1567975us) service_pool.cc:224] Handling call
0709 13:01:47.057849 (+173261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.000358  6371 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:46.974246  6416 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:47.105005  1618 raft_consensus.cc:2410] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:47.102896  2237 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
W20260709 13:01:47.106587  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 49205.9us
I20260709 13:01:47.085098  2231 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:47.148005  2203 leader_election.cc:400] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13.
I20260709 13:01:47.000236  6305 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.110309  6122 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.101876  6390 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:47.116081  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.121891  6167 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:47.122465  6232 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:47.122619  5929 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.123103  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:47.124998  6376 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
I20260709 13:01:47.129520  6132 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:47.179356  1617 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:47.132048  6103 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:47.147099  6371 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:47.138579  6394 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.100600  6291 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:47.138967  6414 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.000s
I20260709 13:01:47.005478  6378 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:46.990494  6405 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.140026  6224 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.104426  6306 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.140640  6317 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.141009  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:46.926155  6325 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.148463  2203 leader_election.cc:403] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:47.156832  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.149684  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2330) took 1499 ms. Trace:
I20260709 13:01:47.149917  1611 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:47.181025  6232 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:47.133476  6432 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.142634  6238 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.138002  6442 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:47.171087  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 187254us
I20260709 13:01:47.182462  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:47.184106  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 109303us
I20260709 13:01:47.182587  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.181875  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2337) took 1116 ms. Trace:
W20260709 13:01:47.188031  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 194621us
I20260709 13:01:47.188254  6252 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:47.188922  1683 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:47.216202  6052 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13.
I20260709 13:01:47.276192  6232 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.179749  6132 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.277351  6132 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.277864  6252 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:47.198700  1615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2333) took 1433 ms (client timeout 1846 ms). Trace:
W20260709 13:01:47.280354  1615 rpcz_store.cc:269] 0709 13:01:45.765860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.519680 (+753820us) service_pool.cc:224] Handling call
0709 13:01:47.198421 (+678741us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54000c7e54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:47.198624 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618142464}
I20260709 13:01:47.281744  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:47.284760  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:47.173763  6167 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.285638  6167 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.179949  6390 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:47.205932  1610 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
W20260709 13:01:47.206115  1684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2697) took 1730 ms (client timeout 2055 ms). Trace:
I20260709 13:01:47.206131  1535 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:47.290511  6390 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.292016  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:47.292913  1535 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:47.207971  1618 rpcz_store.cc:276] 0709 13:01:45.650157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.006209 (+1356052us) service_pool.cc:224] Handling call
0709 13:01:47.149608 (+143399us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.208181  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1842) took 1019 ms. Trace:
I20260709 13:01:47.293855  1611 rpcz_store.cc:276] 0709 13:01:46.188837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.103853 (+915016us) service_pool.cc:224] Handling call
0709 13:01:47.208112 (+104259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.294284  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:47.294878  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.295140  1618 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.173882  6304 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.276652  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.297083  1683 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:47.192914  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.298065  1688 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:47.142802  6381 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:47.290318  1684 rpcz_store.cc:269] 0709 13:01:45.475924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.519853 (+ 43929us) service_pool.cc:224] Handling call
0709 13:01:47.205919 (+1686066us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b54000749d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:47.206055 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":842393472}
I20260709 13:01:47.299352  1609 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:47.230672  1509 leader_election.cc:336] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.662s (SENT)
I20260709 13:01:47.302235  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:47.304929  1616 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:47.216367  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:47.219374  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:47.306970  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 90736.2us
I20260709 13:01:47.305078  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.308149  1617 rpcz_store.cc:276] 0709 13:01:46.065530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.102249 (+1036719us) service_pool.cc:224] Handling call
0709 13:01:47.181792 (+ 79543us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:47.310324  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 162544us
I20260709 13:01:47.311210  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:47.313336  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.314124  1684 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:47.322024  1618 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:47.322897  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2336) took 1367 ms. Trace:
I20260709 13:01:47.323302  1618 rpcz_store.cc:276] 0709 13:01:45.956132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.293913 (+1337781us) service_pool.cc:224] Handling call
0709 13:01:47.322815 (+ 28902us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.324671  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:47.325438  2228 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
W20260709 13:01:47.326735  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 96132us
W20260709 13:01:47.327479  1509 leader_election.cc:336] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.862s (SENT)
I20260709 13:01:47.327867  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:47.328945  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:47.330618  2231 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:47.338361  1689 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:47.338902  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.340245  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.341360  1609 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:47.342005  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.342669  1683 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:47.344666  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1840) took 1156 ms. Trace:
I20260709 13:01:47.345352  1609 rpcz_store.cc:276] 0709 13:01:46.188358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.301297 (+1112939us) service_pool.cc:224] Handling call
0709 13:01:47.344588 (+ 43291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.344627  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:47.346717  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:47.213377  6325 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.344311  2201 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:47.351640  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:47.277071  6227 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.355350  1690 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:47.356576  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.357311  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1847) took 1167 ms. Trace:
I20260709 13:01:47.357662  1617 rpcz_store.cc:276] 0709 13:01:46.189930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.324509 (+1134579us) service_pool.cc:224] Handling call
0709 13:01:47.357255 (+ 32746us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.358335  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.358739  1617 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 9.
I20260709 13:01:47.359368  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.359853  1617 raft_consensus.cc:2410] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
W20260709 13:01:47.360751  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 53328.2us
I20260709 13:01:47.306468  6380 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.364476  2201 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:47.371722  1689 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:47.369410  1507 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:47.370396  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:47.309201  6224 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:47.365103  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1843) took 1176 ms. Trace:
I20260709 13:01:47.366780  1535 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:47.367311  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:47.202569  6414 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:47.376996  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 68973.2us
I20260709 13:01:47.378324  2228 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:47.378522  1583 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:47.375386  1617 rpcz_store.cc:276] 0709 13:01:46.189047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.359207 (+1170160us) service_pool.cc:224] Handling call
0709 13:01:47.365045 (+  5838us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:47.376284  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 48840.1us
W20260709 13:01:47.381024  1509 leader_election.cc:336] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.026s (SENT)
I20260709 13:01:47.347453  6381 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:47.384235  1507 leader_election.cc:400] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
I20260709 13:01:47.386128  2230 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:47.393399  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:47.396279  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.397239  1617 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:47.394392  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:47.398113  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2327) took 1834 ms. Trace:
I20260709 13:01:47.399071  1617 rpcz_store.cc:276] 0709 13:01:45.584923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.395910 (+1810987us) service_pool.cc:224] Handling call
0709 13:01:47.398028 (+  2118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.400491  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:47.401943  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:47.402278  1583 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:47.210846  6317 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.363610  6052 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:47.297477  6432 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.405572  2235 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:47.416417  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:47.350642  6325 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.373358  6458 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:47.431301  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.441629  1617 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:47.432829  1613 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:47.339000  6342 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:01:47.372999  6327 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:47.443459  1683 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:47.443717  6327 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:47.435752  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 55348.3us
I20260709 13:01:47.444336  6327 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.448570  1688 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:47.417277  2236 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:01:47.206771  6408 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.376174  6423 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:47.443502  6342 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:47.449903  6423 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.450064  6342 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:47.450484  6423 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.451210  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1844) took 1253 ms. Trace:
I20260709 13:01:47.451596  1617 rpcz_store.cc:276] 0709 13:01:46.189273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.430996 (+1241723us) service_pool.cc:224] Handling call
0709 13:01:47.442260 (+ 11264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.452590  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:47.374256  6460 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:47.392612  6398 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:47.454159  6398 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.454576  6398 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.462561  6227 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:47.462934  6452 log.cc:927] Time spent T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:01:47.463913  6305 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:47.464532  6451 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:01:47.464598  6305 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:47.465168  6305 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:47.376740  6380 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.367798  6424 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 13:01:47.467563  1617 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:47.461220  6238 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.468081  6424 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:47.450575  6342 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.468492  6380 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.392966  5885 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:01:47.470460  6238 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.466285  6327 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.466102  6305 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.471719  1611 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:47.472182  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2298) took 1122 ms. Trace:
I20260709 13:01:47.475831  1539 rpcz_store.cc:276] 0709 13:01:46.349927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.700276 (+350349us) service_pool.cc:224] Handling call
0709 13:01:46.700796 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:46.728990 (+ 28194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:46.729028 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:46.737009 (+  7981us) log.cc:844] Serialized 26 byte log entry
0709 13:01:47.174497 (+437488us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:47.174555 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.174650 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:47.471969 (+297319us) raft_consensus.cc:1685] finished
0709 13:01:47.471988 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.472122 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34708,"raft.queue_time_us":27445,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":1929,"thread_start_us":395560,"threads_started":1,"wal-append.queue_time_us":462114}
W20260709 13:01:47.477900  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 83656.3us
I20260709 13:01:47.478183  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:47.404320  6193 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:47.475148  1609 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:47.472837  1687 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:01:47.495594  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 802) took 1002 ms. Trace:
I20260709 13:01:47.432909  6132 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.394167  6457 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:47.498602  1685 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:47.479105  1539 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:47.481441  1615 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:47.500510  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.496027  1609 rpcz_store.cc:276] 0709 13:01:46.493666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.346210 (+852544us) service_pool.cc:224] Handling call
0709 13:01:47.495526 (+149316us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.502158  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.502282  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.503062  6454 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.504899  1609 raft_consensus.cc:2410] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 14 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 13:01:47.512941  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.484776  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:47.440716  6304 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.514708  1609 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:47.515242  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:47.441547  6449 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:47.505375  1615 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:47.516290  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.474707  6381 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:47.517488  1688 raft_consensus.cc:2410] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:01:47.393767  6224 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.515961  1539 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:47.518297  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.303356  6416 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.521884  1609 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:47.489403  6305 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.488320  6306 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.487532  6291 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:47.532878  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 60061.5us
I20260709 13:01:47.533030  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:47.533694  1686 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:47.468664  6424 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.535084  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:47.530197  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.497841  6193 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:47.532227  1689 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:47.537034  1690 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:47.528258  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:47.536722  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:01:47.528558  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 126427us
I20260709 13:01:47.520385  6404 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:47.529129  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.534579  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:47.536309  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:47.471202  6442 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.540416  1539 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:47.544632  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.546096  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:47.546793  2232 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:47.548369  2236 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:47.538118  6193 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.517684  6454 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.549721  6424 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.550477  6457 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:47.551231  6454 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.552793  6317 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.538539  6122 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.544273  6408 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:47.558235  6408 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.536722  6052 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:47.558652  6408 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.559356  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.560201  1615 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:47.561265  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.544085  6381 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:47.561981  1615 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:47.562158  6381 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.562573  6381 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.562965  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:47.563565  1615 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.541288  6404 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.540593  1609 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:47.568953  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 806) took 1066 ms. Trace:
I20260709 13:01:47.569793  1609 rpcz_store.cc:276] 0709 13:01:46.502417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.527484 (+1025067us) service_pool.cc:224] Handling call
0709 13:01:47.568893 (+ 41409us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.570827  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.571504  1689 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:47.572515  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:47.574817  6132 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.575378  6371 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.568358  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:47.578405  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:47.565819  6404 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.594596  6469 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.595247  6469 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.597522  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:47.598237  2232 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.599846  6469 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.585443  6368 raft_consensus.cc:2764] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:01:47.602339  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:47.583312  6122 raft_consensus.cc:2764] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:01:47.581842  6449 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.611498  2232 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:47.609918  6129 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:47.614706  6378 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.615957  6414 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.570326  6193 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.557191  6167 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.612006  1539 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:47.621750  6129 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:47.636888  1543 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:01:47.642969  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:47.636276  6423 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.637723  6327 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.642448  6342 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.623701  6449 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.663748  6449 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:47.637737  6378 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.664348  1609 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:47.665037  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:47.665740  2236 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.643573  2232 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:47.568267  6052 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.621708  6224 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.645643  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.649183  2228 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:01:47.647905  6129 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.654852  6193 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.673812  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.676730  2232 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:01:47.663833  6342 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.643951  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:47.646328  6398 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.681210  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:47.677989  1615 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:47.681481  6129 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.684561  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:47.686389  1616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1839) took 2080 ms (client timeout 2026 ms). Trace:
I20260709 13:01:47.686504  6193 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:47.686892  1616 rpcz_store.cc:269] 0709 13:01:45.606557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.576214 (+969657us) service_pool.cc:224] Handling call
0709 13:01:47.686190 (+1109976us) spinlock_profiling.cc:243] Waited 719 ms on lock 0x7b54003499d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:47.686322 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1653178752}
I20260709 13:01:47.687171  6193 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.688666  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2325) took 2371 ms. Trace:
I20260709 13:01:47.691025  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:47.692302  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.693779  6320 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:47.694380  6320 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:47.695353  6320 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.697464  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:47.697686  6371 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.698768  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:47.699752  6167 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.699927  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:47.702394  1613 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1834) took 2098 ms (client timeout 1862 ms). Trace:
W20260709 13:01:47.702827  1613 rpcz_store.cc:269] 0709 13:01:45.604385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.379244 (+774859us) service_pool.cc:224] Handling call
0709 13:01:47.702184 (+1322940us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540008bbd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:47.702319 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2420002176}
I20260709 13:01:47.704025  6193 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.685966  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:47.705369  1539 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.705597  2203 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:47.707557  6342 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:47.710456  6381 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.712909  6381 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.713605  6167 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:47.711793  1615 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:01:47.715801  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2323) took 2402 ms. Trace:
I20260709 13:01:47.716090  6408 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.718730  6398 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:47.720693  6423 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.689049  1616 rpcz_store.cc:276] 0709 13:01:45.317087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.687844 (+2370757us) service_pool.cc:224] Handling call
0709 13:01:47.688169 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.688352 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:47.688375 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:47.688419 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:47.688457 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.688511 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.688611 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.724511  2201 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:47.724998  1692 raft_consensus.cc:2368] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
I20260709 13:01:47.725680  6423 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:47.725575  6327 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:47.726444  6423 raft_consensus.cc:697] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:47.727389  6423 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:47.729000  6446 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.002s	user 0.005s	sys 0.006s
W20260709 13:01:47.730187  1685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2700) took 2167 ms (client timeout 1918 ms). Trace:
W20260709 13:01:47.730629  1685 rpcz_store.cc:269] 0709 13:01:45.563278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.586563 (+ 23285us) service_pool.cc:224] Handling call
0709 13:01:47.730004 (+2143441us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b540041e1d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:47.730101 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4313270528}
I20260709 13:01:47.731014  6408 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:47.724674  1616 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:47.731859  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.733378  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2299) took 1383 ms. Trace:
I20260709 13:01:47.716101  1615 rpcz_store.cc:276] 0709 13:01:45.313368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.711098 (+2397730us) service_pool.cc:224] Handling call
0709 13:01:47.711567 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:47.715546 (+  3979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.715747 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.730062  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.735028  1687 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:47.730073  6404 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.736016  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.704079  1613 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 8. Current term is 9. Ops: []
I20260709 13:01:47.738327  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2326) took 2300 ms. Trace:
I20260709 13:01:47.738627  1613 rpcz_store.cc:276] 0709 13:01:45.439549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.703598 (+2264049us) service_pool.cc:224] Handling call
0709 13:01:47.703887 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.738084 (+ 34197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.738273 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.739944  1613 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:01:47.740638  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 799) took 2133 ms. Trace:
I20260709 13:01:47.733156  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.733844  1543 rpcz_store.cc:276] 0709 13:01:46.350201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.701556 (+351355us) service_pool.cc:224] Handling call
0709 13:01:46.701868 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:47.643225 (+941357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.733095 (+ 89870us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b54000ec4d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:47.733306 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2149676928}
W20260709 13:01:47.732041  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.750896  2203 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:47.731863  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1829) took 2128 ms. Trace:
I20260709 13:01:47.701154  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:47.752395  1616 rpcz_store.cc:276] 0709 13:01:45.603199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.724021 (+2120822us) service_pool.cc:224] Handling call
0709 13:01:47.724337 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.731601 (+  7264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.731785 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.741014  1613 rpcz_store.cc:276] 0709 13:01:45.607163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.739345 (+2132182us) service_pool.cc:224] Handling call
0709 13:01:47.739656 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.740340 (+   684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.740569 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.761914  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.757876  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:47.758916  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.683706  2236 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:01:47.754944  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:47.783012  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:47.783931  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:47.786029  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:47.787761  1613 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:01:47.787983  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.788106  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:47.788445  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2332) took 2063 ms. Trace:
I20260709 13:01:47.788861  1613 rpcz_store.cc:276] 0709 13:01:45.724804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.783868 (+2059064us) service_pool.cc:224] Handling call
0709 13:01:47.784296 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.788196 (+  3900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.788391 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.790918  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:47.755985  6381 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.794191  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:47.795611  1539 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:47.800829  1543 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.756517  6305 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.801283  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:47.802436  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2300) took 1452 ms. Trace:
I20260709 13:01:47.793670  6342 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:47.804980  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:47.753645  6321 consensus_queue.cc:1059] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:01:47.793970  6414 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:47.797376  6288 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:01:47.797199  6132 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:47.781715  6442 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:47.811107  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2343) took 1109 ms. Trace:
I20260709 13:01:47.819352  1611 rpcz_store.cc:276] 0709 13:01:46.701806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.294589 (+592783us) service_pool.cc:224] Handling call
0709 13:01:47.810919 (+516330us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400105254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:47.811030 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405092608}
I20260709 13:01:47.782449  6398 consensus_queue.cc:1059] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:01:47.813043  1582 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:47.814364  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.800102  6205 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:47.797600  6167 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:47.817220  1543 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:01:47.804220  6342 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.811954  1539 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:47.826356  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2826) took 1640 ms. Trace:
I20260709 13:01:47.826741  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:47.810914  6469 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:47.816130  6325 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:47.829937  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:47.830976  1543 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:47.822811  1685 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:47.831573  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:47.819832  6132 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.803282  6305 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.826874  1538 rpcz_store.cc:276] 0709 13:01:46.200477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.329866 (+129389us) service_pool.cc:224] Handling call
0709 13:01:46.330208 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:46.335598 (+  5390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:46.335645 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:46.336229 (+   584us) log.cc:844] Serialized 26 byte log entry
0709 13:01:46.500249 (+164020us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:46.532733 (+ 32484us) log.cc:844] Serialized 14 byte log entry
0709 13:01:46.534877 (+  2144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:46.534987 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:47.620207 (+1085220us) spinlock_profiling.cc:243] Waited 4.8 ms on lock 0x7b180046b760. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff522 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:01:47.728445 (+108238us) spinlock_profiling.cc:243] Waited 5.09 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:47.826098 (+ 97653us) raft_consensus.cc:1685] finished
0709 13:01:47.826128 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.826286 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30194,"prepare.queue_time_us":74757,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":15325,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":41934,"spinlock_wait_cycles":22747264,"thread_start_us":106962,"threads_started":1,"wal-append.queue_time_us":287443}
I20260709 13:01:47.835541  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:47.820494  6414 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:47.804325  6381 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.837415  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.836573  1539 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.818701  6466 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:01:47.827948  6342 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.855446  6404 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.834985  6325 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.856431  6305 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.825634  6288 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:47.858224  6325 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.858939  6288 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.836844  6132 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:47.862614  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:47.863344  2236 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:01:47.842682  1538 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:47.863511  6337 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:01:47.864436  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 894) took 1354 ms. Trace:
I20260709 13:01:47.864799  1538 rpcz_store.cc:276] 0709 13:01:46.510176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.842018 (+1331842us) service_pool.cc:224] Handling call
0709 13:01:47.842452 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.864172 (+ 21720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.864382 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.865762  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:47.866411  1611 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:47.866667  6436 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.834815  6416 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:47.841514  6390 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:47.869478  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:47.869836  6416 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.870301  6416 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:47.873713  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.875706  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:47.876420  1538 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:01:47.845692  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:47.850152  6381 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:47.852500  6414 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.833226  6205 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:47.873558  6342 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.877552  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:47.879292  6324 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:47.880277  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 922) took 1394 ms. Trace:
W20260709 13:01:47.882655  1610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1836) took 2277 ms (client timeout 2322 ms). Trace:
I20260709 13:01:47.886515  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:47.886806  1610 rpcz_store.cc:269] 0709 13:01:45.604849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.005604 (+1400755us) service_pool.cc:224] Handling call
0709 13:01:47.882439 (+876835us) spinlock_profiling.cc:243] Waited 94.7 ms on lock 0x7b5400398ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:47.882577 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":217908992}
I20260709 13:01:47.887176  1543 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:47.845857  1539 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:47.889585  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2328) took 2259 ms. Trace:
W20260709 13:01:47.890663  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:47.888708  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:47.885552  2229 rpcz_store.cc:276] 0709 13:01:46.486123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.486254 (+   131us) service_pool.cc:224] Handling call
0709 13:01:46.509275 (+ 23021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.586675 (+ 77400us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:46.603382 (+ 16707us) log.cc:844] Serialized 14 byte log entry
0709 13:01:47.024650 (+421268us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:47.024731 (+    81us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:47.024798 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.880021 (+855223us) spinlock_profiling.cc:243] Waited 77.2 ms on lock 0x7b5400484ed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:47.880066 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.880200 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24786,"prepare.queue_time_us":64324,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":129,"raft.run_cpu_time_us":839,"raft.run_wall_time_us":7519,"spinlock_wait_cycles":177449856,"thread_start_us":372910,"threads_started":1,"wal-append.queue_time_us":471567}
I20260709 13:01:47.891475  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.837786  6320 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:47.896373  1539 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:47.897310  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 896) took 1386 ms. Trace:
I20260709 13:01:47.898520  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:47.890836  6381 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:47.898322  1539 rpcz_store.cc:276] 0709 13:01:46.510694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.889306 (+1378612us) service_pool.cc:224] Handling call
0709 13:01:47.896204 (+  6898us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.896952 (+   748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.897236 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:47.903204  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:47.895949  6320 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.907893  6320 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:47.910706  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 895) took 1400 ms. Trace:
I20260709 13:01:47.911176  1538 rpcz_store.cc:276] 0709 13:01:46.510450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.877020 (+1366570us) service_pool.cc:224] Handling call
0709 13:01:47.877325 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.883728 (+  6403us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:01:47.897867 (+ 14139us) log.cc:844] Serialized 14 byte log entry
0709 13:01:47.910355 (+ 12488us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:47.910421 (+    66us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:01:47.910469 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.910531 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.910653 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":10684,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":70,"raft.queue_time_us":92,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"spinlock_wait_cycles":2676992,"thread_start_us":7192,"threads_started":1,"wal-append.queue_time_us":11554}
I20260709 13:01:47.913662  6205 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.895638  6305 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:47.919930  1691 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:47.920111  1611 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:47.918123  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:47.921752  1685 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:47.926695  1539 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: [8.6-10.7]
I20260709 13:01:47.928232  1539 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:01:47.917389  6305 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:47.932003  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.932693  1611 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:47.932229  6305 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:47.933238  1683 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:47.933717  1683 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:47.935094  6479 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:01:47.934801  1683 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.936023  1683 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:47.936384  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.938674  1689 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:47.939555  1689 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:39812
I20260709 13:01:47.940167  1689 raft_consensus.cc:606] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Received request to transfer leadership
I20260709 13:01:47.943120  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2321) took 2630 ms. Trace:
I20260709 13:01:47.943532  1609 rpcz_store.cc:276] 0709 13:01:45.313026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.570631 (+2257605us) service_pool.cc:224] Handling call
0709 13:01:47.570918 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.664751 (+ 93833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.942889 (+278138us) spinlock_profiling.cc:243] Waited 93.2 ms on lock 0x7b540008c0d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:47.943061 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214452096}
I20260709 13:01:47.944628  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.933640  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.947113  1611 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:47.949157  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.950052  1618 raft_consensus.cc:2410] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:01:47.904660  6381 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:47.953673  6381 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:47.957921  6423 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:47.958869  6423 consensus_queue.cc:1243] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:01:47.959408  6423 raft_consensus.cc:3055] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:47.959728  6423 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:47.961802  1541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2824) took 1767 ms (client timeout 1556 ms). Trace:
W20260709 13:01:47.962280  1541 rpcz_store.cc:269] 0709 13:01:46.184986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.193977 (+  8991us) service_pool.cc:224] Handling call
0709 13:01:47.951996 (+1758019us) spinlock_profiling.cc:243] Waited 52.8 ms on lock 0x7b54000e92d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:47.952101 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":121394432}
I20260709 13:01:47.965765  1689 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:39812
I20260709 13:01:47.966307  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.966513  1689 raft_consensus.cc:606] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Received request to transfer leadership
I20260709 13:01:47.960659  6423 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.966945  1609 raft_consensus.cc:2376] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12. Re-sending same reply.
I20260709 13:01:47.967980  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:47.968618  1611 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:47.970155  1539 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:01:47.975960  6320 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.989944  1539 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:47.996255  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:47.917313  6132 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:47.998792  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:48.000633  1618 raft_consensus.cc:2410] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:48.002853  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:48.003573  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:48.004266  6132 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:48.004724  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:48.005002  2232 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:48.011727  6408 raft_consensus.cc:3055] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:48.012195  6408 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:48.013705  1426 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for caae9b2a426d42fb8c2353e1c978d0fb already in progress [suppressed 40 similar messages]
I20260709 13:01:48.019218  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:48.013377  6381 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.031386  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:48.032090  2229 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.031388  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:48.033646  1618 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:48.033634  1657 connection.cc:256] RPC call timeout handler was delayed by -1.520846052999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
I20260709 13:01:48.034444  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:48.035401  2232 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:47.973534  6398 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:01:48.002429  6423 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:48.048269  6491 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.138s	user 0.001s	sys 0.001s
I20260709 13:01:48.049103  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.050027  1618 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 11
W20260709 13:01:48.051332  1540 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2307) took 1699 ms (client timeout 1964 ms). Trace:
W20260709 13:01:48.051774  1540 rpcz_store.cc:269] 0709 13:01:46.352045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.354206 (+  2161us) service_pool.cc:224] Handling call
0709 13:01:48.051123 (+1696917us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b54003e1f54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:48.051272 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43742976}
I20260709 13:01:48.056105  2229 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:48.058892  2232 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:48.013018  6408 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:48.062191  1657 connection.cc:256] RPC call timeout handler was delayed by -1.0260049700000309s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
W20260709 13:01:48.072109  1657 connection.cc:256] RPC call timeout handler was delayed by -1.0107321879999631s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1845s for a response
I20260709 13:01:48.073215  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.075910  1657 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:48.081090  6320 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:48.084429  6321 consensus_queue.cc:1059] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:48.085812  1657 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.086617  6321 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:48.087430  6408 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:48.088706  6321 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:48.068349  6342 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:48.090585  1657 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.090588  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:48.082736  6075 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:48.100960  1618 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:48.103235  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.105325  1540 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:48.104107  1618 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:48.081248  6404 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:48.081158  6432 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:48.097826  1426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 96276.9us
I20260709 13:01:48.102239  6320 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:48.115875  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.103837  6075 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:48.103029  6340 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:48.127382  6075 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.128197  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2305) took 1776 ms. Trace:
I20260709 13:01:48.141165  1534 rpcz_store.cc:276] 0709 13:01:46.351554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.743607 (+392053us) service_pool.cc:224] Handling call
0709 13:01:46.751121 (+  7514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.923101 (+171980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.127929 (+1204828us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b54000ada54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:48.128132 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80339328}
I20260709 13:01:48.118898  6352 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
W20260709 13:01:48.132496  6488 log.cc:927] Time spent T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:01:48.133870  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:48.139678  1618 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:01:48.147686  1534 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:48.151652  2232 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:48.152024  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:48.125099  6432 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:48.154309  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.157208  1618 raft_consensus.cc:2368] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16.
I20260709 13:01:48.122171  6317 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:01:48.142937  6320 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:01:48.164850  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:48.165340  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2329) took 2529 ms. Trace:
I20260709 13:01:48.165656  1534 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:48.166764  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:48.167407  1534 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.165998  1616 rpcz_store.cc:276] 0709 13:01:45.636135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.759882 (+2123747us) service_pool.cc:224] Handling call
0709 13:01:47.760265 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:47.857268 (+ 97003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:47.857301 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:47.857837 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:01:47.966169 (+108332us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:01:47.966229 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.015448 (+ 49219us) spinlock_profiling.cc:243] Waited 96.6 ms on lock 0x7b540028ffd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:48.015516 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:48.164909 (+149393us) raft_consensus.cc:1685] finished
0709 13:01:48.164936 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.165256 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6224,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":222222208,"thread_start_us":107942,"threads_started":1,"wal-append.queue_time_us":176653}
I20260709 13:01:48.171013  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:48.171998  1616 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:48.173101  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.176868  1616 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:48.175719  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.178630  1618 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:48.182569  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.178267  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.143690  6404 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.135210  6340 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:48.186115  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.192417  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 898) took 1663 ms. Trace:
I20260709 13:01:48.192950  1690 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:39812
I20260709 13:01:48.193615  1690 raft_consensus.cc:606] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Received request to transfer leadership
I20260709 13:01:48.193674  6320 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:48.194028  1690 raft_consensus.cc:614] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:01:48.194896  6481 log.cc:927] Time spent T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.395s	user 0.001s	sys 0.000s
I20260709 13:01:48.196152  6340 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.197291  6340 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.198186  1540 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:48.199258  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:48.199489  6321 raft_consensus.cc:3055] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:48.199597  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:48.199929  6321 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:48.200500  1535 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:48.206342  6497 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:01:48.207185  2235 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:47.890129  1615 rpcz_store.cc:276] 0709 13:01:45.623707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.742884 (+2119177us) service_pool.cc:224] Handling call
0709 13:01:47.744209 (+  1325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.744425 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:47.827590 (+ 83165us) log.cc:844] Serialized 14 byte log entry
0709 13:01:47.882566 (+ 54976us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:47.882637 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:47.882706 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.882800 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.882941 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":117293,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":1286,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":7979,"thread_start_us":137451,"threads_started":2}
I20260709 13:01:48.215407  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.216189  1615 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:48.217309  1617 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2338) took 2151 ms (client timeout 2708 ms). Trace:
W20260709 13:01:48.217756  1617 rpcz_store.cc:269] 0709 13:01:46.065951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.452304 (+1386353us) service_pool.cc:224] Handling call
0709 13:01:48.217110 (+764806us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b54003c87d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:48.217258 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32533888}
I20260709 13:01:48.218814  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:48.219480  1617 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10.
I20260709 13:01:48.221401  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 790) took 3011 ms. Trace:
I20260709 13:01:48.221850  1614 rpcz_store.cc:276] 0709 13:01:45.209898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.210306 (+   408us) service_pool.cc:224] Handling call
0709 13:01:46.929085 (+1718779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.108632 (+179547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.221170 (+1112538us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b540008c5d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:48.221313 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3918614912}
I20260709 13:01:48.222471  6467 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:48.222877  6467 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:48.223026  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.223619  1614 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.223861  6467 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.225042  6317 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:48.226193  1613 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:48.192868  1543 rpcz_store.cc:276] 0709 13:01:46.529388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.892446 (+1363058us) service_pool.cc:224] Handling call
0709 13:01:48.051269 (+158823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.051468 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.051507 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:48.051549 (+    42us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:48.051588 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.051641 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.192229 (+140588us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54003e1f50. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:48.192349 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364331392}
I20260709 13:01:48.227532  1618 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:48.228339  1543 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:48.217110  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.183928  1616 raft_consensus.cc:2376] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 10. Re-sending same reply.
I20260709 13:01:48.122440  6416 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:48.237128  1535 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:48.220431  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:48.239310  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:48.240412  1617 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:48.225106  6467 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:47.803138  1537 rpcz_store.cc:276] 0709 13:01:46.350430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.714350 (+363920us) service_pool.cc:224] Handling call
0709 13:01:46.714793 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:46.715076 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:46.744001 (+ 28925us) log.cc:844] Serialized 14 byte log entry
0709 13:01:47.710033 (+966032us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b1800338140. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a 00007fe08faf6a2a
0709 13:01:47.802093 (+ 92060us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:47.802143 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:47.802186 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:47.802245 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:47.802382 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29253,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":3873,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":2892,"spinlock_wait_cycles":18554624,"thread_start_us":87940,"threads_started":1}
I20260709 13:01:48.233700  1507 leader_election.cc:400] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
I20260709 13:01:48.249861  1507 leader_election.cc:403] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:01:48.199896  1540 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 8.
I20260709 13:01:48.153537  6394 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:48.172607  6129 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 13:01:48.236251  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.241654  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.269810  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.270366  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.232715  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.271029  1616 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:48.245946  1535 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:48.255928  1543 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:48.272655  1610 raft_consensus.cc:2368] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12.
I20260709 13:01:48.225752  6317 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:48.270429  1612 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:48.239315  1509 leader_election.cc:400] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12.
I20260709 13:01:48.277020  1509 leader_election.cc:403] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:48.277347  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:48.278062  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.280225  1509 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:48.254101  1614 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:48.282306  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:48.247673  6342 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.265893  1541 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:48.285468  1617 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 14: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:48.289510  1615 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:48.284536  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:48.301093  1543 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:48.301642  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.302534  1615 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:48.285048  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.308362  1692 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:01:48.308807  1692 pending_rounds.cc:85] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 12
I20260709 13:01:48.309017  1692 pending_rounds.cc:107] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:01:48.257339  6394 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.309346  1692 raft_consensus.cc:2889] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:48.240168  6408 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.092s
I20260709 13:01:48.232883  6404 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.313814  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:48.314440  1614 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 9.
I20260709 13:01:48.274463  6070 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:48.276434  6122 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.316036  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:48.324718  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:48.325343  1540 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:48.329967  1543 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:01:48.201092  6321 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.319900  5929 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:48.364951  5929 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:48.365362  5929 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.365659  6449 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:48.366063  6449 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:48.366858  6449 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.367820  6449 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:48.313508  6394 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:48.370203  6129 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:48.370661  6129 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:48.371043  6129 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.322651  6416 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:48.274046  6305 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:48.372016  6305 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:48.382941  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:48.383800  1684 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:48.384021  6305 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:48.332057  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:48.286827  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:48.386545  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 99902.7us
I20260709 13:01:48.348951  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:48.353068  2235 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:01:48.388458  1614 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:48.353096  1615 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:48.311955  6342 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:48.360961  1609 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:01:48.393378  1609 raft_consensus.cc:606] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:48.364001  6317 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:48.391281  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:48.361639  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:48.407382  1583 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.407644  1617 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:48.400691  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.409008  1534 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:48.402557  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:48.406780  1615 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:48.401922  1609 raft_consensus.cc:614] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:48.352538  6381 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:48.420303  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:48.409965  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:48.421213  2237 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:48.423010  1687 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.411113  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:48.424206  1684 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:48.424516  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:48.410681  1612 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:01:48.423239  1614 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:01:48.426458  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:48.427520  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2339) took 2359 ms. Trace:
I20260709 13:01:48.428052  1614 rpcz_store.cc:276] 0709 13:01:46.068388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.389182 (+2320794us) service_pool.cc:224] Handling call
0709 13:01:48.422957 (+ 33775us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.427139 (+  4182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.427439 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.430033  1611 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:48.414968  1508 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:48.431346  1508 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.414459  1583 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:48.432196  1612 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:48.410501  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.434724  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2340) took 2009 ms. Trace:
I20260709 13:01:48.435416  1615 rpcz_store.cc:276] 0709 13:01:46.425082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.400588 (+1975506us) service_pool.cc:224] Handling call
0709 13:01:48.406459 (+  5871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.434325 (+ 27866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.434625 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.441174  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:48.442906  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:48.448941  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 803) took 1932 ms. Trace:
I20260709 13:01:48.449530  1612 rpcz_store.cc:276] 0709 13:01:46.501533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.431547 (+1930014us) service_pool.cc:224] Handling call
0709 13:01:48.431960 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.433079 (+  1119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.433302 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.450155  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2711) took 2494 ms. Trace:
I20260709 13:01:48.450345  1617 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 13:01:48.451810  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:48.452646  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2342) took 2022 ms. Trace:
I20260709 13:01:48.453125  1617 rpcz_store.cc:276] 0709 13:01:46.430548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.408433 (+1977885us) service_pool.cc:224] Handling call
0709 13:01:48.435934 (+ 27501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.452295 (+ 16361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.452578 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.454807  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:48.457074  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:48.401139  6337 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:48.458397  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:48.463407  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 53006.7us
I20260709 13:01:48.450839  1691 rpcz_store.cc:276] 0709 13:01:45.955549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:45.955701 (+   152us) service_pool.cc:224] Handling call
0709 13:01:46.010693 (+ 54992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:47.925456 (+1914763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.449895 (+524439us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b540014a854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:48.450100 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4390307712}
I20260709 13:01:48.459095  1685 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:01:48.466131  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:48.409507  6305 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:48.463443  1689 raft_consensus.cc:2376] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12. Re-sending same reply.
I20260709 13:01:48.468611  1615 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 13:01:48.470335  1615 pending_rounds.cc:85] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 6
I20260709 13:01:48.470592  1615 pending_rounds.cc:107] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:01:48.470786  1614 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:48.470809  1615 raft_consensus.cc:2889] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:48.471406  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 801) took 1978 ms. Trace:
I20260709 13:01:48.471655  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2344) took 1769 ms. Trace:
I20260709 13:01:48.471731  1614 rpcz_store.cc:276] 0709 13:01:46.493268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.429066 (+1935798us) service_pool.cc:224] Handling call
0709 13:01:48.470488 (+ 41422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.471186 (+   698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.471357 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.472079  1615 rpcz_store.cc:276] 0709 13:01:46.702089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.436534 (+1734445us) service_pool.cc:224] Handling call
0709 13:01:48.468281 (+ 31747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.471448 (+  3167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.471610 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:48.425613  6317 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:48.470992  1689 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:01:48.420747  5968 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
I20260709 13:01:48.466125  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:48.476624  1689 pending_rounds.cc:85] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 10
I20260709 13:01:48.479140  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:48.479836  1688 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:48.457743  6381 raft_consensus.cc:2764] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:01:48.462256  6416 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:48.483310  1617 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:48.479352  1689 pending_rounds.cc:107] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:01:48.509878  1689 raft_consensus.cc:2889] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:48.508812  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:48.513867  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.513737  1686 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:48.515452  1614 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:48.476217  6388 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:48.512099  1684 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:48.518013  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:48.519119  1688 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:48.520520  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:48.512948  6416 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:48.526008  6416 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:48.532305  1614 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: [11.11-16.12]
W20260709 13:01:48.534158  6494 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:01:48.478121  6132 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:48.462070  6393 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 13:01:48.477888  6337 raft_consensus.cc:697] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:48.535523  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2840) took 1230 ms. Trace:
I20260709 13:01:48.539742  1538 rpcz_store.cc:276] 0709 13:01:47.310130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.912769 (+602639us) service_pool.cc:224] Handling call
0709 13:01:47.913097 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:47.920697 (+  7600us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:47.920726 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:47.921134 (+   408us) log.cc:844] Serialized 26 byte log entry
0709 13:01:48.139838 (+218704us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:48.139896 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.139968 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:48.535233 (+395265us) raft_consensus.cc:1685] finished
0709 13:01:48.535267 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.535442 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":102,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":218403,"threads_started":1,"wal-append.queue_time_us":386500}
I20260709 13:01:48.544149  1689 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:48.545646  6129 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:48.548036  6129 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:48.548952  6129 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:48.549391  6129 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:48.550220  6129 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:48.553908  1689 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:48.554113  2201 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:48.555372  1683 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:48.556563  2237 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:01:48.560421  1609 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:48.559964  1686 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
W20260709 13:01:48.558252  6496 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:01:48.558853  6466 raft_consensus.cc:2764] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:01:48.561980  6381 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:48.560655  6132 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:48.590092  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 899) took 2054 ms. Trace:
I20260709 13:01:48.573585  6122 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:48.593144  1542 rpcz_store.cc:276] 0709 13:01:46.535329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.901182 (+1365853us) service_pool.cc:224] Handling call
0709 13:01:47.917960 (+ 16778us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:47.918371 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:47.918409 (+    38us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:47.919049 (+   640us) log.cc:844] Serialized 78 byte log entry
0709 13:01:48.247977 (+328928us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:48.248024 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.248090 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:48.589851 (+341761us) raft_consensus.cc:1685] finished
0709 13:01:48.589872 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:48.590025 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2306,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":4265,"thread_start_us":328526,"threads_started":1,"wal-append.queue_time_us":404797}
I20260709 13:01:48.561836  6325 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:01:48.618422  1688 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:48.622663  6423 consensus_meta.cc:287] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.571s	user 0.010s	sys 0.002s
I20260709 13:01:48.601452  1616 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
W20260709 13:01:48.655769  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 267967us
I20260709 13:01:48.608748  6405 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:48.656023  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.633181  1540 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
W20260709 13:01:48.640631  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 360456us
I20260709 13:01:48.658759  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.660319  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:48.653932  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 222589us
I20260709 13:01:48.661820  1508 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.678274  1508 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.678803  2229 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:48.627686  6504 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:48.680377  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:48.654732  6193 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12.
I20260709 13:01:48.662328  6342 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:48.628083  6505 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:01:48.621048  6337 consensus_queue.cc:237] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:48.689604  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:48.698681  5929 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.699841  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2351) took 1393 ms. Trace:
I20260709 13:01:48.700390  1617 rpcz_store.cc:276] 0709 13:01:47.306138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.454065 (+1147927us) service_pool.cc:224] Handling call
0709 13:01:48.469659 (+ 15594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.509663 (+ 40004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.699529 (+189866us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b540008bbd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:48.699764 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30726528}
I20260709 13:01:48.701687  5929 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:48.702406  5929 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:48.703105  5929 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:48.704129  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2335) took 2751 ms. Trace:
I20260709 13:01:48.704617  1611 rpcz_store.cc:276] 0709 13:01:45.953277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.362913 (+2409636us) service_pool.cc:224] Handling call
0709 13:01:48.429702 (+ 66789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.430567 (+   865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.703886 (+273319us) spinlock_profiling.cc:243] Waited 66.4 ms on lock 0x7b5400001c50. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:48.704057 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152779008}
I20260709 13:01:48.705755  6132 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.707646  1611 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:01:48.712978  1687 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:48.714344  1692 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:01:48.714599  1611 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:48.675432  6405 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:01:48.687618  6311 raft_consensus.cc:2764] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:01:48.675702  6467 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:01:48.734838  6405 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:48.681721  6340 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:01:48.735324  6405 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:48.710215  6506 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:48.736264  6405 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:48.718248  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 252190us
W20260709 13:01:48.719956  6499 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:01:48.689352  6512 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:48.720041  6394 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.720642  6404 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:48.723105  6466 raft_consensus.cc:3055] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:48.729750  6275 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 13:01:48.731992  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 498452us
W20260709 13:01:48.733184  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.692102  6310 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:01:48.741472  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:48.732041  6103 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:48.750844  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 281703us
I20260709 13:01:48.754757  1583 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:48.760088  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:48.768599  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.768630  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:48.774493  6504 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:01:48.777274  6467 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:48.777956  6467 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:48.776830  6129 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.778896  6467 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:48.780998  6129 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:48.783645  6394 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:48.798843  1583 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.750545  6405 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:48.771821  6505 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.751636  6466 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:48.820546  1687 pending_rounds.cc:85] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 9
I20260709 13:01:48.787212  6393 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:48.825745  1687 pending_rounds.cc:107] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 10.10
W20260709 13:01:48.821015  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 259833us
I20260709 13:01:48.826082  1687 raft_consensus.cc:2889] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:48.801743  6342 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:48.758239  6311 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 13:01:48.772544  6103 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.823705  6394 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.858026  2201 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:48.822021  6515 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:01:48.866459  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.823658  6381 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
W20260709 13:01:48.869233  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 208998us
I20260709 13:01:48.837023  6505 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.852720  6325 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:01:48.899537  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:48.902841  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:48.860502  6423 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:01:48.905145  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:48.888970  6129 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.863579  6394 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.895799  5885 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:48.906833  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:48.912952  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 139289us
I20260709 13:01:48.919379  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:48.904511  6512 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:48.904307  6506 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:48.871136  6466 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:48.892472  6335 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:01:48.932026  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1852) took 1129 ms. Trace:
I20260709 13:01:48.906915  6504 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:01:48.909126  5929 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:48.921666  1509 leader_election.cc:400] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16.
I20260709 13:01:48.932618  1610 rpcz_store.cc:276] 0709 13:01:47.802477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.898188 (+ 95711us) service_pool.cc:224] Handling call
0709 13:01:48.930043 (+1031855us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54002dc8d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:48.931945 (+  1902us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":856152832}
I20260709 13:01:48.935354  1540 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:01:48.919502  5885 raft_consensus.cc:697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:48.936578  1509 leader_election.cc:403] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:01:48.937909  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:48.945199  1509 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:48.946346  1610 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:01:48.944070  1538 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:01:48.946938  1610 raft_consensus.cc:606] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:48.947286  1610 raft_consensus.cc:614] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:48.915030  6129 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:48.930871  6352 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:48.954447  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 96512.2us
W20260709 13:01:48.934551  6513 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:01:48.956390  1542 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:48.958886  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:48.948501  1509 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:48.960883  1610 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:01:48.934737  6524 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:01:48.962024  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.962766  1617 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:48.932430  6340 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 13:01:48.970201  6129 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:48.975577  6500 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.319s	user 0.002s	sys 0.000s
W20260709 13:01:48.967396  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:48.967576  1610 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:48.952862  6506 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:01:48.935415  6337 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:01:48.963904  6342 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:48.982095  6103 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:48.982918  6514 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:48.983371  5929 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:48.983908  2229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:01:49.060108  2229 raft_consensus.cc:606] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:49.060541  2229 raft_consensus.cc:614] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:48.984642  1690 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:01:48.980885  2203 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:48.964864  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:01:48.988976  6522 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:01:48.989779  6324 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:48.955376  6352 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:48.961793  6520 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 13:01:48.948038  6466 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:48.950105  6512 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16.
I20260709 13:01:49.052492  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 815) took 1260 ms. Trace:
I20260709 13:01:49.086210  1609 rpcz_store.cc:276] 0709 13:01:47.792032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.475911 (+683879us) service_pool.cc:224] Handling call
0709 13:01:48.476193 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.560858 (+ 84665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.052128 (+491270us) spinlock_profiling.cc:243] Waited 84 ms on lock 0x7b54000c7e54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:49.052404 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":193298816}
W20260709 13:01:49.062057  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 97281.2us
I20260709 13:01:49.088274  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:49.064883  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 84064.9us
I20260709 13:01:49.067608  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2334) took 3273 ms. Trace:
I20260709 13:01:49.012457  6524 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:48.966082  6325 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
W20260709 13:01:49.060292  6513 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:01:49.089819  2203 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:49.089232  1613 rpcz_store.cc:276] 0709 13:01:45.794287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:47.789842 (+1995555us) service_pool.cc:224] Handling call
0709 13:01:48.225999 (+436157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.226606 (+   607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.067321 (+840715us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b54003c87d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:49.067529 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1002330496}
I20260709 13:01:49.091473  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:49.067665  6129 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:01:49.066407  6514 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:49.098917  6517 log.cc:927] Time spent T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.212s	user 0.001s	sys 0.001s
I20260709 13:01:49.100838  1613 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:01:49.102720  1509 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:49.083649  6352 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:49.105695  1613 raft_consensus.cc:606] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:49.084506  6324 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.106491  1509 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:49.103823  6512 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:01:49.124150  6416 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.598s	user 0.002s	sys 0.011s
I20260709 13:01:49.145408  6506 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:49.101754  6129 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:49.126739  1613 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:01:49.066754  5885 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:49.130584  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.151974  6394 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:49.154928  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2349) took 1853 ms. Trace:
I20260709 13:01:49.153787  6129 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:49.154716  6524 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.155402  1612 rpcz_store.cc:276] 0709 13:01:47.301371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.452032 (+1150661us) service_pool.cc:224] Handling call
0709 13:01:48.462042 (+ 10010us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.506796 (+ 44754us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.506827 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:48.512539 (+  5712us) log.cc:844] Serialized 26 byte log entry
0709 13:01:48.680311 (+167772us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:48.680482 (+   171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.680558 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.154711 (+474153us) raft_consensus.cc:1685] finished
0709 13:01:49.154738 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.154872 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44281,"raft.queue_time_us":45643,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":5995,"thread_start_us":129083,"threads_started":1,"wal-append.queue_time_us":300665}
I20260709 13:01:49.156960  6352 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:49.158867  6324 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:49.154595  6467 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:49.159799  6467 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:49.143651  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 468202us
I20260709 13:01:49.162084  6467 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:49.162591  6352 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:49.164920  6514 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.146997  1509 leader_election.cc:400] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12.
W20260709 13:01:49.148125  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.144950  6518 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:49.130376  6536 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:49.152032  1613 raft_consensus.cc:606] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:01:49.169965  1613 raft_consensus.cc:614] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:49.093986  5929 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:49.109230  6505 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:49.109701  6504 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:49.171219  1688 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:49.172804  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:49.148671  6342 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.149893  6325 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:01:49.173837  1509 leader_election.cc:403] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:01:49.171597  5929 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:49.179584  5929 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:49.191105  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44186.4us
W20260709 13:01:49.193590  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.194511  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.170969  6506 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:49.172070  6518 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:49.172205  6536 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.197453  1613 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eed65fd1add844e7821095e2215fb393"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:57992
I20260709 13:01:49.200824  6466 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:01:49.202890  6526 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:01:49.188959  6505 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:01:49.194401  6512 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12.
I20260709 13:01:49.211365  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 814) took 1419 ms. Trace:
I20260709 13:01:49.151556  6416 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.211807  1615 rpcz_store.cc:276] 0709 13:01:47.791717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.472941 (+681224us) service_pool.cc:224] Handling call
0709 13:01:48.507329 (+ 34388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.681406 (+174077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.681456 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:48.684538 (+  3082us) log.cc:844] Serialized 26 byte log entry
0709 13:01:48.886317 (+201779us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:01:48.915727 (+ 29410us) log.cc:844] Serialized 14 byte log entry
0709 13:01:48.917540 (+  1813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.917632 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.202256 (+284624us) spinlock_profiling.cc:243] Waited 3.53 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:49.211022 (+  8766us) raft_consensus.cc:1685] finished
0709 13:01:49.211147 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.211296 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18651,"prepare.queue_time_us":38706,"raft.queue_time_us":29291,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":23595,"spinlock_wait_cycles":19610240,"thread_start_us":372878,"threads_started":2,"wal-append.queue_time_us":239964}
I20260709 13:01:49.220498  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:49.221360  2230 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:49.223973  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:49.224838  1611 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:49.222895  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:49.228590  1609 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
W20260709 13:01:49.230329  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 98657us
I20260709 13:01:49.233109  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:49.239745  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:49.230484  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:49.242379  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:49.243321  1686 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:49.242379  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:49.242605  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:49.242466  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:49.247577  1613 raft_consensus.cc:606] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:01:49.248077  1613 raft_consensus.cc:614] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:49.248736  2233 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:01:49.209218  6536 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:49.209033  6518 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:49.252686  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:49.249495  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:49.240674  6193 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.248333  6538 raft_consensus.cc:3055] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:01:49.248076  6381 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:49.252446  6423 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:49.257603  6505 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:49.370025  6505 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.370582  6505 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:49.258167  6540 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:49.373797  6540 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.374277  6540 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:49.377825  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:49.239735  6394 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
W20260709 13:01:49.366667  6539 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 13:01:49.346844  6193 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.367619  6506 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:49.244128  6504 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:01:49.344259  6538 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:49.393784  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:49.405495  1537 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 9.
I20260709 13:01:49.403362  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:49.404150  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 173593us
I20260709 13:01:49.256631  6466 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.140s
I20260709 13:01:49.408564  1508 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:49.360950  6514 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.364813  6423 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:49.424319  6394 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:49.426559  6394 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.427028  6394 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:49.436023  6393 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.361368  6381 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:49.436626  6393 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.418078  6504 consensus_queue.cc:1243] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer 8a6a5ae1b7f240408b3ddd30d8a18262 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:01:49.446509  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 215318us
I20260709 13:01:49.455710  6193 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:49.456364  6193 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.458168  6538 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.459451  6538 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:49.459499  6335 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:49.460170  6335 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:01:49.462688  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.464624  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:49.466197  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:49.473873  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:49.456828  6193 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.489380  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:49.512734  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:49.471939  6321 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:49.529150  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.423818  6514 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.535604  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:49.538746  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:49.450734  6504 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 13:01:49.488847  6311 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:49.454080  6129 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:49.461324  6335 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.548820  6335 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:49.549996  6129 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:49.550822  1489 auto_rebalancer.cc:893] Removing replace marker failed: Replica 984c9089d9804652846f3caeb609d56e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:01:49.551345  1489 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:01:49.551667  1489 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:01:49.552394  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:49.553943  6525 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.619s	user 0.000s	sys 0.012s
W20260709 13:01:49.554759  6527 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.492s	user 0.003s	sys 0.000s
I20260709 13:01:49.556586  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 819) took 1763 ms. Trace:
W20260709 13:01:49.557049  6531 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.502s	user 0.000s	sys 0.004s
I20260709 13:01:49.557010  1614 rpcz_store.cc:276] 0709 13:01:47.793243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.533231 (+739988us) service_pool.cc:224] Handling call
0709 13:01:48.535824 (+  2593us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:48.536180 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.536213 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:48.543704 (+  7491us) log.cc:844] Serialized 52 byte log entry
0709 13:01:48.872362 (+328658us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:01:48.902318 (+ 29956us) log.cc:844] Serialized 14 byte log entry
0709 13:01:48.902548 (+   230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.902624 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.179243 (+276619us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b48001a01d8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:01:49.470097 (+290854us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:49.556306 (+ 86209us) raft_consensus.cc:1685] finished
0709 13:01:49.556336 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:49.556507 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":31801,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":28198,"raft.run_cpu_time_us":12267,"raft.run_wall_time_us":611095,"spinlock_wait_cycles":15890560,"thread_start_us":315447,"threads_started":1,"wal-append.queue_time_us":405321}
I20260709 13:01:49.510519  6543 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
W20260709 13:01:49.562747  6534 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.504s	user 0.002s	sys 0.000s
I20260709 13:01:49.570549  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:49.577812  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 977) took 1154 ms. Trace:
I20260709 13:01:49.578452  1683 rpcz_store.cc:276] 0709 13:01:48.423624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.442586 (+ 18962us) service_pool.cc:224] Handling call
0709 13:01:49.577613 (+1135027us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54002cd154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:49.577744 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256787840}
I20260709 13:01:49.586194  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:49.509405  5885 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:01:49.595824  1584 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:49.524709  6405 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:01:49.562822  6381 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:49.573962  6416 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:49.576665  6543 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.582386  6129 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:49.618918  6524 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:49.619452  5885 consensus_queue.cc:1243] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:01:49.511680  6423 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.546252  6340 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:01:49.624425  6476 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:49.578709  6545 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:01:49.597818  6305 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:49.582312  6408 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:01:49.594870  6557 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:49.574344  6311 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:01:49.719808  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:49.585012  6337 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 13:01:49.626458  6342 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:49.779024  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 176933us
W20260709 13:01:49.773938  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 239314us
I20260709 13:01:49.597465  6404 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.121s
I20260709 13:01:49.599834  6322 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.128s
W20260709 13:01:49.768987  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 346980us
I20260709 13:01:49.809473  2203 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:49.809654  1508 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:49.773253  6530 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.718s	user 0.003s	sys 0.000s
I20260709 13:01:49.651118  6476 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:49.808033  2228 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:01:49.783434  5929 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.603s	user 0.012s	sys 0.000s
I20260709 13:01:49.878955  6476 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:49.879871  5929 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.883265  5929 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:49.857918  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:49.862425  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:49.783581  6542 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:49.862769  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 82061.5us
I20260709 13:01:49.885124  6542 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.887382  5929 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:49.887346  6542 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:49.889707  1612 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:01:49.890046  1614 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:01:49.639359  6543 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.640475  6524 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.768361  6559 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:01:49.877327  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.898738  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:49.896919  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 87533.4us
I20260709 13:01:49.880455  6476 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:49.903429  6551 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.306s	user 0.001s	sys 0.000s
I20260709 13:01:49.903795  6524 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:49.639964  6129 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:49.904723  6536 consensus_meta.cc:287] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.643s	user 0.010s	sys 0.000s
I20260709 13:01:49.857852  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:49.906649  2237 raft_consensus.cc:2410] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:01:49.863469  6337 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:01:49.897186  6552 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
I20260709 13:01:49.897533  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:49.861460  5968 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:49.896068  6518 consensus_meta.cc:287] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
I20260709 13:01:49.905845  6305 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.918175  2235 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:01:49.876134  6381 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.900511  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:49.897221  6543 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:49.892364  6542 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:49.822599  6540 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:49.899745  6559 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:49.776098  6423 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:49.905192  6536 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.909865  6337 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:49.944746  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44360.3us
I20260709 13:01:49.779446  6342 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:49.780061  6557 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:49.952062  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:49.968150  1536 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:01:49.970005  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:49.970860  1536 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:01:49.942869  6193 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:49.930097  6518 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.975508  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:49.976351  1539 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:49.977531  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:49.977682  6518 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:49.977885  6503 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.323s	user 0.010s	sys 0.010s
I20260709 13:01:49.967566  5929 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:49.979166  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:49.979318  5929 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:49.949410  6574 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:01:49.981752  6566 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:49.982565  6566 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:49.982589  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:49.986182  1534 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:49.946718  6542 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:49.988265  1539 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:49.988574  1542 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:49.948024  6129 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:49.962155  6394 consensus_meta.cc:287] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.535s	user 0.012s	sys 0.000s
I20260709 13:01:49.959440  6342 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:49.929877  5968 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:49.934190  6381 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.982916  6566 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:49.934316  6305 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:49.942106  6533 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:01:50.004453  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1323) took 1829 ms. Trace:
I20260709 13:01:49.964320  6543 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:01:50.012511  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 202927us
I20260709 13:01:50.019160  2232 rpcz_store.cc:276] 0709 13:01:48.175147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.175308 (+   161us) service_pool.cc:224] Handling call
0709 13:01:48.195781 (+ 20473us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:48.196186 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.196215 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:48.196739 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 13:01:48.436848 (+240109us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:48.436887 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.436953 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.876349 (+1439396us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:49.983560 (+107211us) raft_consensus.cc:1685] finished
0709 13:01:49.983581 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.004379 (+ 20798us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5451,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":9697536,"thread_start_us":239676,"threads_started":1,"wal-append.queue_time_us":394899}
I20260709 13:01:50.001905  6550 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 13:01:50.023141  6550 consensus_queue.cc:1243] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 8a6a5ae1b7f240408b3ddd30d8a18262 log is divergent from this leader: its last log entry 10.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:01:50.025384  6559 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.027077  6559 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:49.943948  6569 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:01:50.028347  1685 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:01:50.028891  6352 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:50.029917  6553 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 13:01:50.031998  5968 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:50.034519  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.039407  6381 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.041989  6569 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:50.042708  6569 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:50.044382  6561 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.149s	user 0.001s	sys 0.001s
I20260709 13:01:50.022078  6414 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.029491  6352 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.994340  6394 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.048802  6414 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:49.996657  6574 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:50.062624  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 59998.2us
I20260709 13:01:50.063257  5929 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:50.064599  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 100386us
I20260709 13:01:50.046603  6540 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:50.017637  6566 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:50.000141  6542 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:50.079752  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:50.080111  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.081538  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.083022  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 824) took 2206 ms. Trace:
I20260709 13:01:50.085404  2230 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:50.085629  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:50.083500  1616 rpcz_store.cc:276] 0709 13:01:47.878159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.656707 (+778548us) service_pool.cc:224] Handling call
0709 13:01:48.657840 (+  1133us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:48.665903 (+  8063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:48.665951 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:48.686292 (+ 20341us) log.cc:844] Serialized 52 byte log entry
0709 13:01:48.923845 (+237553us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:48.940945 (+ 17100us) log.cc:844] Serialized 14 byte log entry
0709 13:01:48.941337 (+   392us) log.cc:844] Serialized 14 byte log entry
0709 13:01:48.941559 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:48.941628 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:49.565251 (+623623us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:50.082755 (+517504us) raft_consensus.cc:1685] finished
0709 13:01:50.082781 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.082920 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2542,"prepare.queue_time_us":404019,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":23071,"raft.run_cpu_time_us":407,"raft.run_wall_time_us":17245,"spinlock_wait_cycles":7100288,"thread_start_us":180770,"threads_started":1,"wal-append.queue_time_us":269328}
I20260709 13:01:50.089447  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.090271  1542 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:50.093727  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.094646  1534 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:50.095264  1542 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:50.096182  1534 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:50.004653  6518 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 13:01:50.102329  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.102680  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.057760  6311 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:50.103659  2231 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 13:01:50.086092  6505 consensus_meta.cc:287] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.715s	user 0.012s	sys 0.000s
I20260709 13:01:50.124902  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.104544  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.105535  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.141754  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:50.151309  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.140558  1539 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:50.153932  2228 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:50.075083  6524 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:50.096560  6570 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:50.157051  2228 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:01:50.126989  2203 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:50.158105  2237 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:01:50.161983  6533 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.043679  6569 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.165642  6569 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:50.168871  6545 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:01:50.172933  1536 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:50.177681  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:50.064199  6337 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:50.122774  6340 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:01:50.102957  6322 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.071241  6574 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.096720  6543 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:50.122579  6540 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:50.155011  6122 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:50.159276  6311 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:50.164717  6533 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:50.226468  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 46679.8us
I20260709 13:01:50.227841  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.228844  1539 raft_consensus.cc:2410] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:01:50.170935  6550 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:50.151880  6505 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.232265  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.234899  1539 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:50.236315  1539 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:50.237453  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:50.238086  1536 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:50.187919  6317 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.239135  1584 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:50.245652  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.238026  6414 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.249505  6305 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:50.251151  6575 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:01:50.260185  1543 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:50.261837  6536 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:50.225664  6322 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.170323  6570 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:50.276494  1543 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:50.287101  1539 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:50.235253  6538 consensus_meta.cc:287] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.772s	user 0.013s	sys 0.000s
I20260709 13:01:50.163178  6524 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:50.313810  2236 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:01:50.314074  1542 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:01:50.213956  6556 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:01:50.171299  6405 consensus_queue.cc:1175] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cecad4f86a114a7696e398b3e8cc511e" responder_term: 12 status { last_received { term: 12 index: 10 } last_committed_idx: 9 last_received_current_leader { term: 12 index: 10 } }
I20260709 13:01:50.273212  6317 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.185598  6567 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.098s
I20260709 13:01:50.174224  6565 consensus_queue.cc:1059] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 13:01:50.097209  6557 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:50.231249  6550 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:50.230652  6122 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:50.305294  6544 log.cc:927] Time spent T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.886s	user 0.000s	sys 0.006s
I20260709 13:01:50.293260  6571 consensus_queue.cc:1059] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:01:50.321028  6505 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.327080  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:50.319092  1536 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
W20260709 13:01:50.313278  6529 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.255s	user 0.006s	sys 0.007s
I20260709 13:01:50.308697  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.364028  1535 raft_consensus.cc:2393] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:50.291400  6584 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:50.236212  6540 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:50.308331  6570 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:50.299960  6533 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.349612  6550 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:50.365593  2203 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:50.389195  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.389921  2228 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:50.391433  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:50.392004  2228 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:50.392287  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.392720  2232 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:50.373253  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:50.323525  6574 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.291038  6414 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:01:50.379778  6505 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.310488  6569 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:01:50.311707  6311 consensus_queue.cc:1059] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:01:50.382043  6317 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.382381  6122 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.291333  6536 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:01:50.365510  6545 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:50.423295  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.395529  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:50.406008  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.372975  6584 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:50.352970  6587 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
W20260709 13:01:50.384814  6388 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.000s	sys 0.013s
I20260709 13:01:50.414705  2228 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:50.393559  6533 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:50.404953  6476 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 13:01:50.328063  6556 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.352232  6543 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.409343  6577 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:01:50.395372  6394 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.390884  6570 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.424819  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:50.415338  6505 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:50.468290  6597 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:50.468888  6597 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:01:50.474035  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 49300.6us
I20260709 13:01:50.474850  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:50.431474  6388 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.000s	sys 0.013s
W20260709 13:01:50.482800  6589 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 13:01:50.484508  6586 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 13:01:50.484862  6593 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:50.477445  6577 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:50.485627  6577 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:50.469309  6597 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:50.485702  6593 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:01:50.486660  1685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 987) took 2022 ms (client timeout 1538 ms). Trace:
W20260709 13:01:50.487056  1685 rpcz_store.cc:269] 0709 13:01:48.464287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.465809 (+  1522us) service_pool.cc:224] Handling call
0709 13:01:50.486467 (+2020658us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54002ce054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:50.486596 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3590438144}
I20260709 13:01:50.487782  1537 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:50.426383  6567 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.425540  6122 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.429057  6584 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:50.493680  6604 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:01:50.431356  6533 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:50.444053  6538 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:50.426520  6596 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:50.469712  6317 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:50.477470  6506 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.479218  6599 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.506477  6577 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:50.507378  6322 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.509385  6505 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:50.509841  6514 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.513019  1536 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:50.514696  6556 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.516955  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.547030  1535 raft_consensus.cc:2393] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:50.428623  6340 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:01:50.490208  5929 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:50.541931  6559 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.514s	user 0.014s	sys 0.000s
I20260709 13:01:50.533646  6317 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:50.505528  6533 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:50.500473  6569 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:01:50.477690  6570 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:50.431888  6545 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.534377  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:50.541860  6335 consensus_meta.cc:287] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.992s	user 0.012s	sys 0.009s
I20260709 13:01:50.539152  6538 raft_consensus.cc:697] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:50.501097  6567 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.562076  6538 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.564086  6567 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.553575  5929 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.551913  6556 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:01:50.568568  6556 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:50.569157  6556 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:50.569906  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.571022  2228 raft_consensus.cc:2410] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:01:50.547144  6325 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:50.501392  6122 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.533659  6593 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:01:50.557511  6577 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:50.574163  5929 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.570291  6556 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.542914  6599 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.541203  6506 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:50.537356  6596 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:50.571868  6540 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:50.593735  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2341) took 4163 ms. Trace:
W20260709 13:01:50.590793  6342 consensus_meta.cc:287] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.539s	user 0.016s	sys 0.004s
I20260709 13:01:50.604693  1618 rpcz_store.cc:276] 0709 13:01:46.430167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:48.401621 (+1971454us) service_pool.cc:224] Handling call
0709 13:01:48.401944 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:48.402386 (+   442us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:01:48.622314 (+219928us) log.cc:844] Serialized 14 byte log entry
0709 13:01:50.170043 (+1547729us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b1800186160. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fc455 00007fe0934e43c7 00007fe08faf69a7 00007fe08fb060bf 00007fe08fb06038
0709 13:01:50.384976 (+214933us) log.cc:844] Serialized 14 byte log entry
0709 13:01:50.591852 (+206876us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:50.591927 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:50.592003 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.592095 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.593651 (+  1556us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25747,"prepare.queue_time_us":612107,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":2276,"raft.queue_time_us":24520,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":10597,"spinlock_wait_cycles":6565248,"thread_start_us":918969,"threads_started":4,"wal-append.queue_time_us":492174,"wal-append.run_cpu_time_us":1116,"wal-append.run_wall_time_us":1007784}
I20260709 13:01:50.587721  6325 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:50.595311  6514 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:50.626264  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 145702us
W20260709 13:01:50.613554  6542 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.003s
I20260709 13:01:50.612401  6556 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:50.624583  6591 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:01:50.627719  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:50.592420  6550 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.653158  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:50.612192  6342 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:50.641759  6592 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:50.611012  6317 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:50.668303  6592 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:50.668821  6592 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:50.674417  6506 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:50.676440  6550 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:50.678472  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:50.678870  6599 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.630545  6569 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:01:50.679324  1537 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:50.635093  6542 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:50.682451  6602 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:01:50.682891  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.683867  2230 raft_consensus.cc:2410] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:01:50.683766  6542 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:50.685181  6122 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.686517  6596 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:50.687476  6545 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:50.687805  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.689568  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.628814  6514 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:50.701138  1537 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:01:50.702809  1609 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 13:01:50.705631  2203 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:50.718122  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 60811.5us
I20260709 13:01:50.658710  6594 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:01:50.706696  6566 consensus_meta.cc:287] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.609s	user 0.012s	sys 0.000s
I20260709 13:01:50.658900  6075 consensus_queue.cc:1059] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:01:50.724296  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.750640  1535 raft_consensus.cc:2410] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:01:50.754837  6514 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:50.762547  6616 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:01:50.765316  6570 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:50.767076  6342 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:50.723038  6545 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:50.768361  6605 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:50.771718  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.776062  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:50.776455  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 57196.7us
I20260709 13:01:50.692504  6325 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.786267  6505 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.787777  6545 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:50.768916  6605 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.815418  6605 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:50.817730  6325 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:50.825469  6597 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.827744  6597 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.831059  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.840484  6571 raft_consensus.cc:3055] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:50.841007  6571 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:50.842113  6466 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:50.841842  6571 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.843101  6571 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:50.845185  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.837821  6533 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:50.848054  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.842571  6466 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:50.850984  1535 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:50.852526  6466 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.855219  6466 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:50.807962  6505 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.858736  6505 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:50.727212  6566 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:50.872537  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:50.877548  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:50.879561  2230 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:50.795054  6594 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:50.884563  2234 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:50.887914  6566 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:50.884436  6593 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.892570  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:50.894747  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 117673us
I20260709 13:01:50.906021  1536 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:50.906021  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:50.906798  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.907564  1535 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:50.909322  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:50.910645  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:50.911439  1611 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:50.913070  1536 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:01:50.915269  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.916509  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:50.916427  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:50.910203  1612 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:50.870797  6504 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.975287  1540 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:50.977799  1540 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:50.978788  1540 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:50.974792  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2334) took 1012 ms. Trace:
I20260709 13:01:50.893388  6335 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 13:01:50.885218  6626 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:50.982916  1538 rpcz_store.cc:276] 0709 13:01:49.962101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.966729 (+  4628us) service_pool.cc:224] Handling call
0709 13:01:49.967059 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:49.967734 (+   675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:49.967766 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:49.981700 (+ 13934us) log.cc:844] Serialized 26 byte log entry
0709 13:01:50.973735 (+992035us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b180043d860. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08f6944bc 00007fe08c9e248b 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1
0709 13:01:50.974160 (+   425us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:50.974212 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:50.974318 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:50.974384 (+    66us) raft_consensus.cc:1685] finished
0709 13:01:50.974400 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:50.974705 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":299,"raft.queue_time_us":22977,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":6721,"spinlock_wait_cycles":9780096,"thread_start_us":13082,"threads_started":1,"wal-append.queue_time_us":66232}
I20260709 13:01:51.026852  1538 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:01:50.901358  6596 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.032297  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:50.985137  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:51.033300  1617 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:50.966718  6506 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.966169  6311 raft_consensus.cc:3055] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:50.999265  6533 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.025431  6629 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:50.977485  6559 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.059695  1617 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:50.925172  6593 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:51.067365  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.044682  6311 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:50.948107  6545 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.061277  6122 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.071660  6629 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.044356  6596 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.126645  6596 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.129066  6629 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.082772  6075 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:51.130362  6075 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.054347  6594 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:51.050227  6506 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.087579  6593 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.093912  6631 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:51.094725  6577 consensus_meta.cc:287] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.000s
I20260709 13:01:51.096849  1583 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:51.148294  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 51521.2us
I20260709 13:01:51.098832  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:51.150166  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 51390.4us
W20260709 13:01:51.102573  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.102299  6342 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:51.103806  6556 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.084595  6311 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:51.104261  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:51.162744  1582 connection.cc:256] RPC call timeout handler was delayed by -1.3075904820000233s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1589s for a response
I20260709 13:01:51.162848  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:51.163753  2235 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:51.029459  6626 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:51.121390  6563 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.222s	user 0.005s	sys 0.000s
I20260709 13:01:51.121886  6335 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.122334  6524 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:51.123030  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1161) took 1315 ms. Trace:
I20260709 13:01:51.158769  2230 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:51.182513  1583 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:51.151634  1508 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:51.187217  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.113889  6625 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:51.175899  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:51.177891  1617 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:51.172729  6596 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:51.179365  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:51.132397  6122 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:51.179948  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:51.246402  2231 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:51.244491  6524 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:51.181890  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:51.182212  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:51.188067  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:51.183005  2233 rpcz_store.cc:276] 0709 13:01:49.807699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.808316 (+   617us) service_pool.cc:224] Handling call
0709 13:01:49.860947 (+ 52631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:49.886540 (+ 25593us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:49.886589 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:49.886650 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:49.886700 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.122765 (+1236065us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b54003bced4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:51.122820 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.122963 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26067,"raft.queue_time_us":21334,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":11887,"spinlock_wait_cycles":57785216}
I20260709 13:01:51.232462  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:51.233649  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:51.270736  6075 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.244792  6625 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.271570  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:51.273663  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:51.274879  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:51.275748  1688 raft_consensus.cc:2410] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 13:01:51.276525  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:51.235236  6193 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:51.277081  1688 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:51.236713  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:51.279579  1692 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:51.237592  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:51.280182  6122 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:51.238883  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 63088.7us
I20260709 13:01:51.281339  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:51.283099  6505 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:51.232441  6311 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:51.231631  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:51.178057  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:51.240511  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1842) took 1776 ms. Trace:
I20260709 13:01:51.171762  6557 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.243232  1690 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:51.290673  2230 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:51.179366  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:51.239573  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:51.292055  1683 raft_consensus.cc:2410] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:01:51.174983  6514 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:51.170420  1582 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:51.071067  6559 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:51.230870  6592 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.002s
I20260709 13:01:51.180373  2229 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:01:51.174782  6515 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:51.172540  6637 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:51.068351  6504 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:51.256991  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.277405  6193 raft_consensus.cc:697] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:51.278218  6584 consensus_meta.cc:287] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.758s	user 0.010s	sys 0.002s
I20260709 13:01:51.264410  6466 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.290306  1685 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:51.292243  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:51.190568  6577 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.295332  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:51.296482  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:51.290257  1689 rpcz_store.cc:276] 0709 13:01:49.464420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.470383 (+  5963us) service_pool.cc:224] Handling call
0709 13:01:49.470834 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:49.471281 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:49.471312 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:49.471871 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 13:01:49.765948 (+294077us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:49.766008 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:49.766108 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:51.120704 (+1354596us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:51.233238 (+112534us) raft_consensus.cc:1685] finished
0709 13:01:51.233264 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.240448 (+  7184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1268,"raft.queue_time_us":550,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":8266,"spinlock_wait_cycles":10088320,"thread_start_us":293597,"threads_started":1,"wal-append.queue_time_us":384475}
I20260709 13:01:51.273205  6625 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.236500  6550 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.238077  1657 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:51.235014  6340 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.307192  1543 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: [10.9-10.9]
W20260709 13:01:51.306576  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 209866us
I20260709 13:01:51.161173  6631 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:51.331229  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 160913us
I20260709 13:01:51.180600  6556 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.332219  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:51.236636  6325 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.335012  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:51.398533  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:51.330281  6132 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:01:51.339669  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 101698us
I20260709 13:01:51.267241  6626 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.291018  6122 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:51.291352  5885 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:51.294361  6075 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:51.346514  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 165082us
I20260709 13:01:51.334625  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:51.343851  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:51.300642  6103 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.100s
I20260709 13:01:51.301057  6637 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.294519  6593 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:51.417831  1657 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:51.401513  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:51.343919  1534 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:51.420326  1536 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:51.461148  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:51.382829  6340 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.294668  6557 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:51.422219  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:51.488332  1617 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:51.497205  1536 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:51.498415  1617 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:51.300652  6514 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:51.487053  1583 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:51.487355  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 189440us
W20260709 13:01:51.487246  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 85885.6us
I20260709 13:01:51.301816  6628 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.120s
I20260709 13:01:51.246419  6335 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.496393  1538 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:51.496680  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:51.501442  1617 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:01:51.501245  1615 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:51.505522  1617 pending_rounds.cc:85] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 10
I20260709 13:01:51.505981  1617 pending_rounds.cc:107] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 10.11
I20260709 13:01:51.506283  1617 raft_consensus.cc:2889] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:51.327193  6592 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:51.333446  6342 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:51.330799  6515 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.485785  6626 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:01:51.556120  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 69115.7us
W20260709 13:01:51.556778  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 56258.1us
I20260709 13:01:51.504182  6593 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.485720  6643 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.334110  6466 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:51.563795  1656 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.031s	sys 0.030s
W20260709 13:01:51.614734  1656 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.031s	sys 0.030s
I20260709 13:01:51.617535  1685 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
W20260709 13:01:51.617478  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.491859  6570 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:51.502189  6538 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:51.618783  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:51.619561  1690 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:51.620590  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:51.621388  1684 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:01:51.488250  6637 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:51.335610  6577 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.618837  1688 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:51.485499  6075 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.632257  1687 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:51.625067  2201 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:51.491797  6505 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:51.626511  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:51.384624  6550 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.610157   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:01:51.485303  6122 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:51.625788  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:51.583228  6626 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.580752  6596 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:51.640480  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:51.488574  6641 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.201s
I20260709 13:01:51.667552  1683 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
W20260709 13:01:51.659086  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.660786  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:51.424543  6325 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.333595  6584 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:51.668829  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:51.667172  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:51.664280  6505 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:51.682950  6505 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:51.685361  2235 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:51.657788  6570 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:51.559283  5885 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.696779  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:51.666790  6626 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:51.670619  6540 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.677235  6625 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:51.555684  6193 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:51.705780  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 288162us
I20260709 13:01:51.709244  1688 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:51.638985  6538 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.720769  1657 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:51.711216  6625 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:51.616665  6643 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:51.713346  6506 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.710008  6540 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:51.747402  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:51.718400  6559 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:51.750655  6625 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:51.745703  6641 raft_consensus.cc:3055] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:51.762317  6515 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.775571  1657 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:51.782948  6075 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.772403  6340 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.781299  6641 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:51.785166  6466 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:51.790293  6514 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:51.767145  6655 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:01:51.792364  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 167385us
W20260709 13:01:51.783485  6646 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
W20260709 13:01:51.796146  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 230260us
I20260709 13:01:51.781409  6631 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:51.802690  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 947) took 1679 ms. Trace:
I20260709 13:01:51.805728  2234 rpcz_store.cc:276] 0709 13:01:50.123303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.159333 (+ 36030us) service_pool.cc:224] Handling call
0709 13:01:50.159633 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.895129 (+735496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.802458 (+907329us) spinlock_profiling.cc:243] Waited 725 ms on lock 0x7b5400484ed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:51.802631 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1666660736}
I20260709 13:01:51.795290  6649 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:51.791249  6335 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:51.799111  6655 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.755514  6570 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:51.824107  6652 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 13:01:51.803334  6592 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:51.829727  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2374) took 1746 ms. Trace:
I20260709 13:01:51.832994  1610 rpcz_store.cc:276] 0709 13:01:50.083300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.087230 (+  3930us) service_pool.cc:224] Handling call
0709 13:01:50.087638 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.106487 (+ 18849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.106518 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:50.124865 (+ 18347us) log.cc:844] Serialized 26 byte log entry
0709 13:01:51.825719 (+1700854us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b180045cbe0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08f6944bc 00007fe08c9e248b 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1
0709 13:01:51.825913 (+   194us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:01:51.825955 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.826023 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:51.826068 (+    45us) raft_consensus.cc:1685] finished
0709 13:01:51.826081 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:51.829659 (+  3578us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18404,"raft.queue_time_us":68746,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":23668,"spinlock_wait_cycles":9708800,"thread_start_us":29845,"threads_started":1,"wal-append.queue_time_us":106783,"wal-append.run_cpu_time_us":1509,"wal-append.run_wall_time_us":1002381}
I20260709 13:01:51.836787  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:51.817828  6557 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:51.837357  1610 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:51.815549  6593 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.804620  6340 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:01:51.821990  6577 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:51.850131  6642 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:01:51.822626  6655 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:51.852145  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:51.823133  6325 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:51.855746  6644 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 13:01:51.856653  1610 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:51.883193  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:51.884056  1610 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:51.857585  6647 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 13:01:51.858074  6504 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.804735  6641 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:51.886096  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:51.887723  6540 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:51.887741  1610 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:51.826365  6122 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:51.826726  6556 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:51.888911  6657 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:01:51.862419  6593 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:51.889564  6593 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.890044  6593 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:51.858662  6658 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:51.888360  5885 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:51.843745  6557 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:51.911052  6557 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:51.886355  6662 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:51.912667  1618 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:51.912783  6662 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:51.913339  6662 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:51.840189  6592 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:01:51.893432  6648 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.166s	user 0.000s	sys 0.002s
I20260709 13:01:51.843200  6584 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:51.896020  6654 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:01:51.899052  6538 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:51.906785  6643 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:51.908664  6550 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:51.886113  6641 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:51.887727  6311 consensus_meta.cc:287] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.602s	user 0.007s	sys 0.004s
I20260709 13:01:51.823297  6649 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:51.922346  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:51.922708  6352 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:51.933984  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:51.934775  1614 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:51.922358  1610 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:51.932191  1684 raft_consensus.cc:2410] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:01:51.938441  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 945) took 1863 ms. Trace:
I20260709 13:01:51.938956  2236 rpcz_store.cc:276] 0709 13:01:50.075432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.144831 (+ 69399us) service_pool.cc:224] Handling call
0709 13:01:50.160098 (+ 15267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.314220 (+154122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.938164 (+1623944us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54006e78d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:51.938349 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346918272}
I20260709 13:01:51.929364  6592 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:01:51.996519  1537 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.010s	sys 0.016s
I20260709 13:01:51.913519  6658 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:51.995198  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.013787  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:51.990948  6644 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 13:01:51.997835  6648 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:01:51.950259  6584 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:51.990049  6643 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.016031  1618 raft_consensus.cc:2410] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:01:51.997211  6592 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:52.003949  6557 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:52.012460  6122 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.015700  6540 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:52.047252  6658 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.050158  6581 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:52.052637   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:01:52.072328  6630 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.093s	user 0.001s	sys 0.000s
I20260709 13:01:52.021353  6584 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:52.103515  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 328027us
I20260709 13:01:52.023088  6643 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.111137  1614 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
W20260709 13:01:52.097812  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 560263us
I20260709 13:01:52.058844  6122 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:52.118287  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.170492  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.171105  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.118697  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 320271us
I20260709 13:01:52.062572  6540 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:52.120154  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.050937  6581 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.102509  6393 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.090602  6592 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.180433  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:52.181289  2236 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:01:52.095204  6593 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.155678  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2375) took 2072 ms. Trace:
I20260709 13:01:52.186787  1613 rpcz_store.cc:276] 0709 13:01:50.083645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.083800 (+   155us) service_pool.cc:224] Handling call
0709 13:01:50.084170 (+   370us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:50.102748 (+ 18578us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:01:50.145028 (+ 42280us) log.cc:844] Serialized 14 byte log entry
0709 13:01:51.095362 (+950334us) spinlock_profiling.cc:243] Waited 5.89 ms on lock 0x7b18002852e0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a 00007fe08faf6a2a
0709 13:01:51.507216 (+411854us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:51.507970 (+   754us) log.cc:844] Serialized 78 byte log entry
0709 13:01:51.508163 (+   193us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:51.508212 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:51.508303 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:52.150963 (+642660us) raft_consensus.cc:1685] finished
0709 13:01:52.155360 (+  4397us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.155597 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1084,"prepare.queue_time_us":47895,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":109432,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":5821,"spinlock_wait_cycles":13558144,"thread_start_us":411567,"threads_started":2,"wal-append.queue_time_us":1256834}
I20260709 13:01:52.128284  6655 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.125166  6467 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:01:52.152585  6643 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.173288  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:52.173801  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:52.173906  2201 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:52.175189  1614 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:01:52.184566  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.226083  2234 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
W20260709 13:01:52.300096  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.192780  1613 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:01:52.194921  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:52.196269  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:52.301227  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:52.307668  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:52.308697  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.304975  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 131533us
I20260709 13:01:52.192806  6628 consensus_queue.cc:1059] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:01:52.175318  6649 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:52.156477  6626 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.309324  1507 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:52.299199  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 831) took 1131 ms. Trace:
W20260709 13:01:52.294279  6625 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.008s
I20260709 13:01:52.314716  6584 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.332000  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:52.312871  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 116729us
W20260709 13:01:52.288636  6596 consensus_meta.cc:287] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.618s	user 0.016s	sys 0.005s
W20260709 13:01:52.312592  6637 consensus_meta.cc:287] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.654s	user 0.022s	sys 0.000s
I20260709 13:01:52.315485  6423 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:52.344071  1536 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:52.335990  1617 rpcz_store.cc:276] 0709 13:01:51.167756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.504791 (+337035us) service_pool.cc:224] Handling call
0709 13:01:51.505225 (+   434us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:51.633809 (+128584us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.633849 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:51.634544 (+   695us) log.cc:844] Serialized 52 byte log entry
0709 13:01:52.098554 (+464010us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:52.190077 (+ 91523us) log.cc:844] Serialized 14 byte log entry
0709 13:01:52.195003 (+  4926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.195074 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:52.298911 (+103837us) raft_consensus.cc:1685] finished
0709 13:01:52.298946 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.299138 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50848,"prepare.queue_time_us":133328,"prepare.run_cpu_time_us":3241,"prepare.run_wall_time_us":42817,"raft.queue_time_us":157316,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":8245,"spinlock_wait_cycles":19826944,"thread_start_us":542874,"threads_started":3,"wal-append.queue_time_us":433122}
I20260709 13:01:52.298173  6592 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:52.293306  6572 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.291334  6467 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.296128  6643 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:52.292928  6593 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.352080  1690 raft_consensus.cc:2368] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14.
I20260709 13:01:52.311447  6504 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:52.314886  6671 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:52.315320  6538 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:01:52.398187  6540 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.398478  6504 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:52.375591  6325 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.375384  6637 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.414805  6325 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:52.398873  6668 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.113s	user 0.006s	sys 0.000s
W20260709 13:01:52.403666  6664 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.130s	user 0.001s	sys 0.003s
I20260709 13:01:52.422292  6325 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:52.355683  6596 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:52.423663  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 836) took 1254 ms. Trace:
I20260709 13:01:52.423960  6574 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.424569  6574 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:52.424041  1615 rpcz_store.cc:276] 0709 13:01:51.169229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.506374 (+337145us) service_pool.cc:224] Handling call
0709 13:01:51.506653 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:51.557410 (+ 50757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.557460 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:51.602293 (+ 44833us) log.cc:844] Serialized 26 byte log entry
0709 13:01:52.024597 (+422304us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:52.188556 (+163959us) log.cc:844] Serialized 14 byte log entry
0709 13:01:52.188806 (+   250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.188879 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:52.423453 (+234574us) raft_consensus.cc:1685] finished
0709 13:01:52.423474 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.423607 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50377,"prepare.queue_time_us":154653,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":45,"raft.queue_time_us":110774,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":39455,"spinlock_wait_cycles":32134528,"thread_start_us":584761,"threads_started":2,"wal-append.queue_time_us":520743}
W20260709 13:01:52.408741  6680 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.096s	user 0.000s	sys 0.004s
I20260709 13:01:52.426820  6574 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:52.427314  6540 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:52.427764  6678 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.417068  1539 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:52.375835  6628 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:52.348425  5885 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
W20260709 13:01:52.230437  6672 consensus_queue.cc:1175] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "83f6f5810950417195f707ff308a8125" responder_term: 12 status { last_received { term: 12 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 12 index: 11 } }
I20260709 13:01:52.348681  6625 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.400389  6423 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:52.419675  6504 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.488555  6504 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:52.384835  6643 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:52.382718  6467 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:52.478503  6122 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:52.493575  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.484586  6666 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:52.475636  6655 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:52.461548  6581 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.409679  6193 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:01:52.460884  6628 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.492151  1537 raft_consensus.cc:2410] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:01:52.385308  6572 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:52.494267  1689 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:52.495105  6643 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:52.502046  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:52.496327  6662 consensus_meta.cc:287] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.004s
I20260709 13:01:52.428341  6678 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.509464  1582 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:52.512827  1688 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:52.513264  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.516999  1689 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:01:52.506753  1536 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:01:52.519620  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:52.520826  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.534040  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:52.535859  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.536535  1688 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:52.536721  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.537454  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.538501  1688 raft_consensus.cc:2410] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:01:52.494477  6423 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.513176  6581 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:52.516943  6643 raft_consensus.cc:3055] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:52.541793  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.544374  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.540724  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:52.548578  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:52.517128  6666 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:52.522332  6574 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:52.555505  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.561103  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.561722  1689 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:52.561838  1683 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:52.563071  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.563567  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:52.559130  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:52.565711  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.566280  1688 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
W20260709 13:01:52.567174  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.563099  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.572685  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.573542  1688 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:52.554955  6643 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:52.574446  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 919) took 2480 ms. Trace:
I20260709 13:01:52.560424  6596 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:52.579159  1582 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:52.578723  1542 rpcz_store.cc:276] 0709 13:01:50.101852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.139769 (+ 37917us) service_pool.cc:224] Handling call
0709 13:01:50.140385 (+   616us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.315755 (+175370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.571375 (+2255620us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54000e92d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:52.574359 (+  2984us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398934912}
I20260709 13:01:52.554193  6626 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.580314  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.556102  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:52.583355  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.584215  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.517673  6662 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.560906  6581 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.586107  1689 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:01:52.587610  1542 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:01:52.583906  6641 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.652s	user 0.013s	sys 0.001s
W20260709 13:01:52.597867  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.597666  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.585273  1688 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:01:52.598435  1689 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:01:52.599803  1683 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:52.600317  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.600936  1688 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:52.580751  6574 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:52.594012  1582 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:52.600323  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.606238  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.617754  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.617964  1688 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:52.616432  1582 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:52.620036  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:52.621775  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.622536  1688 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:52.622162  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.597574  6677 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:52.627512  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:52.633504  6535 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:52.633950  6535 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.634275  6535 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:52.634469  6684 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 13:01:52.635846  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1875) took 1070 ms. Trace:
I20260709 13:01:52.636197  1610 rpcz_store.cc:276] 0709 13:01:51.565589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.939521 (+373932us) service_pool.cc:224] Handling call
0709 13:01:51.997285 (+ 57764us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:52.143608 (+146323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:52.143646 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:52.145318 (+  1672us) log.cc:844] Serialized 26 byte log entry
0709 13:01:52.290280 (+144962us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:52.290351 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.290523 (+   172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:52.635624 (+345101us) raft_consensus.cc:1685] finished
0709 13:01:52.635646 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.635786 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77800,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":273218,"threads_started":2,"wal-append.queue_time_us":346462}
I20260709 13:01:52.647758  6572 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:52.650157  6628 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.655102  6662 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.621439  6596 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:52.623903  6681 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:52.649102  6678 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.624651  6625 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.632810  6574 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:52.599603  6643 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.665880  6681 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:52.676316  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:52.677241  2234 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:52.619374  6666 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.677757  6423 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:52.676756  6681 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:52.691190  6687 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:01:52.692404  6671 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:52.697997  6581 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.701218  6504 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.703293  6681 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.679306  6535 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.704820  6504 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.706521  6681 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.753775  1436 catalog_manager.cc:5697] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:52.760632  1436 catalog_manager.cc:5697] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 8 to 9, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 9 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:52.764537  1436 catalog_manager.cc:5697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 5 to 10, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 10 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:01:52.767298  1440 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40172 (request call id 232) took 18114 ms (client timeout 15000 ms). Trace:
W20260709 13:01:52.767867  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 458382us
W20260709 13:01:52.769120  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 424112us
W20260709 13:01:52.769131  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 427359us
W20260709 13:01:52.773581  1534 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2353) took 1857 ms (client timeout 2219 ms). Trace:
I20260709 13:01:52.775548  1536 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:52.777173  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2361) took 1447 ms. Trace:
I20260709 13:01:52.784720  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:52.786325  1539 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2362) took 1453 ms (client timeout 1916 ms). Trace:
W20260709 13:01:52.786942  1539 rpcz_store.cc:269] 0709 13:01:51.332978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.334367 (+  1389us) service_pool.cc:224] Handling call
0709 13:01:52.785691 (+1451324us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b540012ded4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:52.786080 (+   389us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2132330752}
I20260709 13:01:52.790903  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2373) took 2901 ms. Trace:
I20260709 13:01:52.791569  1609 rpcz_store.cc:276] 0709 13:01:49.889282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.890822 (+  1540us) service_pool.cc:224] Handling call
0709 13:01:49.891153 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:50.704243 (+813090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.790042 (+2085799us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b54002bd254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:52.790602 (+   560us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1865685376}
W20260709 13:01:52.802278  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 174750us
W20260709 13:01:52.806906  1684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 1011) took 1183 ms (client timeout 1545 ms). Trace:
W20260709 13:01:52.807665  1684 rpcz_store.cc:269] 0709 13:01:51.622955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.625563 (+  2608us) service_pool.cc:224] Handling call
0709 13:01:52.806274 (+1180711us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b540014a354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:52.806603 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626113920}
I20260709 13:01:52.810076  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2742) took 1009 ms. Trace:
I20260709 13:01:52.810581  1683 rpcz_store.cc:276] 0709 13:01:51.801204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.803816 (+1002612us) service_pool.cc:224] Handling call
0709 13:01:52.804646 (+   830us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.805356 (+   710us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:52.805390 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:52.809511 (+  4121us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:52.809588 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.809696 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.809875 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:52.811504  1685 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:52.812259  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2744) took 1010 ms. Trace:
I20260709 13:01:52.812427  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.812630  1685 rpcz_store.cc:276] 0709 13:01:51.801787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.810503 (+1008716us) service_pool.cc:224] Handling call
0709 13:01:52.811019 (+   516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.812013 (+   994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.812190 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:01:52.813695  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:52.814641  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:52.815037  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.816283  1686 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:52.819568  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.819644  1688 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:01:52.818637  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:52.852838  1687 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1858) took 1670 ms (client timeout 1713 ms). Trace:
W20260709 13:01:52.860528  2235 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1341) took 1700 ms (client timeout 2219 ms). Trace:
W20260709 13:01:52.860324  1687 rpcz_store.cc:269] 0709 13:01:51.181758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.232237 (+ 50479us) service_pool.cc:224] Handling call
0709 13:01:52.851888 (+1619651us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b54000f5354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:52.852170 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":797752704}
I20260709 13:01:52.861507  1688 pending_rounds.cc:85] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 8
I20260709 13:01:52.861975  1688 pending_rounds.cc:107] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:01:52.862509  1688 raft_consensus.cc:2889] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:52.783025  1535 rpcz_store.cc:276] 0709 13:01:51.329847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.329997 (+   150us) service_pool.cc:224] Handling call
0709 13:01:51.330323 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:51.490105 (+159782us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:51.490150 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:51.490206 (+    56us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:51.490249 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.776545 (+1286296us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54000eddd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:52.776650 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.776952 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366873344}
I20260709 13:01:52.863325  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:52.821836  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.864790  2231 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:52.864966  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1016) took 1240 ms. Trace:
I20260709 13:01:52.827108  2233 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:52.840088  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 1012) took 1216 ms. Trace:
I20260709 13:01:52.866065  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:52.866798  1685 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:01:52.845732  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:52.867298  1685 pending_rounds.cc:85] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 6
I20260709 13:01:52.867238  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:52.867928  1685 pending_rounds.cc:107] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:01:52.820170  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:52.868813  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:52.869496  1686 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:52.870702  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:52.782471  1534 rpcz_store.cc:269] 0709 13:01:50.915642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.916247 (+   605us) service_pool.cc:224] Handling call
0709 13:01:52.772821 (+1856574us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b54000ec4d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:52.773167 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981005568}
I20260709 13:01:52.843014  1540 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:52.852614  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 952) took 1667 ms. Trace:
I20260709 13:01:52.830914  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.873957  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:52.874536  1692 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:01:52.874063  2229 rpcz_store.cc:276] 0709 13:01:51.185393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.423652 (+238259us) service_pool.cc:224] Handling call
0709 13:01:51.429601 (+  5949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:51.716680 (+287079us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:01:51.723342 (+  6662us) log.cc:844] Serialized 14 byte log entry
0709 13:01:52.133438 (+410096us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:52.133497 (+    59us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:01:52.133539 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.851951 (+718412us) spinlock_profiling.cc:243] Waited 287 ms on lock 0x7b54005c0dd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:52.852042 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.852391 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7998,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":81,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":10713,"spinlock_wait_cycles":662724736,"thread_start_us":409653,"threads_started":1,"wal-append.queue_time_us":454086}
I20260709 13:01:52.876035  1692 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:52.876632  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2749) took 1068 ms. Trace:
I20260709 13:01:52.876845  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:52.826610  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1169) took 2439 ms. Trace:
I20260709 13:01:52.878275  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.878690  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:52.868305  2233 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
W20260709 13:01:52.781663  1440 rpcz_store.cc:269] 0709 13:01:34.664192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:34.704068 (+ 39876us) service_pool.cc:224] Handling call
0709 13:01:35.387954 (+683886us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:01:52.766917 (+17378963us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:44.447923 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:44.458585 (+ 10662us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:44.458640 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:01:44.458661 (+    21us) tablet.cc:662] Decoding operations
0709 13:01:44.461503 (+  2842us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:44.461623 (+   120us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:44.461654 (+    31us) tablet.cc:685] Acquiring locks for 20 operations
0709 13:01:44.463199 (+  1545us) tablet.cc:701] Row locks acquired
0709 13:01:44.463245 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:01:44.463446 (+   201us) write_op.cc:270] Start()
0709 13:01:44.463597 (+   151us) write_op.cc:276] Timestamp: P: 1783602104463371 usec, L: 0
0709 13:01:44.463628 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 13:01:44.464378 (+   750us) log.cc:844] Serialized 7936 byte log entry
0709 13:01:45.395715 (+931337us) op_driver.cc:464] REPLICATION: finished
0709 13:01:45.632742 (+237027us) write_op.cc:301] APPLY: starting
0709 13:01:45.639432 (+  6690us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:46.158354 (+518922us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:46.158503 (+   149us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:46.406356 (+247853us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:46.406789 (+   433us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 13:01:46.406851 (+    62us) write_op.cc:312] APPLY: finished
0709 13:01:46.407350 (+   499us) log.cc:844] Serialized 180 byte log entry
0709 13:01:46.511187 (+103837us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:46.552769 (+ 41582us) write_op.cc:454] Released schema lock
0709 13:01:46.581983 (+ 29214us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:46.664571 (+ 82588us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:46.669264 (+  4693us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":975437,"reported_tablets":39,"spinlock_wait_cycles":596281728,"child_traces":[["op",{"apply.queue_time_us":193717,"apply.run_cpu_time_us":43807,"apply.run_wall_time_us":998440,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87847,"num_ops":20,"prepare.queue_time_us":108835,"prepare.run_cpu_time_us":39261,"prepare.run_wall_time_us":255488,"raft.queue_time_us":200931,"raft.run_cpu_time_us":2459,"raft.run_wall_time_us":64276,"replication_time_us":931905,"spinlock_wait_cycles":123008,"thread_start_us":409056,"threads_started":5,"wal-append.queue_time_us":436597,"wal-append.run_cpu_time_us":16265,"wal-append.run_wall_time_us":2710187}]]}
W20260709 13:01:52.819355  1618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1880) took 1252 ms (client timeout 1534 ms). Trace:
I20260709 13:01:52.839236  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:52.866371  1690 rpcz_store.cc:276] 0709 13:01:51.623256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.626305 (+  3049us) service_pool.cc:224] Handling call
0709 13:01:52.839578 (+1213273us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54002cd154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:52.839843 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1642064640}
I20260709 13:01:52.872406  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:52.854106  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:52.876907  1692 rpcz_store.cc:276] 0709 13:01:51.808743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.875363 (+1066620us) service_pool.cc:224] Handling call
0709 13:01:52.875763 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.876408 (+   645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.876581 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:52.878028  2237 rpcz_store.cc:276] 0709 13:01:50.387196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:50.387322 (+   126us) service_pool.cc:224] Handling call
0709 13:01:50.387623 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.590850 (+203227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:50.590895 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:50.590958 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:50.591013 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.826015 (+2235002us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b5400472a54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:52.826117 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:52.826384 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":466823040}
I20260709 13:01:52.868305  1685 raft_consensus.cc:2889] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:52.865696  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:52.861263  2235 rpcz_store.cc:269] 0709 13:01:51.159524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.172793 (+ 13269us) service_pool.cc:224] Handling call
0709 13:01:52.844584 (+1671791us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54007404d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:52.860266 (+ 15682us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937185024}
I20260709 13:01:52.867532  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:52.880801  1618 rpcz_store.cc:269] 0709 13:01:51.567083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.851905 (+284822us) service_pool.cc:224] Handling call
0709 13:01:52.818782 (+966877us) spinlock_profiling.cc:243] Waited 59.3 ms on lock 0x7b5400322654. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:52.819102 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136409472}
I20260709 13:01:52.917503  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:52.894774  1688 rpcz_store.cc:276] 0709 13:01:51.624214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.625211 (+1000997us) service_pool.cc:224] Handling call
0709 13:01:52.818693 (+193482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.863551 (+ 44858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.864680 (+  1129us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b540014a350. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:52.864862 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443870080}
I20260709 13:01:52.909010  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2748) took 1100 ms. Trace:
I20260709 13:01:52.934192  1686 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:52.915096  6625 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.060073  6625 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.057976  1685 rpcz_store.cc:276] 0709 13:01:51.808202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.849543 (+1041341us) service_pool.cc:224] Handling call
0709 13:01:52.850143 (+   600us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.908636 (+ 58493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:52.908913 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.054602  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:52.921808  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 103626us
W20260709 13:01:53.033557  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 179712us
I20260709 13:01:52.774011  1436 catalog_manager.cc:5697] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 6 to 7, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:01:52.925117  2230 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:53.020967  1684 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:52.909765  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:53.067907  1688 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:01:53.072438  2237 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:53.076249  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.064885  2236 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:53.053507  6671 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:52.922544  6681 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:53.076336  2233 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:01:52.883705  6535 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:52.912169  6581 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:52.915693  6574 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:53.090670  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:53.091065  6423 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.095475  6467 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:53.104636  1690 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:52.929633  6132 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:53.096453  1540 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:01:53.064492  6695 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:53.107368  1687 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:53.105060   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:53.105316  1436 catalog_manager.cc:5697] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 4 to 9, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 9 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:53.111218  1436 catalog_manager.cc:5697] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 1 to 8, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 8 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:01:53.084611  6671 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.085973  6681 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.114339  6671 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:53.114560  6681 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.116667  6695 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.117905  6132 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:53.120374  6695 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:53.122058  1534 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:53.121554  6423 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:52.919654  6662 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:52.926288  6570 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:52.917790  6504 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:53.107079  1540 pending_rounds.cc:85] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 9
I20260709 13:01:52.916082  6596 raft_consensus.cc:2764] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 13:01:53.132135  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2761) took 1206 ms. Trace:
I20260709 13:01:53.138370  1690 rpcz_store.cc:276] 0709 13:01:51.926010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.894700 (+968690us) service_pool.cc:224] Handling call
0709 13:01:53.071231 (+176531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.131001 (+ 59770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.132068 (+  1067us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.140753  1540 pending_rounds.cc:107] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:01:53.141229  1540 raft_consensus.cc:2889] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:53.103639  6581 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:53.103317  6574 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:53.151405  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 302570us
I20260709 13:01:53.052233  6696 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 13:01:52.921972  6666 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.227s
W20260709 13:01:53.157081  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 75949.2us
I20260709 13:01:53.169997  6662 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:53.174470  1657 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:53.176013  6467 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:53.149065  6581 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:01:53.182034  6512 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:01:53.156335  6535 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:53.186708  6535 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:53.173334  6662 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:53.188305  1436 catalog_manager.cc:5697] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 8 to 9, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 9 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:01:53.181866  6625 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:53.194110  6535 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:53.152412  6574 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:01:53.206666  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.179793  1508 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:53.217968  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2861) took 3822 ms. Trace:
I20260709 13:01:53.219805  2237 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:01:53.218431  1540 rpcz_store.cc:276] 0709 13:01:49.393680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:49.400382 (+  6702us) service_pool.cc:224] Handling call
0709 13:01:49.412403 (+ 12021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:50.975183 (+1562780us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54004644a0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa63dc0 00007fe08faee1c0 00007fe08faef231 00007fe08faebd8f 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c
0709 13:01:52.841574 (+1866391us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b58008873d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa480aa 00007fe08faca447 00007fe08fae2fe7 00007fe08fae47de 00007fe08faeeb49 00007fe08faef416 00007fe08faebd8f 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4
0709 13:01:53.178925 (+337351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.215827 (+ 36902us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384794368}
W20260709 13:01:53.219905  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 51235us
I20260709 13:01:53.256924  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:53.258647  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 79124.1us
I20260709 13:01:53.265300  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:53.266480  1535 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:53.267393  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.267617  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.268375  1543 raft_consensus.cc:2410] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:01:53.268643  1536 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
W20260709 13:01:53.270380  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.259713  1657 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:53.286185  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:53.288038  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 64157.7us
I20260709 13:01:53.208910  1436 catalog_manager.cc:5697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 6 to 11, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 11 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:53.291734  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.294956  6597 raft_consensus.cc:3055] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:53.295629  6597 raft_consensus.cc:740] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:53.300573  1583 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:53.301003  6611 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 2.641s	user 0.002s	sys 0.002s
W20260709 13:01:53.310684  6618 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 2.640s	user 0.008s	sys 0.005s
I20260709 13:01:53.233075  6504 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:53.314568  6624 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 2.315s	user 0.003s	sys 0.000s
I20260709 13:01:53.291915  6607 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:53.304895  6557 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:01:53.344843  2233 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:01:53.320155  6597 consensus_queue.cc:260] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:53.292593  1436 catalog_manager.cc:5697] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 9 to 10, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 10 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:53.344327  6695 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:53.313320  6132 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.359980  1540 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:01:53.340829  6570 raft_consensus.cc:2764] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:01:53.357463  6695 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.353896  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2763) took 1411 ms. Trace:
I20260709 13:01:53.354645  6696 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:53.366853  2201 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.545s (SENT) [suppressed 11 similar messages]
I20260709 13:01:53.362957  6712 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.372165  6597 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:53.377744  6695 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:53.372222  1683 rpcz_store.cc:276] 0709 13:01:51.931037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.910005 (+978968us) service_pool.cc:224] Handling call
0709 13:01:52.930213 (+ 20208us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:52.930552 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:01:53.179483 (+248931us) log.cc:844] Serialized 14 byte log entry
0709 13:01:53.341021 (+161538us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:53.341128 (+   107us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:53.341195 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.341314 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.342204 (+   890us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":150,"prepare.queue_time_us":357835,"prepare.run_wall_time_us":42,"raft.queue_time_us":5692,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":16881,"spinlock_wait_cycles":5892864,"thread_start_us":404676,"threads_started":2}
W20260709 13:01:53.392030  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 104910us
W20260709 13:01:53.398640  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.399094  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:53.402530  6627 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.403481  6637 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:53.403805  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.403313  6627 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.404649  6637 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.405138  6666 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:53.405838  6666 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:53.406286  6666 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:53.406949  6627 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:53.410576  6700 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:01:53.412622  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1017) took 1788 ms. Trace:
W20260709 13:01:53.413082  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 142805us
I20260709 13:01:53.410386  6653 raft_consensus.cc:3055] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:01:53.422047  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:53.426797  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 164140us
W20260709 13:01:53.433414  2201 leader_election.cc:336] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.857s (SENT)
I20260709 13:01:53.433959  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:53.433007  1436 catalog_manager.cc:5697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 11 to 12, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 12 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:53.438145  1436 catalog_manager.cc:5697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:01:53.413121  1689 rpcz_store.cc:276] 0709 13:01:51.624442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.626435 (+1001993us) service_pool.cc:224] Handling call
0709 13:01:52.627446 (+  1011us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:52.627952 (+   506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:52.627989 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:52.657182 (+ 29193us) log.cc:844] Serialized 26 byte log entry
0709 13:01:53.079827 (+422645us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:53.079901 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.080005 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:53.412091 (+332086us) raft_consensus.cc:1685] finished
0709 13:01:53.412140 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:53.412402 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":824,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":17405,"thread_start_us":404883,"threads_started":1,"wal-append.queue_time_us":592627}
I20260709 13:01:53.467947  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:53.441115  6696 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:53.480428  1612 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:53.417088  6653 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:53.480962  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 53797.3us
W20260709 13:01:53.489269  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 87972.2us
W20260709 13:01:53.490195  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 91220.9us
I20260709 13:01:53.405288  6637 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.504997  6696 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:53.508105  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:53.509541  1618 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:53.509096  6681 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.511422  6653 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.512845  6653 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:53.441380  6712 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:53.517510  6581 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:53.520550  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.467273  1436 catalog_manager.cc:5697] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 8 to 9, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 9 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:01:53.531303  6504 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:53.533766  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.534557  1534 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:01:53.540161  1537 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:01:53.505586  6545 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.127s	user 0.009s	sys 0.009s
I20260709 13:01:53.555846  6605 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:53.556293  1436 catalog_manager.cc:5697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 6 to 7, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 7 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:01:53.558079  1436 catalog_manager.cc:5697] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 5 to 10, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 10 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
W20260709 13:01:53.562403  6714 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:01:53.572350  6504 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:53.573599  6504 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.580365  6423 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:53.549741  6545 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.313s	user 0.010s	sys 0.010s
I20260709 13:01:53.622929  6581 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:53.629839  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:53.634969  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:53.635604  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 139106us
I20260709 13:01:53.636360  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.637634  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:53.638842  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:53.639055  6666 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:53.641399  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.639686  1616 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:53.646956  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:53.671241  1543 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:53.648607  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.651204  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.655572  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.656275  2232 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:01:53.656723  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:53.658658  6574 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:53.661348  6337 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:53.663394  6514 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.668067  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:53.672459  1540 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:53.682648  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:53.684120  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:53.688768  1536 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:53.688953  1616 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:53.690549  6504 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:53.660777  6695 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.686810  1618 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
W20260709 13:01:53.697242  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 212256us
I20260709 13:01:53.701920  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.704154  1540 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:53.705868  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:53.706161  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.711802  6514 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.718606  6514 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:53.705335  1612 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 9.
I20260709 13:01:53.724655  1616 pending_rounds.cc:85] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 6
I20260709 13:01:53.725091  1616 pending_rounds.cc:107] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:01:53.725159  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:53.725410  1616 raft_consensus.cc:2889] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:01:53.727099  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 293728us
W20260709 13:01:53.735117  6634 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 2.749s	user 0.003s	sys 0.000s
W20260709 13:01:53.737457  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.739084  2201 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:53.740164  6726 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:53.740592  6726 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.740929  6726 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.745602  1537 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:53.753125  2201 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:53.754536  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:53.755232  6676 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.525s	user 0.002s	sys 0.000s
I20260709 13:01:53.755582  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:53.757119  6545 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:53.757525  6545 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.757864  6545 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:53.759039  6132 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:53.762110  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 891992us
I20260709 13:01:53.763069  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:53.764030  6388 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:53.765591  2201 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:53.766603  6305 raft_consensus.cc:993] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:01:53.767112  6388 raft_consensus.cc:1081] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:01:53.767320  6423 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:53.767768  6423 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.768141  6423 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:53.769961  6605 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:53.770267  6605 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:53.770483  6605 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:53.750241  6324 raft_consensus.cc:3055] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:53.774111  6324 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:53.751811  6726 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.775553  1618 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
W20260709 13:01:53.714962  6671 consensus_meta.cc:287] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.005s
I20260709 13:01:53.775453  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:53.783092  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:53.788726  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 117006us
I20260709 13:01:53.785741  6205 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:53.790606  6205 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:53.791031  6205 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:53.792238  6554 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 13:01:53.794283  6663 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:53.794741  6663 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.795079  6663 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:53.796995  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.781549  6324 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.799610  1618 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:01:53.799691  6324 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:53.810635  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:53.812968  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.816062  1618 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:53.782457  6671 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.818851  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:53.826372  6685 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.246s	user 0.003s	sys 0.000s
W20260709 13:01:53.827334  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.827754  6726 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:53.826256  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:53.829895  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:53.834159  1609 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:53.834339  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:53.835335  1610 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:53.818750  6317 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:01:53.838327  6725 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:01:53.840703  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 75451.9us
I20260709 13:01:53.814177  6637 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:53.819097  6666 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:53.861243  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.862250  1534 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
W20260709 13:01:53.862721  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.851136  6574 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:53.888331  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:53.897866  6636 consensus_queue.cc:1059] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:01:53.850162  6677 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.910126  6311 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:53.915457  6643 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:53.916913  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.918754  1539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:54758
I20260709 13:01:53.919167  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.920161  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:53.921463  6720 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 13:01:53.922020  6695 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:53.923144  6388 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:53.924083  1584 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:53.820966  6653 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:01:53.925150  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:53.922116  1534 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:53.932592  1611 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:53.932994  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 50722.7us
W20260709 13:01:53.935820  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 166371us
I20260709 13:01:53.939209  1539 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:53.944404  1539 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:53.937813  1610 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:53.945860  1686 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:01:53.946940  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 79658.7us
I20260709 13:01:53.947791  1538 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 10.
I20260709 13:01:53.931913  6666 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:53.962760  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:53.952394  1543 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:53.950743  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1862) took 1140 ms. Trace:
I20260709 13:01:53.953238  1584 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:53.979552  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.980823  6514 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:53.981087  1584 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:53.981983  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2760) took 2066 ms. Trace:
I20260709 13:01:53.981909  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.982951  1535 raft_consensus.cc:2368] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15.
I20260709 13:01:53.978049  1686 rpcz_store.cc:276] 0709 13:01:52.807703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.913053 (+1105350us) service_pool.cc:224] Handling call
0709 13:01:53.937318 (+ 24265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.946264 (+  8946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.947987 (+  1723us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:53.982833  6596 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:53.982563  1687 rpcz_store.cc:276] 0709 13:01:51.915577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.879859 (+964282us) service_pool.cc:224] Handling call
0709 13:01:53.071565 (+191706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:53.107839 (+ 36274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.981757 (+873918us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54001dd3d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:53.981906 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":384682240}
I20260709 13:01:53.991813  1541 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:53.992102  6639 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:53.981977  6672 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:53.994457  6672 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.941704  6311 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:53.994499  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:53.994486  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:54.001891  6672 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:54.001761  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:54.008695  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:53.950109  6574 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:53.939332  6677 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:53.953346  6305 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:53.951027  6476 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:01:54.010999  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.009546  1539 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:54.012647  1538 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
W20260709 13:01:53.998275  2203 connection.cc:256] RPC call timeout handler was delayed by -1.0925330199999621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
I20260709 13:01:54.014564  1687 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:01:53.991755  6610 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 3.326s	user 0.002s	sys 0.004s
I20260709 13:01:54.016420  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:54.020341  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.021238  1686 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:01:54.013736  1688 raft_consensus.cc:2376] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13. Re-sending same reply.
I20260709 13:01:54.023396  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:53.953536  6132 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 13:01:53.984052  6663 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.010607  6574 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:53.990489  6666 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.029048  1539 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:54.029039  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:54.034384  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:54.016076  6476 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:54.041088  2233 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:01:54.041692  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.047032  1685 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:54.041877  6311 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:54.042348  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.043610  6566 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.036072  1584 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:54.035030  1687 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:54.062369  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1338) took 2903 ms. Trace:
I20260709 13:01:54.062659  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.062500  6566 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.062865  2232 rpcz_store.cc:276] 0709 13:01:51.158706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.158827 (+   121us) service_pool.cc:224] Handling call
0709 13:01:51.180121 (+ 21294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:53.678527 (+2498406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.062072 (+383545us) spinlock_profiling.cc:243] Waited 2.42 s on lock 0x7b54002f49d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:54.062301 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5565547264}
I20260709 13:01:54.064724  6566 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:54.050943  6666 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:54.051438  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:54.053815  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.069762  1691 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:01:54.070653  6352 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.071143  6352 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:54.071645  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:54.072249  5885 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.072657  6515 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.057094  6681 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:54.059027  6712 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:54.031973  6574 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.074352  1684 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:54.030599  6388 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:54.035527  6663 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:54.072820  5885 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.077631  1687 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:54.079514  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:54.080569  2237 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:54.078835  6545 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.101367  6545 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:54.073199  6515 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:54.079353  6639 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:54.102917  6639 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.103322  6639 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:54.073197  6476 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:54.105865  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.051469  6205 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.093794  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:54.109308  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:54.110379  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 111903us
I20260709 13:01:54.079496  6653 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:54.116703  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:54.117698  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:54.118469  2230 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:01:54.123035  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 48943us
W20260709 13:01:54.124572  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.125692  6672 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:54.126220  6672 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.126605  6672 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:54.129655  6728 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.130326  6728 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.130802  6205 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:54.131373  6545 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:54.122959  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.136545  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:54.136577  2236 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:54.142884  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:54.147061  6732 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:01:54.147431  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:54.150257  1534 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:54.152544  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:54.113636  6566 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:54.117739  6653 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.156404  6566 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.156853  6305 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:54.158756  6566 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:54.159673  2201 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:54.160853  6731 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:54.160634  6709 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:54.161518  6731 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:54.162137  1543 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:54.162501  6731 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.163272  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:54.163892  1543 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:54.166292  6605 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.167438  2235 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:54.167819  1689 pending_rounds.cc:85] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 9
I20260709 13:01:54.168156  1689 pending_rounds.cc:107] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:01:54.168414  1689 raft_consensus.cc:2889] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:54.168565  6605 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:54.169795  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:54.168282  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:54.170946  1686 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:54.171094  2228 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.172410  1539 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
W20260709 13:01:54.175863  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 139869us
I20260709 13:01:54.174808  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:54.184844  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.186734  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.145177  6712 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:54.183197  1537 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:54.192023  2230 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:54.179251  1541 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:54.195369  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:54.195881  1692 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:01:54.196398  1692 pending_rounds.cc:85] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 11
I20260709 13:01:54.196738  1692 pending_rounds.cc:107] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:01:54.197165  1692 raft_consensus.cc:2889] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:54.181638  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:54.170002  6653 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.210664  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.211468  2232 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:01:54.212481  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.169849  6677 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:54.208952  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:54.214391  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:54.229796  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.230974  2230 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:54.217002  2237 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:54.218657  6605 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:01:54.218798  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:54.219108  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:54.220379  6305 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:54.186827  6666 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:54.224673  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:54.241096  1688 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:54.178416  6514 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:54.244850  1692 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.246503  1611 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:01:54.246587  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:54.246783  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:54.247495  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:54.178618  6681 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:54.247030  1611 pending_rounds.cc:85] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 5
I20260709 13:01:54.240175  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:54.189960  6635 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:01:54.247330  1609 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:54.425067  1612 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
W20260709 13:01:54.351605  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.272667  1616 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:54.369194  1690 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:54.377009  2232 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:54.444054  2228 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
W20260709 13:01:54.425892  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 272839us
I20260709 13:01:54.247596  1613 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:54.447220  1611 pending_rounds.cc:107] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:01:54.337797  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:54.453044  6739 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
I20260709 13:01:54.453122  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:54.454429  2237 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:54.456218  1618 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
W20260709 13:01:54.456499  6704 log.cc:927] Time spent T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.204s	user 0.004s	sys 0.001s
I20260709 13:01:54.275090  6542 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:54.448699  6637 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.457926  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1177) took 1670 ms. Trace:
I20260709 13:01:54.458128  6637 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.459203  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:54.458312  2234 rpcz_store.cc:276] 0709 13:01:52.787774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.825620 (+ 37846us) service_pool.cc:224] Handling call
0709 13:01:52.826457 (+   837us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:52.827300 (+   843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:52.827333 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:52.828301 (+   968us) log.cc:844] Serialized 26 byte log entry
0709 13:01:53.050837 (+222536us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:53.050904 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:53.051102 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:54.455867 (+1404765us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:54.457528 (+  1661us) raft_consensus.cc:1685] finished
0709 13:01:54.457552 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:54.457745 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8563,"raft.queue_time_us":82016,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":2506,"spinlock_wait_cycles":2705280,"thread_start_us":167301,"threads_started":1,"wal-append.queue_time_us":317543}
I20260709 13:01:54.459817  1690 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:54.460305  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.460647  6637 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:54.461619  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:54.461824  5885 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:54.276521  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:54.378273  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 128772us
W20260709 13:01:54.465538  6738 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 13:01:54.466288  6734 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
W20260709 13:01:54.451537  6423 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.683s	user 0.014s	sys 0.000s
I20260709 13:01:54.250241  1610 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:01:54.269954  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:54.443359  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 235019us
I20260709 13:01:54.370829  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:54.450196  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 210122us
I20260709 13:01:54.469437  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.207891  6712 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:54.458263  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:54.471844  1686 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:54.473721  1690 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:01:54.452004  1611 raft_consensus.cc:2889] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:54.424319  6629 raft_consensus.cc:3055] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:54.238983  6666 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.247997  6340 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.476632  2232 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.376657  6663 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.461292  6637 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:54.477298  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:54.480645  1686 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: []
I20260709 13:01:54.483379  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:54.485339  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:54.485368  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:54.489012  2237 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:54.489724  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.492256  1584 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:54.492976  6671 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:54.475279  6423 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.335479  6672 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:54.495683  2230 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:54.490367  1583 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:54.394383  6305 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.483270  6666 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.494429  6317 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:01:54.568048  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.545805  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:54.560942  2234 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:54.571468  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:54.545856  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 51615.5us
I20260709 13:01:54.568324  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:54.574225  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 89562.1us
I20260709 13:01:54.489446  6681 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:54.578218  2229 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:54.577596  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:54.477231  6629 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:54.567421  6712 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.582374  6317 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:54.588156  6317 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:54.497983  5885 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:54.598235  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.572985  6671 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.481772  6677 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.098s
I20260709 13:01:54.603964  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:54.501132  6637 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:54.611083  2234 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:54.579312  6681 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:54.489535  6663 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.488737  6340 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.613109  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:54.496850  6381 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.106s
I20260709 13:01:54.617494  6629 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:54.619385  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:54.644346  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.644447  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.646073  2237 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:54.606635  6535 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:01:54.648729  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 165455us
I20260709 13:01:54.612159  6728 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:54.647926  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:54.614473  6681 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:54.614426  6393 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:01:54.655630  1685 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:54.656302  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:54.657670  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.658608  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:54.659503  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 377583us
I20260709 13:01:54.680748  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.682341  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:54.683562  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:54.643711  6671 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:54.676182  6639 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.572s	user 0.007s	sys 0.004s
I20260709 13:01:54.689611  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:54.677572  6635 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:54.699832  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 209569us
W20260709 13:01:54.691179  6574 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.579s	user 0.010s	sys 0.000s
W20260709 13:01:54.714165  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:54.721524  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:54.722774  1509 leader_election.cc:336] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.514s (SENT)
I20260709 13:01:54.723574  6476 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:54.723601  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:54.728636  2230 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:01:54.680320  6695 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:01:54.705529  6731 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:54.760478  1509 leader_election.cc:336] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.534s (SENT)
I20260709 13:01:54.771114  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:54.773109  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:54.774209  1688 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:54.774477  6643 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:01:54.775712  6643 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:54.776288  6643 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.780997  6643 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:54.781797  6643 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:54.782346  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:54.783583  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:54.785995  6715 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.255s	user 0.000s	sys 0.004s
I20260709 13:01:54.789924  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:54.790777  1688 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:54.791275  1688 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:54.791488  1542 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 16. Ops: []
I20260709 13:01:54.792203  1688 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:01:54.793015  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 203126us
I20260709 13:01:54.793887  1688 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:54.770067  2232 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:01:54.707693  6629 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:54.763906  2228 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:54.765844  2233 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:54.768066  6352 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:54.770419  2234 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:54.864110  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:54.848564  1692 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.001s
I20260709 13:01:54.868556  1692 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:01:54.701169  6639 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.720979  6574 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:54.853628  6423 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:54.903002  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 238839us
I20260709 13:01:54.899741  2232 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:54.904402  2232 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:54.906785  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:54.907979  1684 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:54.708676  6637 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.735936  6635 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.869410  6672 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:54.917657  2203 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:54.919086  1686 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:54.900588  6305 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:54.932582  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:54.930075  6556 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.080s
W20260709 13:01:54.936861  6747 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:01:54.937952  6556 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:54.938804  6556 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:54.925834  6663 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:54.942185  2232 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:54.943878  2232 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:54.952468  6574 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:54.954859  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:54.919667  6535 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.016s
W20260709 13:01:54.964574  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 204143us
I20260709 13:01:54.966012  1692 raft_consensus.cc:2410] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:01:54.938825  6550 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:54.923003  6596 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:01:54.956959  6756 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:01:54.967900  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 191568us
I20260709 13:01:54.970656  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:54.974084  1509 leader_election.cc:336] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.602s (SENT)
I20260709 13:01:54.978168  1690 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:54.973724  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:54.985872  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:54.986632  1535 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:54.987613  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:54.988495  1535 raft_consensus.cc:2410] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:01:54.988358  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:54.989512  6663 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:54.990041  6663 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:54.990463  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:54.990571  6663 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:54.992775  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:54.997736  6671 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:55.001068  6695 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:55.001698  6569 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.001706  6695 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:55.002422  6569 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.004453  6695 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:54.967528  6550 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:55.005460  1538 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 13:01:54.968515  6756 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:55.005703  6550 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.007004  6550 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:55.007390  6756 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.014328  1583 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:55.015570  1583 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:55.017999  1691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1852) took 3839 ms (client timeout 1602 ms). Trace:
W20260709 13:01:55.018407  1691 rpcz_store.cc:269] 0709 13:01:51.178702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:51.178818 (+   116us) service_pool.cc:224] Handling call
0709 13:01:55.017777 (+3838959us) spinlock_profiling.cc:243] Waited 2.79 s on lock 0x7b540003e154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:55.017910 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6419496832}
I20260709 13:01:55.019954  6556 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:55.020932  6556 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:55.022485  6556 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:55.022799  6340 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:55.023381  6556 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:55.024320  6542 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:54.980633  6423 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 13:01:55.026162  6535 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:55.027621  6728 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:55.030966  6639 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:55.031266  1540 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:55.031927  1508 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:55.033270  6569 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:55.021315  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:55.098361  1690 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:01:55.009398  6756 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:01:55.099767  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 67915us
I20260709 13:01:55.118119  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:55.118897  1543 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:55.112977  6663 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.119963  1689 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:55.106734  6762 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:55.127125  6762 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:55.128100  6762 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.038509  2203 leader_election.cc:400] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15.
I20260709 13:01:55.131019  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:55.131320  2203 leader_election.cc:403] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:55.040505  6666 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:55.041357  6352 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:55.041563  6675 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:55.041383  6672 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:01:55.042953  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50271.1us
W20260709 13:01:55.043761  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:55.145793  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:55.146239  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.048523  6574 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:55.048748  2201 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:55.049557  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:55.148245  1685 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:55.051636  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.053154  1688 raft_consensus.cc:2410] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:01:55.053658  6635 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:55.053167  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 966) took 1020 ms. Trace:
I20260709 13:01:55.149587  2235 rpcz_store.cc:276] 0709 13:01:54.033157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.034432 (+  1275us) service_pool.cc:224] Handling call
0709 13:01:54.034739 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.167776 (+133037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.052819 (+885043us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54005c0dd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:55.053080 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304748928}
W20260709 13:01:55.058115  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.058935  6515 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:55.060576  2232 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:01:55.174647  2232 pending_rounds.cc:85] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 9
I20260709 13:01:55.174903  2232 pending_rounds.cc:107] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:01:55.175165  2232 raft_consensus.cc:2889] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:55.063256  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:55.003190  6653 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.754s	user 0.011s	sys 0.000s
W20260709 13:01:55.033905  1537 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2891) took 1236 ms (client timeout 1605 ms). Trace:
I20260709 13:01:55.176041  2236 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
W20260709 13:01:55.078627  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 284435us
I20260709 13:01:55.096956  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 952) took 1118 ms. Trace:
I20260709 13:01:55.183877  1540 rpcz_store.cc:276] 0709 13:01:53.978173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.978315 (+   142us) service_pool.cc:224] Handling call
0709 13:01:55.030839 (+1052524us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b5800003998. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:01:55.031082 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:55.096521 (+ 65439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.096857 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11896192}
I20260709 13:01:55.134514  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:55.135716  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:55.184959  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:55.140199  6767 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:01:55.145574  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 107280us
I20260709 13:01:55.061239  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1036) took 1047 ms. Trace:
I20260709 13:01:55.068897  6681 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:55.185202  1543 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:55.137984  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:55.186980  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 138318us
W20260709 13:01:55.183250  1537 rpcz_store.cc:269] 0709 13:01:53.797174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:53.797379 (+   205us) service_pool.cc:224] Handling call
0709 13:01:55.033759 (+1236380us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54003458d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:55.033854 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":835857664}
W20260709 13:01:55.190385  1509 leader_election.cc:336] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.854s (SENT)
I20260709 13:01:55.193989  1542 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:55.193887  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:55.186295  1683 rpcz_store.cc:276] 0709 13:01:54.014353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.014484 (+   131us) service_pool.cc:224] Handling call
0709 13:01:54.047468 (+ 32984us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.060898 (+ 13430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.060932 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:54.094415 (+ 33483us) log.cc:844] Serialized 26 byte log entry
0709 13:01:54.578028 (+483613us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:54.578482 (+   454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:54.578569 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:54.976703 (+398134us) raft_consensus.cc:1685] finished
0709 13:01:54.976727 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.060970 (+ 84243us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b540014a350. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:55.061175 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1239,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":16352,"spinlock_wait_cycles":67179264,"thread_start_us":440849,"threads_started":1,"wal-append.queue_time_us":575750}
W20260709 13:01:55.197484  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.199625  6515 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:55.200017  6518 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15.
I20260709 13:01:55.202802  6352 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:55.204362  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 69918.1us
W20260709 13:01:55.206760  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.211463  2201 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:55.174024  6574 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:55.183488  6629 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:55.187219  6767 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:55.215375  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1195) took 1059 ms. Trace:
I20260709 13:01:55.196070  6554 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:55.223166  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:55.238516  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 322689us
I20260709 13:01:55.200080  6515 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:55.241730  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.211086  6663 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:55.218655  6731 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:55.250097  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:55.250726  2203 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:55.189002  6653 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:55.174096  6556 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:55.254905  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.226433  6639 raft_consensus.cc:2764] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 13:01:55.256639  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.226635  6635 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:55.257798  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 73583.4us
I20260709 13:01:55.268472  6672 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:01:55.268743  1509 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.879s (SENT)
I20260709 13:01:55.269027  6672 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:55.249346  6515 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:55.269204  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:55.271093  1509 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.903s (SENT)
I20260709 13:01:55.269528  6672 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:55.238336  6596 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:01:55.272742  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:55.281644  6767 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.282586  6748 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:55.283511  6767 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:55.285391  1509 leader_election.cc:336] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.918s (SENT)
I20260709 13:01:55.274331  6574 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.276613  6675 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:55.290980  6675 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:55.291390  6574 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:55.277115  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:55.292049  6675 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:55.279007  6653 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:55.293152  6675 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:55.279299  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:55.221891  2230 rpcz_store.cc:276] 0709 13:01:54.156143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.777449 (+621306us) service_pool.cc:224] Handling call
0709 13:01:54.777950 (+   501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:54.778247 (+   297us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:01:55.065085 (+286838us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.214248 (+149163us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.214904 (+   656us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:55.214975 (+    71us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:55.215049 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.215150 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.215300 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":286072,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":101,"raft.queue_time_us":4949,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":10563,"spinlock_wait_cycles":12399744,"thread_start_us":431083,"threads_started":2}
I20260709 13:01:55.294119  1690 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:55.279003  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:55.281687  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:55.288801  6709 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:55.277817  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:55.259397  6554 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:55.294844  1509 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:55.295909  1543 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:01:55.302942  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:55.303725  1690 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:55.252923  6728 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:55.308717  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:55.309733  1543 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:01:55.300035  1685 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:01:55.316027  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:55.322078  1543 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:01:55.294474  6698 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:55.339133  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:55.340163  1688 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:55.253235  6514 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:01:55.301076  6653 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:01:55.350533  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 160816us
I20260709 13:01:55.353454  1657 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:55.253909  6629 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.358657  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:55.354806  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 337965us
I20260709 13:01:55.281220  6515 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.370472  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:55.371135  1540 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:55.317482  6635 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:55.366532  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:55.409442  1543 raft_consensus.cc:2393] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:55.367641  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 82309.6us
I20260709 13:01:55.373654  6393 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.374575  6663 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:55.376143  6381 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:55.370326  6352 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:55.378648  6554 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.381606  6340 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:01:55.383842  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2901) took 1177 ms. Trace:
I20260709 13:01:55.352272  6728 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:55.358125  6653 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:55.357781  6748 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:55.394570  6629 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:55.417575  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:55.396193  1507 leader_election.cc:336] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.557s (SENT)
W20260709 13:01:55.453030  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 56892.8us
W20260709 13:01:55.455278  1507 leader_election.cc:336] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.602s (SENT)
I20260709 13:01:55.455720  1507 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:55.398340  6681 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:55.400552  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:55.339958  6698 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:55.405807  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 52485.8us
I20260709 13:01:55.407188  6515 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:55.404985  1690 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:01:55.351868  6132 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
W20260709 13:01:55.413995  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.414572  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:55.456990  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:55.479588  1538 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:55.479809  1534 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:55.448901  1541 rpcz_store.cc:276] 0709 13:01:54.206871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.783688 (+576817us) service_pool.cc:224] Handling call
0709 13:01:54.784088 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:54.784624 (+   536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:54.784663 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:54.859240 (+ 74577us) log.cc:844] Serialized 52 byte log entry
0709 13:01:55.231610 (+372370us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:01:55.231668 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.231769 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.383584 (+151815us) raft_consensus.cc:1685] finished
0709 13:01:55.383602 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.383779 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":115,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"thread_start_us":311421,"threads_started":1,"wal-append.queue_time_us":475543}
I20260709 13:01:55.481541  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:55.481865  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:55.483510  2236 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:55.485746  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:55.486063  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:55.487434  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:55.485802  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:55.490521  2231 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:01:55.491731  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:55.489085  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:55.494412  2232 raft_consensus.cc:2376] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14. Re-sending same reply.
I20260709 13:01:55.499336  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:55.500499  1685 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:55.416049  6666 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:55.506093  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:55.509292  1612 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:01:55.507442  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:55.510607  2237 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:55.450460  6653 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:55.512230  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:55.512398  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:55.505713  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:55.515796  2233 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:55.466277  6698 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:55.511343  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 156952us
I20260709 13:01:55.513276  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:55.479801  6769 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:55.520466  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 65217.1us
W20260709 13:01:55.521808  1507 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.629s (SENT)
I20260709 13:01:55.522200  1507 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:55.522815  2229 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:55.517961  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:55.480648  6381 consensus_queue.cc:1243] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 13:01:55.526129  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 125712us
I20260709 13:01:55.519855  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:55.569141  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:55.570163  1690 raft_consensus.cc:2410] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:01:55.571185  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:55.572853  1687 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:01:55.577672  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 71076.1us
I20260709 13:01:55.578627  1657 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:55.508805  6629 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:55.579777  6769 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.584358  6352 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:55.584551  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.593961  6672 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.460309  6663 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:55.596328  6305 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:55.517844  6317 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.600230  6728 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.602389  1685 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: [10.10-14.11]
W20260709 13:01:55.604131  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 82346us
I20260709 13:01:55.604246  6671 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:55.606143  1507 leader_election.cc:336] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.713s (SENT)
I20260709 13:01:55.600781  6317 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.527915  6748 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:55.613369  6571 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:55.613615  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:55.614569  1615 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 12: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:55.616155  6663 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:55.525460  2201 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:55.627405  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 102049us
I20260709 13:01:55.629055  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:55.614112  6571 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.630003  1535 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:55.625056  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:55.639907  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2900) took 1434 ms. Trace:
I20260709 13:01:55.640437  1539 rpcz_store.cc:276] 0709 13:01:54.205355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:54.488636 (+283281us) service_pool.cc:224] Handling call
0709 13:01:54.508865 (+ 20229us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:54.550053 (+ 41188us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:01:54.937173 (+387120us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b1800362f20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fc455 00007fe0934e43c7 00007fe08faf69a7 00007fe08fb060bf 00007fe08fb06038
0709 13:01:55.231320 (+294147us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.449258 (+217938us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:55.449813 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 13:01:55.449976 (+   163us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:55.451739 (+  1763us) log.cc:844] Serialized 14 byte log entry
0709 13:01:55.452026 (+   287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:55.452134 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:55.619435 (+167301us) spinlock_profiling.cc:243] Waited 6.93 ms on lock 0x7b1800362f20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:01:55.639634 (+ 20199us) raft_consensus.cc:1685] finished
0709 13:01:55.639664 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:55.639838 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40610,"prepare.queue_time_us":132620,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":81,"raft.queue_time_us":50694,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":26570,"spinlock_wait_cycles":23072000,"thread_start_us":442959,"threads_started":2,"wal-append.queue_time_us":357851}
W20260709 13:01:55.646656  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.597038  6305 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:55.645727  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:55.656378  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:55.657099  1542 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:55.654960  2201 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:55.656625  1536 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
W20260709 13:01:55.669633  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.670831  2235 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:55.670555  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:55.673738  1615 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:55.516178  6393 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.674889  6122 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:55.596381  6672 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:55.676242  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:55.643467  6518 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:01:55.672465  6767 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:55.675590  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:55.697145  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 208489us
I20260709 13:01:55.695756  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:55.670661  6601 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:55.672955  6569 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:55.672946  5885 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:55.727293  1538 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:01:55.685962  6542 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:55.741019  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:55.742193  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 163659us
W20260709 13:01:55.744024  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44324us
I20260709 13:01:55.748013  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:55.741770  1615 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
W20260709 13:01:55.755640  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.755551  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:55.749300  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:55.760083  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:55.725548  6767 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:55.756603  1612 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:55.729352  5885 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:55.743641  6542 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:55.763761  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:01:55.744865  6317 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:55.773092  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:55.798635  1507 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.922s (SENT)
W20260709 13:01:55.786430  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.795337  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:55.801506  1615 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:55.744741  6783 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:55.802466  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:55.766393  6388 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:55.762943  6381 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:55.806236  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:55.773087  6542 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:55.769007  6515 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:55.769088  5885 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:01:55.808667  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.738226  6569 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.810966  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:55.818990  1612 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
W20260709 13:01:55.816879  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.805166  6388 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:55.833714  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 160206us
I20260709 13:01:55.810559  6535 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:55.817378  6515 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:55.816073  6783 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:55.819324  6381 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:55.856303  1685 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:55.845152  1615 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:55.863929  6305 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:55.835546  6388 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.896829  6767 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.859462  6515 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:01:55.867022  6782 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:01:55.818759  6731 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:01:55.870450  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:55.871176  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:55.913250  1618 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:55.871905  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:55.913564  1615 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:01:55.872659  1609 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 13:01:55.875545  6756 consensus_meta.cc:287] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.773s	user 0.009s	sys 0.002s
W20260709 13:01:55.876523  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.915530  1611 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 12: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:55.878319  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:55.879452  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:55.883529  6780 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:01:55.884182  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:55.889546  6783 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:55.866114  6786 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:01:55.877401  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:55.902499  6785 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:01:55.900151  6569 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:55.897792  6535 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:55.941882  6731 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:55.914845  6393 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:55.906311  6571 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:55.918195  6653 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:55.905821  6305 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:55.951103  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:55.989842  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 222424us
I20260709 13:01:55.990458  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:55.990653  1610 raft_consensus.cc:2410] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:01:55.906658  6698 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.926036  6554 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:55.965654  1616 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
W20260709 13:01:55.971743  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 276058us
I20260709 13:01:55.906363  6381 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.994815  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:55.995008  1657 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:55.995469  1613 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:56.004364  2228 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:56.005946  1658 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:56.006567  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.992442  6340 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:01:56.008499  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:55.996743  6554 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:55.971272  6601 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:55.994421  6352 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:01:56.073674  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.011099  6675 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:56.078234  6675 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.079305  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.080134  1613 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:01:56.033774  6677 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:56.081696  6677 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:56.082741  6768 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:56.083207  6768 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:56.082672  6677 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.083889  6677 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:56.086299  6571 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.087049  6571 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.090426  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.091125  1534 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:56.091198  6666 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:01:56.092312  1539 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:56.092423  6666 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:56.093659  6675 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:55.998862  6569 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:56.091207  1508 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:55.993144  6515 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:55.986690  6518 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:56.097539  6569 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:56.098104  1540 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:01:56.093703  6764 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:01:56.109155  1507 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.068s (SENT)
I20260709 13:01:56.023435  6554 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:56.100100  6388 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:56.125698  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 267689us
I20260709 13:01:56.095722  6798 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:56.127961  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:56.131322  1612 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 12: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:56.108379  6569 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.108315  6764 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:56.138538  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:56.127555  6761 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.813s	user 0.000s	sys 0.006s
I20260709 13:01:56.132580  1657 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:56.109016  6769 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:56.140942  1617 raft_consensus.cc:1217] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Deduplicated request from leader. Original: 11.9->[13.10-13.10]   Dedup: 13.10->[]
I20260709 13:01:56.073634  6535 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:56.163801  1540 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:56.125921  6763 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:56.169706  1617 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:01:56.076915  6542 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.171622  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:56.172315  2233 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
W20260709 13:01:56.175266  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.128822  6514 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.200773  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:56.206259  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:56.201371  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 68859.7us
I20260709 13:01:56.207346  1657 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:56.129498  6798 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:56.207569  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.114190  5885 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.213037  1613 raft_consensus.cc:2410] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:01:56.212787  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:56.161798  6675 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:01:56.209703  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.160576  6569 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:56.205518  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:56.218250  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.169484  6763 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:01:56.222035  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:56.223421  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:56.217681  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 211788us
I20260709 13:01:56.221392  1615 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:01:56.200893  6672 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:56.222322  1657 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:56.140378  6776 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:01:56.224766  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:56.205593  6423 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:56.243120  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:56.243495  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:56.243808  2228 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:56.241182  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:56.214870  6629 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.216524  6675 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.250699  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:56.227173  6763 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:56.253443  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:56.244258  6352 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:56.242630  6423 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.268115  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:56.269060  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:56.269707  1616 raft_consensus.cc:2376] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10. Re-sending same reply.
I20260709 13:01:56.269838  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.270223  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:56.270531  1612 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:56.270614  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:56.251173  6675 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:56.270987  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.219206  6796 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:01:56.271395  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.271792  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.272233  1612 raft_consensus.cc:2410] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:01:56.272545  1609 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:56.255797  2236 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:56.273481  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.269884  6423 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:56.259110  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:56.259703  1657 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:56.275178  2234 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:01:56.275518  6809 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:56.276021  6352 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:56.276383  6809 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:56.276823  1657 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:56.261061  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:56.084136  6768 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.277189  6352 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.277918  2230 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.278198  6768 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.280520  1610 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:56.275190  2235 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:56.281674  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.282497  1610 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:56.271013  1618 raft_consensus.cc:2410] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:01:56.255482  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.273123  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.214293  6514 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.271337  1616 raft_consensus.cc:2368] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12.
I20260709 13:01:56.274298  1609 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:56.289953  1613 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:56.274209  2232 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:56.305192  1617 raft_consensus.cc:2393] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:56.307469  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:56.308163  1536 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:01:56.312232  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.257822  6672 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:56.255040  6653 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:56.338513  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:56.340605  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.316128  6672 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:56.341741  6672 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:56.344666  6554 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.322158  2235 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: [10.10-14.11]
I20260709 13:01:56.325573  6812 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:01:56.326030  6802 log.cc:927] Time spent T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 13:01:56.326575  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.219691  6518 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:56.373808  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.373931  2233 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:01:56.329733  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 52995.6us
I20260709 13:01:56.374840  6554 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:56.375677  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:56.376025  6388 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:56.376446  2232 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:56.330845  6542 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:56.208134  6601 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.383098  2235 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:56.383116  2233 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
W20260709 13:01:56.336136  6122 consensus_meta.cc:287] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.651s	user 0.010s	sys 0.003s
I20260709 13:01:56.251557  6629 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.345997  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.346951  6514 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:56.315440  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.394181  1534 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:56.347939  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:56.350410  2230 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:01:56.352025  6352 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:01:56.361493  6515 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:56.348374  6809 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 13:01:56.363792  6764 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.365887  6675 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.366876  6698 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:56.411221  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.411638  1536 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:56.370167  6666 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:56.374934  1657 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:56.424582  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 49718.1us
I20260709 13:01:56.424697  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:56.389081  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:56.395522  1542 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:56.399972  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:56.426092  1535 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:56.405408  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:56.406972  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.422322  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:56.399053  6122 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:56.432058  1690 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:56.433624  1685 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:56.433947  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.425271  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:56.430099  2233 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:01:56.436095  1683 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:56.436336  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:56.436043  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:56.437208  1692 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:56.437470  1536 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:56.434423  1617 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:56.408607  6340 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:56.440083  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.435501  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.440752  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:56.449275  1684 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:56.422185  6653 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.423580  6812 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.457386  1543 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:56.458863  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:56.470053  1540 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:56.470355  6569 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.472535  6816 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.473117  6816 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:56.474583  6629 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:56.475232  6629 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:56.475981  6629 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.476454  6629 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:56.458447  6653 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:56.479208  6672 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:56.422674  6762 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:01:56.486472  6675 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:56.426113  6635 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:01:56.488862  6798 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.489845  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:56.429064  6810 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:56.494081  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:56.431702  6809 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:56.514210  1690 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:56.494122  1686 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:01:56.479820  6653 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:56.495913  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 74309.6us
I20260709 13:01:56.503720  1535 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:01:56.438153  6783 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.518119  6672 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:56.585021  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.585752  1612 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:56.426170  6671 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 13:01:56.586199  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.590531  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:56.587761  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.475270  6816 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:56.593016  6783 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:56.594517  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 312021us
I20260709 13:01:56.595386  6800 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.495298  6812 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:56.596294  6800 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.597885  6556 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.598121  6812 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:56.598457  6556 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:56.599083  6796 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:56.599341  6317 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:56.599723  6796 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:56.600008  6317 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:56.600567  6556 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:56.603312  6629 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:56.603147  6728 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:01:56.605110  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:56.605911  1537 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:56.607113  6317 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:56.610952  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:56.611856  6571 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:56.614095  1583 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:56.615903  6639 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:56.616464  6639 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:56.618422  6388 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.619071  6388 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:56.619803  6769 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:56.620692  6769 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:56.622133  5885 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:56.623081  5885 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:01:56.624230  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:56.625636  6806 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:01:56.624177  5885 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:01:56.626063  6806 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:56.626802  6550 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:56.627228  6550 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:56.627095  6806 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.628553  6806 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:56.582015  6340 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.449741  6804 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:56.514853  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:56.595469  6783 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:56.600311  6796 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.628147  6550 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.577960  6569 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:56.589883  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:56.570398  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:56.634074  1617 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:01:56.634958  1539 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:56.636636  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.636649  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.653123  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:56.638531  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:56.566011  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 136997us
I20260709 13:01:56.655536  1690 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:56.655867  1543 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.575840  1692 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:56.643199  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:56.652375  2232 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 13:01:56.658478  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.661064  1583 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:56.655124  1609 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:56.665014  2233 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:01:56.665392  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:56.665562  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.666543  1617 raft_consensus.cc:2393] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:56.564114  6809 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.670642  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:56.672115  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:56.682986  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.685283  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.686632  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:56.687366  1618 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:56.693364  1543 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:01:56.674654  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:56.695676  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:56.651089  6635 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:01:56.698588  1686 raft_consensus.cc:2410] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
I20260709 13:01:56.473215  6763 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.696393  2237 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:01:56.697934  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:56.702302  1609 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:56.698189  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:56.703236  1690 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:56.695753  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.703904  1617 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
W20260709 13:01:56.700424  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.667270  6812 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:56.703598  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:56.694604  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:56.707717  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:56.708593  1617 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:56.708719  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:56.709378  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:56.753681  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 50163.6us
I20260709 13:01:56.710923  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:56.754653  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.684533  6681 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:56.685882  6671 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:01:56.765053  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:56.769708  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:56.654098  6804 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.711602  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:56.783447  1692 raft_consensus.cc:2368] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17.
W20260709 13:01:56.783519  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 88853.8us
W20260709 13:01:56.757575  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 332338us
I20260709 13:01:56.786909  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:56.761644  1535 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:56.718258  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.712698  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.720410  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:56.789660  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:56.790222  1686 raft_consensus.cc:2376] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15. Re-sending same reply.
I20260709 13:01:56.791024  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:56.792236  1686 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:56.793583  6574 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:56.794100  6574 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.680734  6816 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 13:01:56.753942  6550 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:56.676177  6809 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:56.753199  6569 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:56.798026  6569 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:56.798465  6569 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:56.805792  6809 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.796336  6574 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:56.806315  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.807265  1689 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:56.808658  6809 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:56.809336  1536 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:56.805866  6569 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:56.810703  1534 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:01:56.808694  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.813077  6569 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:56.761101  6629 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:56.834326  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:56.698760  6317 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:01:56.844179  1537 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:56.852007  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:56.846484  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.819110  1686 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
W20260709 13:01:56.850668  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 92603.2us
I20260709 13:01:56.676726  6639 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:56.764624  6388 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:56.855086  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:56.865008  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.865790  6675 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:56.866580  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:56.867269  2237 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:01:56.867653  6629 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:56.868714  6629 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:56.877859  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.879261  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.880447  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.882666  1537 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:01:56.883601  1537 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:56.887820  1687 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:56.845917   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:01:56.887646  1618 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:56.906530  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:56.907595  1543 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:01:56.894079  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:56.784974  6800 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:56.915238  2203 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:56.912210  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.918507  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.919391  1539 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:56.928328  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.927173  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:56.926455  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:56.930490  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:56.931255  1692 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:01:56.936275  6352 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:01:56.937048  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.816182  6556 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.105s
I20260709 13:01:56.938772  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.939947  1617 raft_consensus.cc:2410] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:01:56.916580  6388 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:01:56.939108  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:56.949364  1537 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:01:56.949780  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:56.951467  1617 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:56.953346  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:57.000253  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:56.926864  6601 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:56.925205  6810 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.011418  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:57.020510  1509 leader_election.cc:336] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.557s (SENT)
I20260709 13:01:57.012027  1684 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:01:57.073102  1543 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:01:57.075441  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:57.032240  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:57.079501  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.079396  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.080193  1617 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:57.012619  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:57.082243  1688 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:01:57.080911  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 69555.1us
I20260709 13:01:56.948787  6388 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.084743  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.085626  6798 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:57.086853  6352 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:57.083004  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:01:57.019806  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 80764us
I20260709 13:01:57.089110  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.103035  1617 raft_consensus.cc:2393] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 10: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:57.103530  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:01:57.109170  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 88683.2us
I20260709 13:01:57.108574  1507 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:57.111125  1507 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.107450  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:57.009066  6800 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:57.111866  1509 leader_election.cc:336] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.577s (SENT)
W20260709 13:01:57.113695  1542 consensus_meta.cc:287] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.684s	user 0.007s	sys 0.004s
I20260709 13:01:57.117782  1542 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:57.070163  6635 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:57.118455  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 331586us
I20260709 13:01:57.120995  1657 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.076010  6601 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:57.120208  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.127127  1688 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:01:57.132861  1692 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:01:57.140281  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.141114  1507 leader_election.cc:400] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17.
I20260709 13:01:57.125815  1657 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:57.141765  1507 leader_election.cc:403] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:01:57.143265  1657 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.098793  6804 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:57.129784  6569 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.107973  6352 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.145388  1617 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:57.160025  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:57.160597  1617 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:57.154233  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:57.119809  6831 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 13:01:57.157579  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.103538  6388 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:57.012321  6554 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.117s
I20260709 13:01:57.119812  6748 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:57.120530  6629 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.168982  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.119215  6759 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:01:57.120400  6671 raft_consensus.cc:2764] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:01:57.124581  6810 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:01:57.183918  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:57.157115  6825 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.303s	user 0.000s	sys 0.000s
I20260709 13:01:57.188436  1617 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
W20260709 13:01:57.157226  6820 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
W20260709 13:01:57.157157  6828 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.286s	user 0.000s	sys 0.004s
W20260709 13:01:57.146229  6837 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:01:57.191017  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:57.192518  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 72036.3us
I20260709 13:01:57.157245  6833 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:57.157135  6822 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:57.156822  6569 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:57.145613  6635 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.234032  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:57.143819  6518 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:57.167521  6340 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:57.156662  6122 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:57.238081  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 54224.8us
W20260709 13:01:57.157231  6821 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.458s	user 0.001s	sys 0.003s
I20260709 13:01:57.239243  1610 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:57.239768  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.280404  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:57.145931  6829 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:57.192477  6556 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:57.285835  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:57.157146  6838 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:57.286273  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.286445  6556 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:57.286684  1690 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:57.286121  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 46501.6us
I20260709 13:01:57.287427  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:57.287629  1507 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:57.288118  6840 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.191128  6810 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.249702  6601 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.288739  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:57.288959  6840 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:57.251926  6352 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.289511  6840 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:57.291177  6352 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:57.242285  6833 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.291834  6833 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:57.290613  6840 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.292523  1610 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:57.281107  6518 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.295169  6518 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:57.287484  6556 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.254051  6569 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:57.298961  6569 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:57.241451  6671 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:01:57.299688  1610 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:57.301167  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:57.192324  6629 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:57.256171  6830 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
W20260709 13:01:57.256711  6796 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.570s	user 0.003s	sys 0.006s
W20260709 13:01:57.257033  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 146746us
W20260709 13:01:57.261550  6826 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.461s	user 0.002s	sys 0.000s
I20260709 13:01:57.261801  6832 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.263339  1616 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:01:57.264458  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.267604  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:57.241302  6836 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:57.234988  6709 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.329730  6709 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.330502  6629 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:57.304687  6810 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:57.331868  6709 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:57.219940  6763 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:57.271692  1609 consensus_meta.cc:287] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.564s	user 0.007s	sys 0.003s
I20260709 13:01:57.333334  1609 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:01:57.271750  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:57.334317  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:57.334968  1540 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:01:57.335206  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:57.335963  1683 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 10: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:57.336767  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:57.159432  6800 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.337404  1685 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:57.286568  6838 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.338032  6838 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:57.339030  6800 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:57.285748  6829 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:57.339787  6829 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:57.339828  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:57.189504  6759 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:57.341200  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:57.341795  1613 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:57.342168  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:57.342227  6759 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:57.343999  6759 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:57.344714  1535 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:01:57.334317  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.287732  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:57.346803  1683 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:57.254209  6843 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:01:57.302914  1689 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:57.305387  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.306481  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.318518  6832 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.320569  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1204) took 1071 ms. Trace:
W20260709 13:01:57.314503  6768 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.032s	user 0.006s	sys 0.006s
I20260709 13:01:57.325670  6796 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.335793  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:57.274837  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.288769  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.348901  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:57.366160  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.366941  1539 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:57.349357  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:57.354460  2229 rpcz_store.cc:276] 0709 13:01:56.249334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.249601 (+   267us) service_pool.cc:224] Handling call
0709 13:01:56.254710 (+  5109us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.282673 (+ 27963us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:56.282710 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:56.282747 (+    37us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:56.282775 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.320232 (+1037457us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b5400560c54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:57.320306 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.320492 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63881472}
W20260709 13:01:57.323406  6681 consensus_meta.cc:287] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.558s	user 0.009s	sys 0.000s
I20260709 13:01:57.356793  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:57.357488  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 52233.4us
I20260709 13:01:57.282794  6340 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:57.374459  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.375407  1610 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:57.363566  6833 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:57.369608  1540 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:57.370447  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 81758.1us
I20260709 13:01:57.360121  6832 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:57.412663  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.411923  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.371328  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:57.382388  1613 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:01:57.415158  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:57.416141  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.416925  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.417132  1613 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:01:57.417826  1610 raft_consensus.cc:2376] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15. Re-sending same reply.
W20260709 13:01:57.419953  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 48712.4us
I20260709 13:01:57.377853  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.420876  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:57.382342  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:57.423651  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:57.424015  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:57.425005  1610 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:57.431097  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:57.432107  1609 raft_consensus.cc:2376] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15. Re-sending same reply.
I20260709 13:01:57.369259  6835 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:57.434394  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.435611  1613 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:57.370124  6681 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:57.372610  6666 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:57.456935  1613 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:57.430153  6832 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:57.442255  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:57.462224  1618 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:01:57.459022  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:57.435398  6672 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:01:57.446182  1610 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:57.467744  1616 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:01:57.470475  1609 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:57.373347  6768 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:57.449682  6122 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.426000  6822 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:01:57.481603  1609 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:57.487793  1609 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:57.469538  6681 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:57.473496  6666 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:57.473832  6832 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:01:57.494309  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 179871us
I20260709 13:01:57.496228  1609 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:01:57.476229  6672 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:57.500741  1616 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
W20260709 13:01:57.511586  1509 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.629s (SENT)
I20260709 13:01:57.487666  2229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:01:57.514159  2229 raft_consensus.cc:606] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:57.488613  6847 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:01:57.522501  1509 leader_election.cc:336] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.713s (SENT)
I20260709 13:01:57.523061  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:57.523676  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1053) took 1687 ms. Trace:
I20260709 13:01:57.524255  1687 rpcz_store.cc:276] 0709 13:01:55.836613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.837621 (+  1008us) service_pool.cc:224] Handling call
0709 13:01:55.838013 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:56.888315 (+1050302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.523325 (+635010us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540028f0d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:57.523593 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2413874944}
W20260709 13:01:57.528476  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 116654us
W20260709 13:01:57.529204  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.479914  6768 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:01:57.535463  1509 leader_election.cc:336] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.751s (SENT)
W20260709 13:01:57.535567  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 179569us
W20260709 13:01:57.545370  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:57.546097  1509 leader_election.cc:336] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.799s (SENT)
I20260709 13:01:57.546625  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:57.538378  6550 consensus_meta.cc:287] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.740s	user 0.012s	sys 0.000s
I20260709 13:01:57.508764  6681 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.539127  6822 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:01:57.556499  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.541764  6666 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:01:57.528859  6847 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.557974  1582 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.557462  6796 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:57.580152  6855 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:57.569130  6832 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:57.576501  6833 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:57.593145  6764 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:57.554776  6672 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:57.591454  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.584098  6340 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:57.605537  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:57.612802  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.586900  6854 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:01:57.617717  1615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1887) took 1649 ms (client timeout 1713 ms). Trace:
W20260709 13:01:57.619846  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 61997.9us
I20260709 13:01:57.622035  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:57.621440  1615 rpcz_store.cc:269] 0709 13:01:55.950393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:55.959284 (+  8891us) service_pool.cc:224] Handling call
0709 13:01:57.599191 (+1639907us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54001b8854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:57.599301 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":529531648}
I20260709 13:01:57.593271  6681 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:57.631702  2229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:01:57.632392  2229 raft_consensus.cc:606] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:57.638125  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.645181  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.645468  6635 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.644275  6855 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:57.653925  6859 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.655839  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:57.597395  6838 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:57.656598  6855 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:57.663472  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:57.656816  6859 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.665409  1582 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:57.669474  6851 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:01:57.626457  6854 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.671514  6847 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:57.672014  6671 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17.
I20260709 13:01:57.638767  6853 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.683068  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:57.640415  6863 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:57.641065  6556 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:01:57.659447  6681 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:01:57.704948  6388 consensus_meta.cc:287] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.516s	user 0.008s	sys 0.002s
I20260709 13:01:57.696007  6768 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:57.687065  6305 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:57.708338  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1915) took 1128 ms. Trace:
I20260709 13:01:57.708998  1612 rpcz_store.cc:276] 0709 13:01:56.579825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.441221 (+861396us) service_pool.cc:224] Handling call
0709 13:01:57.452789 (+ 11568us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:57.453302 (+   513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.453343 (+    41us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:57.459075 (+  5732us) log.cc:844] Serialized 78 byte log entry
0709 13:01:57.551561 (+ 92486us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:01:57.551599 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.551676 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:57.708004 (+156328us) raft_consensus.cc:1685] finished
0709 13:01:57.708032 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:57.708256 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92031,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":14763,"thread_start_us":92069,"threads_started":1,"wal-append.queue_time_us":113687}
I20260709 13:01:57.718106  6829 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.722409  6810 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:01:57.723176  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:57.725237  1509 leader_election.cc:336] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.956s (SENT)
I20260709 13:01:57.725392  1688 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 17. Ops: []
I20260709 13:01:57.725733  1509 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:57.699802  1687 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:01:57.728438  1687 raft_consensus.cc:606] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:57.683387  6860 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:01:57.733620  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 137386us
I20260709 13:01:57.743381  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:57.735036  6829 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:57.695295  6863 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:57.747738  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:57.724838  6810 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:01:57.747747  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 64813.6us
I20260709 13:01:57.729460  6847 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:57.757155  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:57.757583  1509 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.985s (SENT)
W20260709 13:01:57.760075  1509 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.068s (SENT)
I20260709 13:01:57.760489  1509 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:57.678292  6635 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.771981  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.780455  1541 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:01:57.749476  6860 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:01:57.782182  6860 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:57.784503  6681 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:57.793632  6635 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:57.795526  6666 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.796177  6666 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.800118  6864 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.801578  6854 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:57.802646  1687 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:01:57.803429  1687 raft_consensus.cc:606] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:57.803984  1687 raft_consensus.cc:614] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:57.802187  6629 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.805771  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:57.806116  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:57.806854  1692 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:01:57.807255  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:57.775018  1509 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.086s (SENT)
I20260709 13:01:57.775697  6847 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.808182  6847 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.808467  6797 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.808408  6629 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.809315  1689 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:57.809114  6797 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.758605  6853 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:57.745879  6759 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:57.749958  6838 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:57.777832  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.816674  1616 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:57.739115  6388 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.757236  6863 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:57.806490  1686 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:57.811403  1509 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.814919  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:57.750867  6855 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:57.815302  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:57.826882  2234 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:57.816037  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:57.818434  2201 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:57.828155  2230 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:01:57.828559  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:57.832791  2201 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:57.816131  6542 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:57.833913  6550 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:57.834606  6550 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:57.834026  6542 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.834694  6629 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:57.835057  6550 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:57.836709  6542 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:57.839315  1617 raft_consensus.cc:2368] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18.
W20260709 13:01:57.842643  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 67647.7us
W20260709 13:01:57.844096  1509 leader_election.cc:336] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.847s (SENT)
I20260709 13:01:57.825851  6847 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:57.857028  1509 leader_election.cc:336] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.125s (SENT)
I20260709 13:01:57.839262  6796 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:57.857976  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:57.859692  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.838959  6800 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:01:57.818157  6759 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:57.854259  6865 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:01:57.860304  6862 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:57.845721  6542 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:57.866406  1686 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:01:57.840606  6799 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:01:57.858789  6797 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:01:57.879506  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 105161us
I20260709 13:01:57.880919  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.872567  6671 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:57.889693  6797 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:57.890179  6797 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:57.890256  6768 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:01:57.891091  6797 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:57.892437  6797 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:57.894107  6550 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:57.900378  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 57238.7us
I20260709 13:01:57.901198  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:57.881527  6798 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:01:57.903934  2201 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:57.904315  6798 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:57.905929  6542 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:01:57.907457  6542 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:01:57.908329  6542 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:01:57.909837  6601 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:57.910355  6601 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:57.910732  6601 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:57.906198  6388 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:57.922680  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:57.924463  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:57.899614  6550 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:57.928289  1582 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.931435  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:57.936362  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:57.937079  2230 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:57.937256  1582 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:57.937773  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:57.939672  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:57.940307  1687 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:57.914568  6675 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:01:57.937601  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:57.922725  6798 raft_consensus.cc:3055] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:57.926709  6854 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:01:57.926482  6768 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:57.986555  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:57.971385  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:57.953454  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:57.989849  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.999496  2231 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:01:57.990309  6800 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:57.989157  1687 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:01:57.991544  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2418) took 1345 ms. Trace:
I20260709 13:01:57.996865  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:58.000204  1509 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:58.001933  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:58.004070  1685 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:58.002887  1610 rpcz_store.cc:276] 0709 13:01:56.646143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.445721 (+799578us) service_pool.cc:224] Handling call
0709 13:01:57.446011 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.465736 (+ 19725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.991255 (+525519us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b540009b8a0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa63dc0 00007fe08faefa02 00007fe08faebdd6 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:01:57.991483 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7139200}
I20260709 13:01:58.004432  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.003270  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:57.929617  6874 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:57.998795  1611 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:58.004835  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.031562  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:58.054106  1610 raft_consensus.cc:2410] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 13:01:58.039863  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.020824  2230 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:58.003778  1690 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:58.006125  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:01:58.046965  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 109253us
I20260709 13:01:58.061293  1583 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:58.028484  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 96652us
I20260709 13:01:57.948266  6388 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:01:58.065889  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:57.948441  6675 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:57.948459  6847 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:01:57.986410  6798 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:58.075505  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:57.946683  6796 raft_consensus.cc:697] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:58.077858  6878 raft_consensus.cc:993] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:01:58.079538  6879 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:01:58.082688  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:57.996212  6860 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:58.100816  6860 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:58.101313  6860 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:58.083786  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:58.085987  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:58.086090  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 80061.2us
I20260709 13:01:58.005210  6822 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:01:58.088466  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:58.089282  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:58.090173  1612 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:58.006484  6748 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:01:58.091127  6675 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:58.099782  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:58.110340  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:58.074553  6854 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:58.082924  6759 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:58.134549  6762 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.074108  6542 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:01:58.135181  6762 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:58.135572  6542 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:58.136631  6542 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.110754  2230 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:58.087224  6635 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:58.111095  2237 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:58.100236  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:58.112900  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.142922  2234 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:01:58.076263  6768 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:58.106832  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:58.006541  6800 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.158748  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:58.109900  6876 raft_consensus.cc:1081] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:01:58.100831  6884 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:58.165774  2234 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:01:58.118993  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:58.084985  6809 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:58.125794  6675 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:58.125494  6796 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.126485  6854 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:58.133662  6862 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:58.134110  1690 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:01:58.146970  6635 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:58.109320  2203 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.687s (SENT)
I20260709 13:01:58.147049  2228 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:01:58.184465  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:58.115892  6882 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 13:01:58.111096  6388 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:58.163228  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.164669  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:58.119156  6799 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:01:58.166630  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:58.192934  1540 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
W20260709 13:01:58.170029  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 51171.6us
I20260709 13:01:58.170200  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:58.138620  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 63198.3us
I20260709 13:01:58.158241  6762 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:58.159056  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:58.184510  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 75216us
I20260709 13:01:58.161728  6812 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.186404  1539 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:01:58.188977  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:58.189234  2230 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:01:58.134110  6601 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.175575  6748 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:58.195226  1538 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:58.185807  6635 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:58.190862  1611 raft_consensus.cc:2376] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12. Re-sending same reply.
W20260709 13:01:58.196838  2203 leader_election.cc:336] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.739s (SENT)
W20260709 13:01:58.206507  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 47549.3us
I20260709 13:01:58.190549  6388 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.211898  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.205575  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:58.207646  1658 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:58.214048  1539 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:01:58.192705  6799 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:58.185693  6862 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:58.215266  1539 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:58.215340  6799 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.216405  6799 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:58.177270  6809 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.169868  6768 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.217352  6862 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:58.217540  6762 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:01:58.227214  6388 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:58.229617  6822 consensus_queue.cc:1059] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:01:58.229911  2203 leader_election.cc:336] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.847s (SENT)
I20260709 13:01:58.230734  6822 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:01:58.230520  6862 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:01:58.231109  6822 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:58.232136  6822 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:58.232586  6863 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.234730  6863 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:58.235409  1534 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:01:58.240839  1611 raft_consensus.cc:2410] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:01:58.241822  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.242432  1611 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:58.244350  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:58.219370  6809 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:58.231150  2203 leader_election.cc:336] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.976s (SENT)
I20260709 13:01:58.250262  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:58.250746  1535 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
W20260709 13:01:58.252245  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.252945  1657 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:58.253759  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46177.5us
I20260709 13:01:58.254510  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:58.220587  2231 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:01:58.225411  2233 raft_consensus.cc:2410] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:01:58.257474  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:58.261044  6833 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:58.261701  6833 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:58.262612  6833 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:58.265885  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.266786  1543 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:58.267164  6854 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:58.274304  1543 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:01:58.275178  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:58.275869  1539 raft_consensus.cc:2410] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:01:58.270277  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:58.271744  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:01:58.277873  2230 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:58.233315  6822 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:01:58.250049  6810 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:01:58.280725  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:58.281349  6550 raft_consensus.cc:2764] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:01:58.307066  6601 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:58.308145  6800 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:58.311563  6829 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:01:58.315440  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:58.315595  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:58.316133  1535 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:58.269905  6824 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:01:58.269642  6879 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:01:58.277037  6809 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:58.316443  1611 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:58.222469  6768 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:58.256239  6884 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.208004  6812 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:58.332857  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.333640  1611 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:01:58.336429  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:58.329118  6797 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:01:58.328959  6800 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:01:58.278558  6340 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:01:58.346290  6884 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.330320  6877 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:58.224304  6860 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.347668  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.293865  6569 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:58.353010  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:58.354934  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.334712  6809 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.357625  1611 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 12: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:01:58.357739  2203 leader_election.cc:336] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.007s (SENT)
I20260709 13:01:58.378141  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.379251  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.338213  6810 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:58.383638  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.381101  1535 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:58.350690  6800 consensus_queue.cc:1243] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 12.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:01:58.385815  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:58.387295  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:58.384593  1611 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:01:58.387977  1535 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:01:58.381119  6809 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:58.389487  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.391258  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.392313  1611 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:58.393365  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.394147  1611 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:58.382946  6854 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.398726  6854 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:58.401042  6854 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:58.405141  6812 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:58.348862  6797 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:58.406522  6797 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:58.411602  6860 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:58.413398  6569 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:58.414733  6569 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.417583  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:58.418463  1536 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:58.423950  2203 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.685s (SENT) [suppressed 76 similar messages]
I20260709 13:01:58.431195  1535 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:01:58.355522  6877 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.438162  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:58.439208  1535 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:01:58.337937  6635 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:58.441612  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:58.446884  1535 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:01:58.448024  1536 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:58.383239  6810 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:58.453982  1536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:33722
I20260709 13:01:58.441962  6877 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:58.454668  1536 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:01:58.455185  1536 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:58.457896  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:58.456141  1535 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:01:58.459858  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:58.461555  1611 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
W20260709 13:01:58.461843  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 190144us
I20260709 13:01:58.463882  2236 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:58.464426  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:58.465593  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 186036us
W20260709 13:01:58.469218  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.498605  1691 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:01:58.500279  1686 raft_consensus.cc:606] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:01:58.500710  1686 raft_consensus.cc:614] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:01:58.505828  2231 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
W20260709 13:01:58.506525  2203 leader_election.cc:336] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.533s (SENT)
W20260709 13:01:58.508014  1541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2406) took 1864 ms (client timeout 2071 ms). Trace:
I20260709 13:01:58.508422  1537 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 13:01:58.509541  2203 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.554s (SENT)
W20260709 13:01:58.508486  1541 rpcz_store.cc:269] 0709 13:01:56.643955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.644067 (+   112us) service_pool.cc:224] Handling call
0709 13:01:58.507779 (+1863712us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540012cad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:58.507947 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2522208000}
I20260709 13:01:58.523926  1536 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.511513  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:58.511224  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 44742.3us
I20260709 13:01:58.510842  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.545208  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.533725  6884 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:58.555603  1508 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:58.510629  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44430.5us
I20260709 13:01:58.531262  6900 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:01:58.535087  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:58.506839  6768 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:01:58.549288  1536 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:58.557380  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:58.558825  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 49393.6us
I20260709 13:01:58.560182  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:01:58.565976  2234 raft_consensus.cc:3055] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:58.566478  2234 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:58.557459  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:01:58.568567  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.563206  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:58.570463  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.570910  1611 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:01:58.574146  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:58.566282  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:58.574925  1536 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:01:58.545521  6635 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:58.567766  2234 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.578769  2234 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:58.579746  6900 consensus_queue.cc:1243] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:01:58.581002  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:58.585544  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:58.588025  1538 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:01:58.563331  6863 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:58.563609  6601 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:58.592219  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2427) took 1667 ms. Trace:
I20260709 13:01:58.593497  1611 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:01:58.595476  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:58.596298  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.597478  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.597947  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:58.598834  1692 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:58.598708  2234 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:01:58.602355  1692 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:01:58.611720  6763 raft_consensus.cc:3055] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:01:58.612185  6763 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:58.612962  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 982) took 1704 ms. Trace:
I20260709 13:01:58.613687  1537 rpcz_store.cc:276] 0709 13:01:56.908775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.012356 (+103581us) service_pool.cc:224] Handling call
0709 13:01:58.507953 (+1495597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.508984 (+  1031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.612705 (+103721us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540012cad0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:58.612889 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3438745472}
I20260709 13:01:58.510406  6810 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:58.622191  6810 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:58.581342  6768 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:01:58.606731  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.607674  6903 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:01:58.642550  6903 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:58.608254  6340 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:58.642990  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.592634  1614 rpcz_store.cc:276] 0709 13:01:56.920284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.477040 (+556756us) service_pool.cc:224] Handling call
0709 13:01:57.477344 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:57.552695 (+ 75351us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:01:57.564872 (+ 12177us) log.cc:844] Serialized 14 byte log entry
0709 13:01:57.854633 (+289761us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:57.854703 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:57.854765 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.587198 (+732433us) spinlock_profiling.cc:243] Waited 75.1 ms on lock 0x7b54002bdc54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:01:58.587244 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.587368 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21662,"prepare.queue_time_us":12820,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":42924,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":6957,"spinlock_wait_cycles":172825856,"thread_start_us":274657,"threads_started":2,"wal-append.queue_time_us":76308}
I20260709 13:01:58.643797  1616 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:58.644001  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.644556  1614 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:58.644775  1616 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:58.646514  1614 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:01:58.646643  1540 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:58.647821  1614 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:01:58.649151  1614 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:01:58.600401  1541 pending_rounds.cc:85] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 11
I20260709 13:01:58.649595  1541 pending_rounds.cc:107] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 13:01:58.649859  1541 raft_consensus.cc:2889] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:58.631958  1536 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:01:58.604532  1688 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 9. Current term is 10. Ops: []
I20260709 13:01:58.632885  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:58.651952  1617 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1918) took 2071 ms (client timeout 2317 ms). Trace:
W20260709 13:01:58.652694  1617 rpcz_store.cc:269] 0709 13:01:56.580711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.233563 (+652852us) service_pool.cc:224] Handling call
0709 13:01:58.651684 (+1418121us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b5400104d54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:58.651874 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1302906240}
I20260709 13:01:58.653322  1611 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:01:58.642978  6903 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:01:58.654768  1617 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:01:58.632310  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:58.658373  6864 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:01:58.658943  6864 raft_consensus.cc:697] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:01:58.659869  6864 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:58.662467  6869 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.887s	user 0.006s	sys 0.000s
W20260709 13:01:58.666605  6901 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:01:58.667426  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1219) took 1519 ms. Trace:
I20260709 13:01:58.667816  2235 rpcz_store.cc:276] 0709 13:01:57.148415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.148580 (+   165us) service_pool.cc:224] Handling call
0709 13:01:57.148949 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:57.173679 (+ 24730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.173717 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:57.581743 (+408026us) spinlock_profiling.cc:243] Waited 3.3 ms on lock 0x7b2c000cc068. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:01:57.584920 (+  3177us) log.cc:844] Serialized 26 byte log entry
0709 13:01:57.660521 (+ 75601us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:01:57.660577 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:57.661060 (+   483us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:58.661794 (+1000734us) spinlock_profiling.cc:243] Waited 40.1 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:01:58.667191 (+  5397us) raft_consensus.cc:1685] finished
0709 13:01:58.667212 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:58.667369 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15401,"raft.queue_time_us":15604,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":12711,"spinlock_wait_cycles":99737728,"thread_start_us":33900,"threads_started":1,"wal-append.queue_time_us":178817}
I20260709 13:01:58.610035  6863 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:58.683298  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.684348  6797 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.686084  6666 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:58.686393  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:58.723534  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 60628.1us
I20260709 13:01:58.725131  1692 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:58.730250  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:58.730566  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:58.643379  6340 raft_consensus.cc:697] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:58.732973  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:58.733484  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:01:58.734946  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:58.735822  1687 raft_consensus.cc:2393] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:58.736555  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:58.740429  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.741557  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.742756  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.730804  6832 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.743579  1611 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:01:58.743698  6832 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.744545  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.739189  6795 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 13:01:58.745757  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.746860  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.748164  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.748376  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.749841  1617 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 11: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:58.750274  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:58.613040  6763 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:58.751434  6671 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:58.752027  6768 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.746914  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.756536  1611 raft_consensus.cc:2410] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:01:58.748921  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.763559  1614 raft_consensus.cc:2393] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:58.757205  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:58.759759  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:01:58.760756  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.765101  6797 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:58.761560  1689 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:01:58.759196  2228 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:58.762511  1610 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:01:58.763762  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:01:58.771390  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.774050  1534 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:58.773998  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:01:58.777704  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.781808  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:01:58.781904  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.746025  6832 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:58.779729  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:58.787180  1683 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:01:58.787664  1683 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:58.783586  1614 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:01:58.786237  1509 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:58.783320  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.789561  1610 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:01:58.787168  1691 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:01:58.787112  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:58.790783  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.792059  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:58.794852  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:58.795812  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:58.759892  6681 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:01:58.780893  6799 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.000s
I20260709 13:01:58.772758  6874 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.772292  6870 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.802053  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:58.794485  1683 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.839747  1683 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:58.776619  6666 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.776531  6671 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 13:01:58.791101  6829 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:01:58.846304  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:58.791512  6762 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:01:58.853797  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:58.803494  6635 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:01:58.807906  6797 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:01:58.780484  6822 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 13:01:58.833302  6832 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:01:58.832998  6835 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:58.871225  6797 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:58.863930  1614 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 12: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:01:58.872229  6797 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.873574  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:58.739419  6863 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.874075  2201 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:01:58.856098  6388 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.628s	user 0.011s	sys 0.000s
I20260709 13:01:58.878180  6388 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.880995  6388 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:01:58.884198  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:58.870153  6832 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:58.853645  6675 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:58.883205  1614 raft_consensus.cc:2368] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15.
I20260709 13:01:58.844965  6874 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.889036  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:58.857583  6569 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:01:58.843415  6122 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:01:58.905081  6795 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:01:58.925611  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 133613us
I20260709 13:01:58.858112  6762 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:58.836431  6681 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:58.943017  1611 raft_consensus.cc:2376] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12. Re-sending same reply.
I20260709 13:01:58.902524  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:58.944892  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:58.944029  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68473us
I20260709 13:01:58.869045  6748 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:58.936287  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 202851us
I20260709 13:01:58.948032  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:58.952752  2203 leader_election.cc:336] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.040s (SENT)
I20260709 13:01:58.850081  6666 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.899355  6829 consensus_queue.cc:1059] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:01:58.937755  6569 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:01:58.849920  6870 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:58.936628  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:01:58.961074  1687 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:01:58.997931  1540 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:01:58.957772  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:01:58.964234  2203 leader_election.cc:336] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.176s (SENT)
I20260709 13:01:58.847584  6340 consensus_queue.cc:237] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:58.944003  6762 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:58.993139  1542 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:58.938652  6653 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:01:58.937832  6824 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:58.999208  1535 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:01:58.949980  6629 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:01:58.900329  6675 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:58.973106  6809 consensus_meta.cc:287] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.583s	user 0.008s	sys 0.003s
I20260709 13:01:59.026885  1683 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
I20260709 13:01:59.059973  1683 pending_rounds.cc:85] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 12
I20260709 13:01:59.060382  1683 pending_rounds.cc:107] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 13:01:59.060640  1683 raft_consensus.cc:2889] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:01:58.885648  6832 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:01:59.032634  1691 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
W20260709 13:01:59.038620  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 74427.4us
I20260709 13:01:58.951200  6877 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:59.049921  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 129719us
I20260709 13:01:58.952522  6900 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:01:59.054409  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 54388.3us
I20260709 13:01:59.071704  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:59.054973  6824 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:01:59.045182  6122 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.100876  1535 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:01:59.058270  6629 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:01:59.138055  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:01:59.138638  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 65057.6us
I20260709 13:01:59.139343  1582 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:59.059331  6635 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:01:59.059583  6748 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:01:59.145294  2201 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:01:59.204185  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 60849.6us
I20260709 13:01:59.107610  6824 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:59.243644  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:01:59.145100  6629 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:59.068153  6809 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:59.251645  1536 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:01:59.253402  1541 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.026s	sys 0.083s
I20260709 13:01:59.254858  1536 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:01:59.255055  1541 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.026s	sys 0.083s
I20260709 13:01:59.154165  6848 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:01:59.255287  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 50118us
I20260709 13:01:59.070240  6922 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.176s
I20260709 13:01:59.265945  6800 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:01:59.266609  6800 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:01:59.267532  6800 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:59.067319  6569 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:59.275178  6666 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:59.277058  6877 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:59.278873  6675 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:59.283515  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:59.203150  6601 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
W20260709 13:01:59.305145  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.147295  6122 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:01:59.313000  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:01:59.322271  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 385739us
W20260709 13:01:59.322808  1508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.003s	sys 0.002s
W20260709 13:01:59.325896  1508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.003s	sys 0.002s
W20260709 13:01:59.326272  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 526280us
I20260709 13:01:59.324667  1509 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:59.338421  6639 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:59.338056  6629 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:59.339745  6762 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:59.340236  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:01:59.340480  6629 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:01:59.341102  6629 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.347815  1609 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:01:59.309849  6848 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.343312  6122 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:59.325238  6877 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:01:59.341470  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:01:59.357622  1618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 1928) took 2492 ms (client timeout 1751 ms). Trace:
W20260709 13:01:59.359580  1618 rpcz_store.cc:269] 0709 13:01:56.843755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:56.911928 (+ 68173us) service_pool.cc:224] Handling call
0709 13:01:59.335408 (+2423480us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b54002dc3d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:01:59.336235 (+   827us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1254628224}
I20260709 13:01:59.362597  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:59.363255  1688 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:01:59.374604  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:01:59.396776  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.323455  6574 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:01:59.261441  6905 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:01:59.324704  6900 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.407450  1618 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:01:59.411860  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 273877us
I20260709 13:01:59.364162  1509 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:59.405670  6919 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.074s	user 0.001s	sys 0.002s
I20260709 13:01:59.374022  6681 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:59.417338  6916 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.418591  6940 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.430174  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:01:59.416669  6855 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:01:59.432221  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 68160.9us
I20260709 13:01:59.377445  6848 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:59.420662  6870 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:59.374370  6629 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:59.435663  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:59.425702  6900 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:59.454341  1616 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:01:59.437744  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:59.470643  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:01:59.454948  6836 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:01:59.475031  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:59.490864  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:59.497058  1687 raft_consensus.cc:2410] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:01:59.493196  1688 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:01:59.421046  6939 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:01:59.454241  6921 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.209s	user 0.000s	sys 0.000s
W20260709 13:01:59.506268  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 68616.7us
I20260709 13:01:59.422398  6574 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:59.495985  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:59.497912  1618 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:59.498018  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:59.500967  6938 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.501327  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:59.436029  6762 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:59.507464  1683 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:01:59.506567  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:59.433650  6675 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:01:59.497545  6748 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:59.508862  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:59.510655  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1000) took 1085 ms. Trace:
I20260709 13:01:59.511317  1539 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:01:59.436260  6855 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:59.499699  6795 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:01:59.453400  6840 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:01:59.507262  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:01:59.451911  6903 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.797s	user 0.009s	sys 0.000s
W20260709 13:01:59.530879  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 60315us
W20260709 13:01:59.649951  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 142784us
I20260709 13:01:59.650107  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:59.534049  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:01:59.451177  6848 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:59.452199  6941 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:01:59.558545  1541 rpcz_store.cc:276] 0709 13:01:58.425605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.582056 (+156451us) service_pool.cc:224] Handling call
0709 13:01:58.600188 (+ 18132us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:58.674967 (+ 74779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.675011 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:58.675584 (+   573us) log.cc:844] Serialized 52 byte log entry
0709 13:01:59.255568 (+579984us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:01:59.289933 (+ 34365us) log.cc:844] Serialized 14 byte log entry
0709 13:01:59.290296 (+   363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.290410 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:01:59.510368 (+219958us) raft_consensus.cc:1685] finished
0709 13:01:59.510396 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.510585 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24325,"prepare.queue_time_us":35378,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":26588,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":33744,"thread_start_us":579578,"threads_started":1,"wal-append.queue_time_us":749309}
I20260709 13:01:59.657320  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:59.651772  1690 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:01:59.518155  6809 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:59.674755  1535 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:01:59.677882  1538 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:01:59.678313  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:01:59.679139  1542 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:01:59.561724  6759 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:01:59.435256  6916 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:59.438411  6940 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:01:59.691867  1685 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:01:59.736133  6795 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:01:59.737272  6917 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:01:59.669836  6762 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:01:59.772651  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.834184  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:01:59.835067  1690 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:01:59.678859  6932 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:01:59.872588  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2450) took 1097 ms. Trace:
I20260709 13:01:59.874956  1618 rpcz_store.cc:276] 0709 13:01:58.775477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.735618 (+960141us) service_pool.cc:224] Handling call
0709 13:01:59.872066 (+136448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.872263 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.872293 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:01:59.872332 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:01:59.872361 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.872413 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:01:59.872533 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:59.690034  6759 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:01:59.678610  6877 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.151s
W20260709 13:01:59.869601  6340 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.007s	sys 0.020s
I20260709 13:01:59.562536  6938 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:01:59.889873  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:01:59.887511  6639 consensus_meta.cc:287] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.549s	user 0.010s	sys 0.000s
I20260709 13:01:59.678725  6939 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.207s
I20260709 13:01:59.902186  6914 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.218s
I20260709 13:01:59.879318  6759 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:01:59.912170  1542 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:01:59.919835  1618 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:01:59.918614  1685 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
W20260709 13:01:59.925199  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 267974us
W20260709 13:01:59.949412  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:01:59.990336  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2451) took 1213 ms. Trace:
I20260709 13:01:59.877785  6935 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:01:59.968616  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:01:59.991372  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:01:59.888279  6877 consensus_queue.cc:1243] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 8a6a5ae1b7f240408b3ddd30d8a18262 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:01:59.997956  1688 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:01:59.874658  6932 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:01:59.905328  6629 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:01:59.977839  1690 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:01:59.990726  1618 rpcz_store.cc:276] 0709 13:01:58.777256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.885568 (+1108312us) service_pool.cc:224] Handling call
0709 13:01:59.919463 (+ 33895us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.990005 (+ 70542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.990284 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:01:59.678232  6903 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.018836  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:01:59.968452  6672 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.031595  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.033178  1618 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:02:00.033707  1690 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:00.033865  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2452) took 1256 ms. Trace:
I20260709 13:02:00.034337  1618 rpcz_store.cc:276] 0709 13:01:58.777544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.032591 (+1255047us) service_pool.cc:224] Handling call
0709 13:02:00.032876 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.033626 (+   750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.033811 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:00.026645  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:00.026762  6932 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:00.048501  1612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 859) took 2096 ms (client timeout 1687 ms). Trace:
W20260709 13:02:00.048978  1612 rpcz_store.cc:269] 0709 13:01:57.951941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.952106 (+   165us) service_pool.cc:224] Handling call
0709 13:02:00.048285 (+2096179us) spinlock_profiling.cc:243] Waited 85 ms on lock 0x7b5400104354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:00.048431 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":195423104}
I20260709 13:02:00.054036  6941 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:02:00.056277  6924 log.cc:927] Time spent T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.993s	user 0.003s	sys 0.018s
I20260709 13:02:00.058758  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2453) took 1110 ms. Trace:
I20260709 13:02:00.060217  1616 rpcz_store.cc:276] 0709 13:01:58.947981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.341189 (+393208us) service_pool.cc:224] Handling call
0709 13:02:00.058470 (+717281us) spinlock_profiling.cc:243] Waited 94.8 ms on lock 0x7b54003c96d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:00.058669 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218124672}
I20260709 13:02:00.061313  1615 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:00.063153  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:00.063948  1538 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.064052  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 1946) took 2883 ms. Trace:
I20260709 13:02:00.064553  1609 rpcz_store.cc:276] 0709 13:01:57.180696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.500803 (+320107us) service_pool.cc:224] Handling call
0709 13:01:59.336348 (+1835545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.355831 (+ 19483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.063819 (+707988us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54002dc3d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:00.063988 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4220071168}
I20260709 13:02:00.030246  6917 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:02:00.068014  6903 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.067595  1536 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.068735  1692 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:00.068944  1536 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:00.022444  6601 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:00.058925  1612 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 13:02:00.029505  6810 consensus_meta.cc:287] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.407s	user 0.009s	sys 0.003s
I20260709 13:02:00.074870  6855 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:01:59.992434  6675 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:02:00.021915  6639 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:02:00.039535  6629 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:00.085623  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.030320  6795 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:02:00.097195  1538 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
W20260709 13:02:00.089519  6832 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.020s	sys 0.040s
I20260709 13:02:00.022322  6935 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.070680  6672 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:00.090158  6824 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.836s	user 0.006s	sys 0.005s
I20260709 13:02:00.109869  6949 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.083099  6900 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:00.082734  6809 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:00.111477  6832 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.020s	sys 0.040s
I20260709 13:02:00.106439  6629 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:00.110302  6574 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:00.131479  1684 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:00.141381  6950 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.136528  6629 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:00.142323  6950 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.145043  6950 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:00.147658  2228 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:00.149235  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.150112  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.150853  2234 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:00.151046  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:00.151790  2228 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:00.152074  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:00.150053  2233 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:02:00.152823  2237 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:00.152911  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:00.153788  2232 raft_consensus.cc:2410] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:02:00.154659  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.155733  2235 raft_consensus.cc:2410] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 13:02:00.156682  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 486201us
I20260709 13:02:00.121068  6949 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.131371  6877 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:00.188426  2237 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:00.130040  6809 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:00.124559  6824 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:00.216012  2228 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:00.171805  6916 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:00.190080  6762 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:00.175657  6940 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:00.185503  6950 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:00.225095  1614 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 888) took 1455 ms (client timeout 1924 ms). Trace:
W20260709 13:02:00.225735  1614 rpcz_store.cc:269] 0709 13:01:58.769693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.873189 (+103496us) service_pool.cc:224] Handling call
0709 13:02:00.224804 (+1351615us) spinlock_profiling.cc:243] Waited 8.63 ms on lock 0x7b540008c0d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:00.225009 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19842304}
I20260709 13:02:00.217712  6938 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.239444  2229 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:00.187629  6877 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.236828  6941 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:00.256834  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2421) took 3564 ms. Trace:
I20260709 13:02:00.245074  6810 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:00.237453  6762 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.257485  1613 rpcz_store.cc:276] 0709 13:01:56.693121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.458074 (+764953us) service_pool.cc:224] Handling call
0709 13:01:57.460236 (+  2162us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:01:57.460644 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:57.460674 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:01:57.464071 (+  3397us) log.cc:844] Serialized 78 byte log entry
0709 13:01:57.574194 (+110123us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:01:59.920862 (+2346668us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b1800208360. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fc455 00007fe0934e43c7 00007fe08faf69a7 00007fe08fb060bf 00007fe08fb06038
0709 13:02:00.103509 (+182647us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.245444 (+141935us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.249744 (+  4300us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.251067 (+  1323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.256499 (+  5432us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.256561 (+    62us) raft_consensus.cc:1685] finished
0709 13:02:00.256579 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.256759 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8536,"prepare.queue_time_us":12291,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":129,"raft.queue_time_us":102020,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":13074,"spinlock_wait_cycles":10269696,"thread_start_us":399328,"threads_started":3,"wal-append.queue_time_us":162432,"wal-append.run_cpu_time_us":7297,"wal-append.run_wall_time_us":1076834}
I20260709 13:02:00.240195  6950 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.252192  6305 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:00.290386  6305 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:00.295405  6935 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:00.296334  6868 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 2.522s	user 0.015s	sys 0.021s
I20260709 13:02:00.299010  6938 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:00.299810  6938 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.302791  6949 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:00.309209  6941 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.314627  6824 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:00.319669  6950 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:00.323259  6877 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:00.267735  6810 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.328543  6810 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.331748  6762 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:00.336953  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.337702  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.394263  6932 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.396409  6938 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.396970  6932 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:00.389129  6877 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:00.398118  6877 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:00.394975  6748 consensus_meta.cc:287] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.763s	user 0.010s	sys 0.000s
W20260709 13:02:00.406176  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 188354us
I20260709 13:02:00.407301  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:00.408457  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.407106  6748 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:00.406903  6762 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 13:02:00.413887  6759 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.415143  1542 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:00.418761  6853 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:00.418848  6556 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:00.420264  6913 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.479s	user 0.000s	sys 0.016s
I20260709 13:02:00.420635  6556 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:00.421132  6877 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:00.423070  6556 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:00.419482  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2791) took 3150 ms. Trace:
I20260709 13:02:00.423800  1684 rpcz_store.cc:276] 0709 13:01:57.268134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.270457 (+  2323us) service_pool.cc:224] Handling call
0709 13:01:57.281481 (+ 11024us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.133187 (+2851706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.418141 (+284954us) spinlock_profiling.cc:243] Waited 2.85 s on lock 0x7b5400028354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:00.418385 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6553492736}
I20260709 13:02:00.425277  6853 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.428762  6853 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.431483  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.433938  6877 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:00.434602  6877 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:00.435513  6877 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.441165  6671 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.441905  6671 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.444931  6671 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.445779  6938 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.446355  6938 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.448665  6938 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.449905  6759 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.451812  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 893) took 1674 ms. Trace:
I20260709 13:02:00.452225  1612 rpcz_store.cc:276] 0709 13:01:58.777829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.049946 (+1272117us) service_pool.cc:224] Handling call
0709 13:02:00.058636 (+  8690us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:00.072816 (+ 14180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.451626 (+378810us) spinlock_profiling.cc:243] Waited 8.32 ms on lock 0x7b54003c96d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:00.451757 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19134848}
I20260709 13:02:00.453390  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:00.454035  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 993) took 1900 ms. Trace:
I20260709 13:02:00.456193  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.456589  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:00.454490  2229 rpcz_store.cc:276] 0709 13:01:58.553367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.555268 (+  1901us) service_pool.cc:224] Handling call
0709 13:01:58.555757 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.241648 (+1685891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.453753 (+212105us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b54002a3fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:00.453966 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3871631744}
I20260709 13:02:00.456925  1613 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:00.459381  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:00.460326  1543 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:00.464306  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:00.465255  1537 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 11.
I20260709 13:02:00.469129  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.468940  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:00.470222  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:00.472513  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.490291  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:00.491429  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:00.493686  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.494357  1615 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:00.496219  1582 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:00.502261  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:00.491739  6872 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:00.506270  2203 leader_election.cc:336] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.823s (SENT)
I20260709 13:02:00.495488  6853 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:00.499111  6574 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:00.514564  6906 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.649s	user 0.005s	sys 0.000s
I20260709 13:02:00.506947  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:00.529842  2203 leader_election.cc:336] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.924s (SENT)
I20260709 13:02:00.530624  2203 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:00.506682  6962 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.537971  6799 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:00.528878  6872 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.010s
I20260709 13:02:00.539436  6872 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:00.539855  6872 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:00.540732  6872 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.541929  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1398) took 2076 ms. Trace:
I20260709 13:02:00.542053  6872 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:00.536922  6962 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.542474  2236 rpcz_store.cc:276] 0709 13:01:58.466023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.508589 (+ 42566us) service_pool.cc:224] Handling call
0709 13:01:58.508946 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.564753 (+ 55807us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:01:58.581650 (+ 16897us) log.cc:844] Serialized 14 byte log entry
0709 13:01:58.963229 (+381579us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:58.987085 (+ 23856us) log.cc:844] Serialized 26 byte log entry
0709 13:01:58.987253 (+   168us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:01:58.987288 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:58.987357 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.513954 (+1526597us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:00.537343 (+ 23389us) raft_consensus.cc:1685] finished
0709 13:02:00.541726 (+  4383us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.541873 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95541,"prepare.queue_time_us":20906,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":162200,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":8738,"spinlock_wait_cycles":901767936,"thread_start_us":295661,"threads_started":2,"wal-append.queue_time_us":200126}
I20260709 13:02:00.545377  6962 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:00.537267  1582 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:00.534725  6941 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:00.550904  6822 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.558689  6762 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:00.558472  6822 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:00.559228  6762 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:00.560679  6822 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:00.562906  6874 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.551636  6748 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:00.563591  6574 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:00.538226  6853 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:02:00.564062  6574 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.564459  6574 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:00.563099  6799 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:00.563766  6748 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.571543  6798 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.572619  6748 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.579761  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1088) took 1843 ms. Trace:
I20260709 13:02:00.580255  1689 rpcz_store.cc:276] 0709 13:01:58.736153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.736324 (+   171us) service_pool.cc:224] Handling call
0709 13:01:58.736715 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:58.774468 (+ 37753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.579458 (+1804990us) spinlock_profiling.cc:243] Waited 24.5 ms on lock 0x7b540028f0d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:00.579684 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56428672}
I20260709 13:02:00.561254  6762 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:00.587175  6812 raft_consensus.cc:3055] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:00.587682  6812 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:00.596453  6305 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:00.597793  6912 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.555s	user 0.004s	sys 0.000s
W20260709 13:02:00.600154  6926 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.445s	user 0.002s	sys 0.020s
I20260709 13:02:00.577247  6763 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:00.572646  6798 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:00.601774  6929 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.240s	user 0.002s	sys 0.010s
I20260709 13:02:00.602223  6574 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.602938  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1003) took 1649 ms. Trace:
I20260709 13:02:00.603330  1534 rpcz_store.cc:276] 0709 13:01:58.953879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.955732 (+  1853us) service_pool.cc:224] Handling call
0709 13:01:58.956145 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:01:58.956655 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:58.956696 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:01:58.957278 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 13:01:59.058911 (+101633us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:01:59.058954 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:01:59.059025 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.601145 (+1542120us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:00.602634 (+  1489us) raft_consensus.cc:1685] finished
0709 13:02:00.602654 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.602863 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8913,"raft.queue_time_us":72468,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":4903,"spinlock_wait_cycles":72564480,"thread_start_us":54594,"threads_started":1,"wal-append.queue_time_us":291186}
I20260709 13:02:00.604646  6574 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.610291  1534 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:00.610985  1538 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:00.610384  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:00.612367  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:00.613276  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.613646  1537 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:00.614606  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:00.611699  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:00.621867  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:00.622754  1538 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:00.622982  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:00.588680  6812 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:00.625012  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.625675  1610 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:02:00.628322  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.605295  6574 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.632017  1582 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:00.629015  1614 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:00.629586  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:00.624073  6556 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:00.630995  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.631386  6853 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:00.626317  6305 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:00.641204  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:00.626926  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:00.644299  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:00.646441  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:00.653043  6799 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:00.639017  6556 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.653594  6799 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:00.654075  6799 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:00.655697  6556 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.656040  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.656519  6556 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:00.656425  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.657387  6550 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:00.658007  6550 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:00.657295  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:00.658607  6550 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:00.658830  1609 raft_consensus.cc:2376] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11. Re-sending same reply.
I20260709 13:02:00.659574  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1005) took 1705 ms. Trace:
I20260709 13:02:00.659981  1536 rpcz_store.cc:276] 0709 13:01:58.954580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.958278 (+  3698us) service_pool.cc:224] Handling call
0709 13:01:58.958662 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.067181 (+1108519us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b540038dc20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa6a06f 00007fe08fae2f60 00007fe08fae47de 00007fe08faeeb49 00007fe08faef416 00007fe08faebd8f 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62
0709 13:02:00.086173 (+ 18992us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:00.086217 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:00.086276 (+    59us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:00.086331 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.659303 (+572972us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b540012ded4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:00.659356 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.659514 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2219,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":14978,"spinlock_wait_cycles":685712000}
I20260709 13:02:00.657155  1614 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:00.662132  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:49352 (request call id 32) took 2796 ms. Trace:
I20260709 13:02:00.662557  1686 rpcz_store.cc:276] 0709 13:01:57.865890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:57.866237 (+   347us) service_pool.cc:224] Handling call
0709 13:02:00.661918 (+2795681us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b540028e6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fae6853 00007fe094ab460a 00007fe08e03a7d4 00007fe08e03a762 00007fe08e03a691 00007fe08e03a60c 00007fe08e0398c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:00.662042 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1444657024}
I20260709 13:02:00.664487  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2435) took 1894 ms. Trace:
I20260709 13:02:00.657276  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.664953  1539 rpcz_store.cc:276] 0709 13:01:58.769914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.770075 (+   161us) service_pool.cc:224] Handling call
0709 13:01:59.511081 (+741006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:01:59.562445 (+ 51364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.664259 (+1101814us) spinlock_profiling.cc:243] Waited 726 ms on lock 0x7b54004182d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:00.664423 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1670928128}
I20260709 13:02:00.665714  1616 raft_consensus.cc:2376] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11. Re-sending same reply.
I20260709 13:02:00.655732  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:00.667309  1612 raft_consensus.cc:2376] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17. Re-sending same reply.
I20260709 13:02:00.673738  6629 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:00.674350  1612 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:02:00.634583  6829 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:02:00.674448  6629 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.633466  6833 consensus_queue.cc:1059] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:02:00.675680  6829 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:00.676083  6829 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:00.677743  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1918) took 1939 ms. Trace:
I20260709 13:02:00.678166  1692 rpcz_store.cc:276] 0709 13:01:58.737912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.739246 (+  1334us) service_pool.cc:224] Handling call
0709 13:01:58.739556 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.069178 (+1329622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.677509 (+608331us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54002ce054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:00.677682 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3056329600}
I20260709 13:02:00.645546  6853 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:00.679431  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.680109  6556 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:00.679956  6853 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.680207  1612 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
W20260709 13:02:00.681301  6915 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.431s	user 0.015s	sys 0.007s
I20260709 13:02:00.636741  6935 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:02:00.673898   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:00.637018  6574 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.683185  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:00.694053  1536 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:00.676808  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2437) took 1005 ms. Trace:
I20260709 13:02:00.689639  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:00.676993  6629 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.648094  6305 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.708304  2230 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:02:00.676954  6829 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:00.720974  6574 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.724329  1615 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:00.694772  1541 rpcz_store.cc:276] 0709 13:01:59.671191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.672315 (+  1124us) service_pool.cc:224] Handling call
0709 13:01:59.672751 (+   436us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:01:59.872679 (+199928us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:01:59.872707 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:01:59.873285 (+   578us) log.cc:844] Serialized 52 byte log entry
0709 13:02:00.251467 (+378182us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:00.251523 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:00.676383 (+424860us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54003e2954. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:00.676443 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:00.676496 (+    53us) raft_consensus.cc:1685] finished
0709 13:02:00.676509 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:00.676746 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":140,"raft.queue_time_us":1498,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":6968,"spinlock_wait_cycles":458894208,"thread_start_us":281130,"threads_started":1,"wal-append.queue_time_us":421514}
W20260709 13:02:00.733357  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 162682us
W20260709 13:02:00.755738  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.758721  1692 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:02:00.760336  1689 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:00.765861  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.766841  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:00.767905  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:00.769276  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 144321us
I20260709 13:02:00.775804  1618 pending_rounds.cc:85] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 9
I20260709 13:02:00.776165  1618 pending_rounds.cc:107] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:02:00.776417  1618 raft_consensus.cc:2889] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:00.776726  1618 pending_rounds.cc:107] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:02:00.776983  1618 raft_consensus.cc:2889] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:00.777137  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.776741  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.778023  1691 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:02:00.779271  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.784787  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.785360  1541 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:02:00.785604  1683 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:00.784809  1614 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:00.786528  1614 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:00.781214  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.788766  1615 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:00.796027  1536 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:02:00.798079  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:00.798943  1616 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:00.799917  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.801286  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.802435  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.802850  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:00.803268  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:00.803632  1688 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:00.804965  1692 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:00.808060  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.808748  1690 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:00.809710  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.752307  6764 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:00.813702  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.815899  1689 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:00.804097  1614 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.817443  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:00.681209  6556 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.784430  6748 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:00.829948  1688 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:00.831406  1582 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:00.827070  1614 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.784198  6874 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:02:00.821395  6829 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.823110  6798 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:00.805672  6822 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 13:02:00.821759  6764 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:00.860035  6305 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:00.857125  6762 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:00.847069  6748 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:00.864274  1616 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
W20260709 13:02:00.876886  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.877750  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:02:00.853355  6864 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.847s	user 0.028s	sys 0.048s
I20260709 13:02:00.883210  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:00.881630  6864 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.879s	user 0.029s	sys 0.049s
I20260709 13:02:00.872198  6574 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:00.866623  6935 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:00.873342  6762 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:00.882231  6877 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:02:00.891862  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 84781.6us
W20260709 13:02:00.893627  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:00.927511  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.927798  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1399) took 1850 ms. Trace:
I20260709 13:02:00.894639  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.899191  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:00.928251  2230 rpcz_store.cc:276] 0709 13:01:59.077679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.077826 (+   147us) service_pool.cc:224] Handling call
0709 13:02:00.927627 (+1849801us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400530054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:00.927720 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3365958400}
I20260709 13:02:00.901618  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.902024  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.883694  6799 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.904336  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.904350  1616 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:00.912883  2236 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:02:00.948822  2236 pending_rounds.cc:85] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 10
I20260709 13:02:00.949147  2236 pending_rounds.cc:107] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:02:00.949445  2236 raft_consensus.cc:2889] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:00.912120  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.949458  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:00.950417  2229 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:00.893944  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.929075  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.946589  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:00.954311  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.956962  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:00.926729  6762 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.958734  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:00.963688  1534 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:02:00.963955  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.964143  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:00.965330  2231 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:02:00.987520  2232 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:00.966070  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.967968  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.905215  6629 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:00.968887  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:01.006354  2237 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:01.006899  1657 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:00.912120  6876 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:00.971859  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.961144  6748 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:00.946527  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:01.020819  1687 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:00.976550  6796 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:00.977042  6977 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:00.947111  6574 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:00.982096  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:00.948426  6935 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:00.984952  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:01.046131  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:01.049131  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:00.983687  6671 raft_consensus.cc:3055] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:00.991245  1616 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
W20260709 13:02:00.936757  6797 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.059s	user 0.008s	sys 0.018s
I20260709 13:02:01.012159  6763 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:02:01.010221  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 56818.9us
I20260709 13:02:01.021364  1657 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:01.030975  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:01.037477  2201 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:01.040614  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:01.040879  2232 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:01.042228  2229 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:02:00.978338   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:01.054754  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:01.069737  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:01.070652  1614 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:01.042529  6748 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.012246  6764 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:01.005635  6759 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:01.068553  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47282.5us
W20260709 13:02:01.056641  6797 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.179s	user 0.009s	sys 0.018s
I20260709 13:02:01.071913  1614 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
W20260709 13:02:00.972607  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 151536us
I20260709 13:02:01.027966  6796 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:01.110773  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:01.111110  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:01.061626  1508 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:01.021806  6629 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:01.081409  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 132082us
I20260709 13:02:01.095763  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:01.095774  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:01.096828  1657 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:01.099320  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 57377.2us
I20260709 13:02:01.058137  6671 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:01.021955  6876 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.041719  6833 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:01.102676  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:01.094868  6748 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:01.108662  2234 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:01.109575  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:01.056985  6976 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:02:01.078881  6810 raft_consensus.cc:2764] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:02:01.081085  6992 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.047274  6996 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 13:02:01.275056  6993 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:01.279472  6938 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:02:01.127602  6979 log.cc:927] Time spent T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:02:01.279733  6993 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:01.280587  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 183839us
W20260709 13:02:01.248312  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 186788us
W20260709 13:02:01.264330  6909 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 2.311s	user 0.000s	sys 0.012s
I20260709 13:02:01.281555  1657 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:01.282541  6938 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:01.266340  2230 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:02:01.283432  6938 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:01.283910  1540 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:02:01.284294  1540 pending_rounds.cc:85] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 9
I20260709 13:02:01.284233  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.284803  1540 pending_rounds.cc:107] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:02:01.285146  1540 raft_consensus.cc:2889] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:01.284432  6938 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.273591  6798 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:01.260926  6601 raft_consensus.cc:3055] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:01.239488  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:01.286284  6601 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:01.284545  2230 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 13:02:01.287050  2233 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:02:01.287156  6601 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.288290  6601 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.288579  6993 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:01.288882  2231 raft_consensus.cc:2410] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:02:01.253789  6812 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:01.126677  6978 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:02:01.097949  6759 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:01.127283  6975 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:02:01.238608  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:01.228767  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:01.292222  2237 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:01.200058  6653 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:01.285923  6798 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.293496  6653 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:01.185525  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.243743  6768 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:01.294682  6768 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.295115  6798 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:01.231145  6910 log.cc:927] Time spent T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 2.289s	user 0.008s	sys 0.003s
I20260709 13:02:01.298177  6768 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:01.112041  6796 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:01.308637  6833 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.309195  6855 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:01.159147  6799 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:01.156462  6762 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:01.309933  6855 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:01.169375  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:01.310441  6855 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:01.126112  6971 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:02:01.208824  1686 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:01.171022  6809 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:01.136826  6550 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:01.313584  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 131822us
I20260709 13:02:01.293221  6992 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:01.314927  6550 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:01.317081  6748 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:01.242703  6840 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:01.317936  6840 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.300347  1684 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:02:01.318368  6840 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.301960  2228 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:01.131043  6574 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.180735  1691 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:02:01.135346  1611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 886) took 2366 ms (client timeout 1823 ms). Trace:
I20260709 13:02:01.135305  6838 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.187s
W20260709 13:02:01.322062  1611 rpcz_store.cc:269] 0709 13:01:58.769113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:58.790524 (+ 21411us) service_pool.cc:224] Handling call
0709 13:02:01.135162 (+2344638us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b5400087ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:01.135277 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":348545920}
I20260709 13:02:01.303479  6671 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.181537  6824 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.142s
I20260709 13:02:01.323920  6671 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:01.324332  6796 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.113341  1687 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:01.132787  6935 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:01.117796  2203 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:01.120215  6947 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.010s	user 0.004s	sys 0.006s
I20260709 13:02:01.110864  6962 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:01.319096  6992 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:01.331148  6762 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:01.134681  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2449) took 2363 ms. Trace:
I20260709 13:02:01.333329  6796 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:01.124886  6672 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:01.191064  6872 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.648s	user 0.010s	sys 0.001s
I20260709 13:02:01.096199  6877 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:01.130983  6629 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:01.098424  6763 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:01.333613  1617 rpcz_store.cc:276] 0709 13:01:58.771158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:59.679100 (+907942us) service_pool.cc:224] Handling call
0709 13:01:59.679392 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:00.410355 (+730963us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:00.410688 (+   333us) log.cc:844] Serialized 14 byte log entry
0709 13:02:00.466232 (+ 55544us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:00.466313 (+    81us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:00.466386 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.133783 (+667397us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b18001eb2c0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9cd2a6 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419
0709 13:02:01.134453 (+   670us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b5400105754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:01.134498 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:01.134620 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18715,"prepare.queue_time_us":627,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":292,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":2741,"spinlock_wait_cycles":1762120192,"thread_start_us":54661,"threads_started":1,"wal-append.queue_time_us":97567}
I20260709 13:02:01.336766  6877 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.330760  2236 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:02:01.337531  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 56038.8us
I20260709 13:02:01.336678  6763 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:01.340358  6763 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:01.340502  6976 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:02:01.341444  6863 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.464s	user 0.008s	sys 0.019s
W20260709 13:02:01.341816  6863 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.464s	user 0.008s	sys 0.019s
I20260709 13:02:01.309216  6833 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:01.342878  2231 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:01.333228  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 223734us
W20260709 13:02:01.345577  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 49504.2us
I20260709 13:02:01.347157  6992 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:01.347689  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:01.348706  6666 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:01.349071  6666 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:01.349171  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:01.349944  6666 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.351014  6666 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:01.351434  1583 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:01.345578  2237 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:01.353202  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:01.353734  6835 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:01.354627  6859 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:01.355370  6859 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:01.357162  6569 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:01.357720  6569 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:01.358637  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 240917us
I20260709 13:02:01.358570  6569 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:01.359992  6812 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:01.365998  6798 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:01.366582  6798 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.367004  6798 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:01.367692  6855 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:01.369992  1534 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:01.371600  1537 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:01.373608  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:01.374183  1537 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:01.376766  1535 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:01.377180  1538 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:01.377193  6763 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.379149  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.381683  6962 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.383167  1537 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:02:01.382385  6962 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:01.387470  6796 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:01.388135  1539 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:01.388067  6796 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:01.389436  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.389736  6763 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:01.390275  6796 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:01.391124  1538 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:02:01.391400  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.393070  1538 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:01.394531  1439 catalog_manager.cc:5697] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 3 to 8, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 8 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.396476  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.397607  1535 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:01.395190  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.398744  1538 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
W20260709 13:02:01.401774  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55516.5us
I20260709 13:02:01.402199  1541 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:01.402997  1439 catalog_manager.cc:5697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:01.407573  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 69274.2us
I20260709 13:02:01.410288  1657 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:01.411712  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.412565  1542 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:01.421594  6855 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:01.360414  6812 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.422978  6812 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:01.434077  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 46556.3us
I20260709 13:02:01.435727  6797 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:01.428975  6759 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:01.438266  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:01.302547  6876 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:01.445914  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.440351  1439 catalog_manager.cc:5697] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 1 to 8. New cstate: current_term: 8 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:02:01.442140  6797 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:01.435122  6796 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:01.442579  6768 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:01.448892  1508 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:01.448939  1688 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:01.450858  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.457690  1541 raft_consensus.cc:2410] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:02:01.458552  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.446985  1542 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:01.460829  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.461537  1534 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:02:01.435868  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:01.463801  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.465214  1535 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:01.461549  1542 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:01.466715  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 56564.9us
I20260709 13:02:01.467798  1657 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:01.470001  1508 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:01.458484  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 108432us
I20260709 13:02:01.470712  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:01.471138  1584 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:01.474014  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:01.494740  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:01.495317  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.500299  2201 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:01.502815  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 67030.2us
I20260709 13:02:01.503391  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:01.503799  1683 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:02:01.502696  1657 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:01.505084  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.507594  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:01.507916  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:01.510071  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:01.510912  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:01.512405  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:01.513939  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:01.514501  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:01.508267  1687 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
W20260709 13:02:01.428996  1436 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55374 (request call id 323) took 37376 ms (client timeout 15000 ms). Trace:
I20260709 13:02:01.517447  1657 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:01.471828  6759 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:01.484014  6938 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:01.519965  1657 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:01.517704  2234 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:01.520880  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:01.521627  6675 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:01.484109  6574 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:01.483745  6748 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:01.525411  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:01.526960  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:01.530665  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:01.530763  1507 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:01.544492  6556 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 13:02:01.551388  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:01.516976  7020 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:01.553579  6863 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.554626  1542 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:01.548034  6809 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 13:02:01.554864  6863 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:01.544023  6675 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.556732  6675 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:01.557022  6863 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:01.557276  6653 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.552840  1507 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:01.554291  6937 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:01.538746  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:01.559432  1617 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:02:01.539783  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:01.560463  1614 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:01.540688  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:01.541060  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:01.561326  1613 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:02:01.561444  1610 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:01.562273  6762 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:01.562155  6833 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.563752  1507 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:01.541301  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:01.544080  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.506610  6860 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:02:01.565529  6666 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:01.566088  6666 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:01.506934  6993 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:01.566883  6993 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:01.567019  6666 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:01.502674  6853 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:01.568643  1616 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:01.532601  1538 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:01.571352  1507 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:01.572412  6812 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:01.573016  6763 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:01.573530  6763 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:01.558406  1692 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:01.575865  6763 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.515810  6877 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:01.515611  6859 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:01.579152  6768 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:01.578956  6859 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:01.567817  6993 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:01.585602  6991 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:02:01.559954  6653 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:01.573654  1683 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:01.594476  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:01.547375  1539 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:01.550453  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:01.578995  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 52170.6us
I20260709 13:02:01.552425  1439 catalog_manager.cc:5697] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 7 to 11, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 11 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:01.596829  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 46457.8us
I20260709 13:02:01.538641  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.546481  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:01.527678  6829 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:01.598618  1540 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:01.606392  1616 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:01.611235  1610 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:01.618283  1584 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:01.619467  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 77268us
I20260709 13:02:01.629230  1439 catalog_manager.cc:5697] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 5 to 9, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 9 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.564922  6833 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:01.544727  6996 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:01.603906  6929 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:02:01.628782  6796 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:02:01.577140  6798 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.544026  6992 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:01.608364  6936 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:01.594358  6759 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:01.644446  1540 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:01.608349  7014 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 13:02:01.646514  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:02:01.650082  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 52199.4us
I20260709 13:02:01.650195  7006 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:01.710121  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 63695.6us
I20260709 13:02:01.711344  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:01.651130  6812 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:01.652386  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:01.720148  1692 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:01.651165  6550 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
W20260709 13:02:01.519375  1436 rpcz_store.cc:269] 0709 13:01:24.052981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:24.064913 (+ 11932us) service_pool.cc:224] Handling call
0709 13:02:01.428841 (+37363928us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:01:53.979277 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:01:53.980028 (+   751us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:01:53.980092 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:01:53.980111 (+    19us) tablet.cc:662] Decoding operations
0709 13:01:53.996489 (+ 16378us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:01:54.018133 (+ 21644us) write_op.cc:641] Partition lock acquired for write op
0709 13:01:54.020777 (+  2644us) tablet.cc:685] Acquiring locks for 14 operations
0709 13:01:54.041490 (+ 20713us) tablet.cc:701] Row locks acquired
0709 13:01:54.041537 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:01:54.041728 (+   191us) write_op.cc:270] Start()
0709 13:01:54.041892 (+   164us) write_op.cc:276] Timestamp: P: 1783602114041666 usec, L: 0
0709 13:01:54.041921 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:01:54.042621 (+   700us) log.cc:844] Serialized 5619 byte log entry
0709 13:01:54.702430 (+659809us) op_driver.cc:464] REPLICATION: finished
0709 13:01:55.064897 (+362467us) write_op.cc:301] APPLY: starting
0709 13:01:55.065007 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 13:01:55.363410 (+298403us) tablet.cc:1370] finished BulkCheckPresence
0709 13:01:55.363576 (+   166us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:01:55.392156 (+ 28580us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:01:55.392867 (+   711us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=28,mrs_lookups=0
0709 13:01:55.392934 (+    67us) write_op.cc:312] APPLY: finished
0709 13:01:55.394875 (+  1941us) log.cc:844] Serialized 129 byte log entry
0709 13:01:55.395162 (+   287us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:55.395386 (+   224us) write_op.cc:454] Released schema lock
0709 13:01:55.395470 (+    84us) write_op.cc:341] FINISH: Updating metrics
0709 13:01:55.404424 (+  8954us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:01:55.404456 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":928584,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":358840,"apply.run_cpu_time_us":24065,"apply.run_wall_time_us":339496,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62731,"num_ops":14,"prepare.queue_time_us":89350,"prepare.run_cpu_time_us":23230,"prepare.run_wall_time_us":119860,"raft.queue_time_us":534689,"raft.run_cpu_time_us":39448,"raft.run_wall_time_us":271114,"replication_time_us":639350,"thread_start_us":567135,"threads_started":4,"wal-append.queue_time_us":76681,"wal-append.run_cpu_time_us":38564,"wal-append.run_wall_time_us":2277973}]]}
I20260709 13:02:01.655596  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:01.656486  1508 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:01.658625  6889 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:01.661679  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:01.648713  6859 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:01.667121  1691 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:02:01.666782  6675 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.673668  1684 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:02:01.674487  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:01.675859  1509 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:01.684463  2230 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:02:01.629043  6965 log.cc:927] Time spent T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:02:01.689606  1439 catalog_manager.cc:5697] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 8 to 9, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 9 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:01.636458  6976 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 13:02:01.608459  6829 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.628477  6937 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:01.608165  6635 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:01.703955  2203 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:01.707749  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 159044us
I20260709 13:02:01.728207  1584 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:01.691094  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:01.691012  6768 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:02:01.749174  1610 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
W20260709 13:02:01.737546  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 61778.1us
I20260709 13:02:01.750903  1509 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:01.697624  6936 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:01.643610  6574 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 13:02:01.721371  6809 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:01.757228  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:01.752620  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:01.734045  6874 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:01.773023  6996 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:02:01.777580  6798 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:01.747598  6937 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:01.778973  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 75120us
I20260709 13:02:01.779225  6937 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:01.780046  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:01.781962  6956 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:01.782431  6956 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.782802  6956 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:01.785560  6768 consensus_queue.cc:1243] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:02:01.738942  6762 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:01.704159  6992 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:01.786265  6838 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:01.788074  6992 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:01.788652  6992 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:01.789045  6992 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:01.789359  6796 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:01.742895  6556 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 13:02:01.785756  6994 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:02:01.754657  6936 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:01.755189  6759 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:01.796492  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 43999.4us
I20260709 13:02:01.797888  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:01.800151  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:01.800783  1613 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:01.725683  6990 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:01.737757  6859 raft_consensus.cc:3055] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:01.743146  6829 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.800679  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:01.762714  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:01.808768  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:01.825167  1543 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: []
W20260709 13:02:01.847513  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 119406us
W20260709 13:02:01.814971  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 52373.9us
I20260709 13:02:01.849607  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:01.850633  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44556.4us
I20260709 13:02:01.851238  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:01.853060  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:01.759310  6835 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:02:01.786345  6762 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:01.790807  6992 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.862771  1543 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:01.803771  6759 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:01.811116  6889 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:01.871002  1613 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:01.833455  6569 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:01.803225  6990 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:01.812225  6838 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:01.842743  6996 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:01.811817  6859 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:01.857764  6762 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:01.851346  1658 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:01.907934  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 56654.1us
I20260709 13:02:01.759574  6635 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:01.795015  6877 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:02:01.883630  6759 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:01.882694  6840 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.001s
I20260709 13:02:01.803603  6798 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:02:01.858443  6853 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:01.854351  6874 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:02:01.888191  6889 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:01.906487  6768 raft_consensus.cc:2764] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:02:01.902495  6990 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:01.917301  6962 consensus_meta.cc:287] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.001s
I20260709 13:02:01.916764  6923 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:01.914552  6938 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:02.022384  1610 raft_consensus.cc:3055] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:02.022966  1610 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:01.983821  6671 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:02.023950  1610 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:02.044862  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 166163us
I20260709 13:02:02.046059  1610 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:02.016886  6675 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:02.022315  6938 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:02.057344  1610 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:02.042562  6671 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:01.977030  6840 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.038789  7035 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:02:02.065529  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 214350us
I20260709 13:02:02.079879  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:02.019556  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 110164us
W20260709 13:02:02.050598  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 190839us
I20260709 13:02:02.081584  1584 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:02.081789  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 425360us
W20260709 13:02:02.060153  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 210624us
I20260709 13:02:02.082654  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:02.045003  6800 raft_consensus.cc:3055] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:02.022114  6962 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:02.083089  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.086233   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:02.099917  6574 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:02.102036  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:02.105475  1536 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:02.091539  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:02.058911  6938 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:02.015712  6859 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.061285  6764 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.102910  6768 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:02.123045  2232 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:02.056082  6763 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.091009  6800 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:02.121465  2203 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:02.136648  1584 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:02.123031  6879 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:02.186498  2229 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:02.105919  6838 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:02:02.187173  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:02.120127  6569 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:02.055374  6889 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.232507  6763 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:02.234385  6840 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:02.125394  6764 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.213761  6810 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:02.183887  6824 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 13:02:02.197149  6569 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:02.246784  6810 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.269151  6810 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:02.190677  6671 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.278556  6962 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:02.279474  7031 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
W20260709 13:02:02.280117  6973 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
W20260709 13:02:02.257051  7033 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 13:02:02.257331  7036 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:02:02.258649  6840 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:02.259725  7049 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:02.361387  7049 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.210675  6653 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 13:02:02.261489  7026 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:02:02.261590  7037 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:02.263042  7006 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
I20260709 13:02:02.247017  6879 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:02:02.265868  7030 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 13:02:02.267432  6937 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.213941  6800 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.223711  6574 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 13:02:02.364913  7049 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:02.313627  7029 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:02:02.435380  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:02.318046  6824 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:02.319818  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:02.320896  6764 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:02.322029  7042 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:02.237970  6863 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:02.251598  6635 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:02.235797  6835 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 13:02:02.290066  6569 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:02.386684  6936 consensus_meta.cc:287] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.554s	user 0.008s	sys 0.002s
W20260709 13:02:02.257344  7034 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:02:02.444501  1692 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:02.445957  2234 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:02:02.447953  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 254305us
I20260709 13:02:02.422932  7006 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:02.456526  2229 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:02.429930  6653 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:02.482414  6653 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.430478  6879 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:02.486608  7032 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.336s	user 0.002s	sys 0.003s
I20260709 13:02:02.486433  6879 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.454229  6763 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:02.386565  6956 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.488548  6763 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.473119  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:02.446300  6824 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:02.471975  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:02.493238  6824 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.495136  6824 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:02.495630  6937 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:02.512913  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1938) took 1009 ms. Trace:
I20260709 13:02:02.513412  1689 rpcz_store.cc:276] 0709 13:02:01.503112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.504261 (+  1149us) service_pool.cc:224] Handling call
0709 13:02:01.504544 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:01.504734 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:01.504758 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:01.505176 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 13:02:01.737210 (+232034us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:01.737244 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:01.737300 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.512630 (+775330us) raft_consensus.cc:1685] finished
0709 13:02:02.512652 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.512829 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":216092,"threads_started":1,"wal-append.queue_time_us":463144}
I20260709 13:02:02.454077  6863 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:02.422205  7037 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.453447  6759 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:02.531802  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.543167  1692 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:02:02.548292  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:02.478283  6838 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:02:02.573312  6992 consensus_meta.cc:287] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.685s	user 0.012s	sys 0.000s
I20260709 13:02:02.573864  6992 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.576403  6992 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:02.529376  6863 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:02.485286  6653 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:02.568837  6556 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:02.594681  6853 consensus_meta.cc:287] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.619s	user 0.011s	sys 0.000s
W20260709 13:02:02.591962  7056 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 13:02:02.570981  6768 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:02.571444  6763 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:02.568832  6937 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:02.589268  6838 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:02.595144  6889 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:02.595605  6812 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.476516  6936 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.639235  6763 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:02.640084  6763 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:02.644882  6763 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:02.648710  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:02.651507  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:02.652668  1539 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:02.604600  6556 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.617671  6853 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:02.692260  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 479505us
W20260709 13:02:02.695781  1507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.000s	sys 0.017s
W20260709 13:02:02.696170  1507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.000s	sys 0.017s
I20260709 13:02:02.677760  6759 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:02.632468  6937 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:02.699517  1690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.009s	sys 0.009s
I20260709 13:02:02.664964  6812 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:02.703259  1690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.009s	sys 0.009s
I20260709 13:02:02.704497  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:02.708115  1543 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:02.700110  6838 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.775501  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:02.775789  1611 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:02.776366  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:02.684361  7049 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:02:02.776535  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:02.776815  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:02.777669  1617 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:02:02.776386  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:02.790109  1613 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:02.792419  1616 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:02.792922  1613 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:02:02.793848  1616 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: [12.10-14.11]
W20260709 13:02:02.794119  2202 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.050s	sys 0.103s
W20260709 13:02:02.794600  2202 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.050s	sys 0.103s
I20260709 13:02:02.791275  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:02.785966  1611 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:02:02.787128  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:02.788002  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:02.701601  6666 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:02.796404  1612 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:02.797138  2203 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.012s	sys 0.001s
W20260709 13:02:02.799034  2203 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.012s	sys 0.001s
W20260709 13:02:02.702257  7035 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.013s	sys 0.005s
W20260709 13:02:02.806756  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:02.803216  6829 consensus_meta.cc:287] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.956s	user 0.010s	sys 0.005s
I20260709 13:02:02.773746  6877 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.779233  7045 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:02.793618  6762 consensus_meta.cc:287] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.876s	user 0.012s	sys 0.003s
I20260709 13:02:02.823096  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:02.821307  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 699884us
W20260709 13:02:02.790534  6990 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.775s	user 0.005s	sys 0.010s
I20260709 13:02:02.797573  6993 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.838543  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:02.839505  2231 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:02:02.840481  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:02.841832  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:02.842272  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:02.842748  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:02.826680  7035 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.014s	sys 0.005s
I20260709 13:02:02.801083  7047 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:02.819871  6653 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:02.798976  7072 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.248s	user 0.002s	sys 0.001s
I20260709 13:02:02.788661  7069 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.801789  6838 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:02.808926  1539 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:02.854255  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:02.799683  6810 consensus_meta.cc:287] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.524s	user 0.005s	sys 0.008s
I20260709 13:02:02.835060  6877 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.832131  7037 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:02.903976  7062 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:02.904692  7062 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.906669  6877 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:02.906905  7062 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:02.802277  6569 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.908032  2236 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:02.909178  6569 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:02.867156  6550 consensus_queue.cc:1059] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:02:02.911387  6550 raft_consensus.cc:3055] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:02.824146  7048 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:02.870231  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:02.870850  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1036) took 1054 ms. Trace:
I20260709 13:02:02.861689  2203 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:02.819597  6666 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.914393  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:02.915845  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.919747  1683 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:02.920126  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 58531.9us
I20260709 13:02:02.917776  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1123) took 1110 ms. Trace:
I20260709 13:02:02.911779  6550 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:02.917251  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:02.874166  7053 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:02:02.823275  6829 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.923440  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:02.919251  1540 rpcz_store.cc:276] 0709 13:02:01.816367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.843387 (+ 27020us) service_pool.cc:224] Handling call
0709 13:02:01.843764 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:01.979038 (+135274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:01.979068 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:01.979660 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.133686 (+154026us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:02.133726 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.870345 (+736619us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b540012ded4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:02.870420 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.870481 (+    61us) raft_consensus.cc:1685] finished
0709 13:02:02.870500 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.870779 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":310541184,"thread_start_us":153674,"threads_started":1,"wal-append.queue_time_us":287697}
I20260709 13:02:02.921038  1690 rpcz_store.cc:276] 0709 13:02:01.807159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.808227 (+  1068us) service_pool.cc:224] Handling call
0709 13:02:01.832821 (+ 24594us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:01.854816 (+ 21995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:01.854847 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:01.855306 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.703880 (+848574us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:02.703923 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.703997 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.917554 (+213557us) raft_consensus.cc:1685] finished
0709 13:02:02.917578 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:02.917715 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21571,"raft.queue_time_us":25036,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":14577,"thread_start_us":801443,"threads_started":1,"wal-append.queue_time_us":1023898}
I20260709 13:02:02.927909  1688 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:02.845755  7035 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:02.941635  6969 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:02.942242  6969 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.942533  7035 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.943789  2236 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:02.846827  6990 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.948284  1609 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:02.949726  6990 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:02.861069  6796 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:02.950714  6666 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:02.933848  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 66960.4us
I20260709 13:02:02.935758  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:02.861450  6936 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:02.937083  6840 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:02.937371  7066 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:02.940326  7038 log.cc:927] Time spent T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.746s	user 0.001s	sys 0.004s
I20260709 13:02:02.843992  6762 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.961261  1609 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:02.875559  6835 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 13:02:02.926651  6937 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:02.928578  7062 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:02.899928  6653 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.855779  6993 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.973860  6556 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:02.980715  1683 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
W20260709 13:02:02.987059  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:02.912482  6810 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.952521  6812 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:02.838338  7045 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.914176  7069 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:02.951181  6796 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.875043  7047 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.991416  6675 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:02.926502  7048 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:02.992719  6877 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.030723  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1251) took 1357 ms. Trace:
I20260709 13:02:03.042393  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1038) took 1221 ms. Trace:
I20260709 13:02:03.058594  1535 rpcz_store.cc:276] 0709 13:02:01.820857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.822015 (+  1158us) service_pool.cc:224] Handling call
0709 13:02:01.822432 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.060698 (+238266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.060728 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:02.061264 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.436385 (+375121us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:02.436464 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.436567 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.042078 (+605511us) raft_consensus.cc:1685] finished
0709 13:02:03.042111 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.042309 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":237545,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":125,"thread_start_us":611350,"threads_started":2,"wal-append.queue_time_us":557939}
W20260709 13:02:03.041695  6824 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.004s
I20260709 13:02:03.010104  6762 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.056644  2235 rpcz_store.cc:276] 0709 13:02:01.673121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.680719 (+  7598us) service_pool.cc:224] Handling call
0709 13:02:01.681019 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:01.681385 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:01.681413 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:01.681925 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.045318 (+363393us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:02.045358 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.045445 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:02.939640 (+894195us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:03.014320 (+ 74680us) raft_consensus.cc:1685] finished
0709 13:02:03.014351 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.030631 (+ 16280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18168,"raft.queue_time_us":1006,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":10747,"spinlock_wait_cycles":7080320,"thread_start_us":363076,"threads_started":1,"wal-append.queue_time_us":444463}
W20260709 13:02:03.003270  7042 consensus_meta.cc:287] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.529s	user 0.013s	sys 0.000s
W20260709 13:02:03.051625  6938 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.912s	user 0.004s	sys 0.014s
I20260709 13:02:03.063051  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:03.095034  1614 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:03.104511  1535 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:02:03.109403  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:03.110605  1535 raft_consensus.cc:2368] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13.
I20260709 13:02:03.113265  1584 leader_election.cc:400] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13.
I20260709 13:02:03.113785  1584 leader_election.cc:403] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:02:03.114944  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:02.979553  7066 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.053539  6675 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:02.993008  6550 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.137969  1612 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:03.061996  7059 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
W20260709 13:02:03.123652  7077 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:02:03.125926  7045 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:03.126890  6762 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:03.128258  7048 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:02.993379  7062 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.118050  1535 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:03.141921  7063 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.175894  7069 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:03.176581  1539 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:03.178103  7066 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.097666  6812 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:03.176452  7063 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.178784  6812 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:03.179832  6889 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.002s
I20260709 13:02:03.180583  1617 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:02:03.180917  6812 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.180979  7063 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.092583  7074 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:03.182003  6993 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:03.182376  7074 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.183254  7070 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:03.098910  6990 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:03.153981  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.108067  7042 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:03.096975  6877 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.140506  6574 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13.
I20260709 13:02:03.186017  6990 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.191951  1535 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:03.188391  6550 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:03.203020  6956 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:03.193087  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:03.185361  7074 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.187541  7047 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:03.189145  6877 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:03.191175  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:03.189693  6675 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.212383  6675 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:03.212102  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:03.213222  6990 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.213320  1537 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
W20260709 13:02:03.215341  6863 consensus_meta.cc:287] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.612s	user 0.004s	sys 0.006s
I20260709 13:02:03.216104  6863 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:03.217408  1540 raft_consensus.cc:2393] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:03.143757  6810 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:03.174440  6829 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.122557  6938 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.175755  6653 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.224759  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:03.243598  6812 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 13:02:03.172147  6762 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.246866  6859 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:03.247532  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:03.248324  6762 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.243239  6824 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:02:03.249092  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:03.249740  1612 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:02:03.248340  1609 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:03.251291  6938 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.244529  6748 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:03.245595  7063 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.253976  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:03.252975  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:03.251410  1684 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:03.255834  1612 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.256042  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:03.256925  1683 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:03.255127  1535 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:02:03.239461  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:03.231596  6877 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:03.298784  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 59426.8us
I20260709 13:02:03.299543  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:03.300048  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2954) took 1903 ms. Trace:
I20260709 13:02:03.263165  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:03.278218  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:03.300647  1536 rpcz_store.cc:276] 0709 13:02:01.396411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.396550 (+   139us) service_pool.cc:224] Handling call
0709 13:02:01.396871 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.105942 (+709071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.299831 (+1193889us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54000ed3d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:03.299994 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629090176}
W20260709 13:02:03.284792  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.277635  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:03.296025  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:03.300190  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:03.303270  2228 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:03.304443  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:03.253558  6889 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:03.308584  1686 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:03.249776  6796 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:03.312585  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:03.313694  2237 raft_consensus.cc:2368] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15.
I20260709 13:02:03.314575  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:03.315420  2235 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:03.323124  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:03.323649  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 402616us
I20260709 13:02:03.256665  7073 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:03.328675  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:03.297199  6762 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.299891  6990 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:03.334385  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:03.303972  6996 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:03.359771  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:03.305886  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:03.362721  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:03.307147  6838 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:03.309214  6889 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.368151  1687 raft_consensus.cc:2393] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:03.363300  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:03.367102  1683 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
W20260709 13:02:03.363857  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 62449.1us
I20260709 13:02:03.369401  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:03.321066  6863 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:03.255502  6748 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.317744  6574 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:03.375003  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 50329.6us
I20260709 13:02:03.377743  1690 raft_consensus.cc:2368] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13.
W20260709 13:02:03.409510  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 53645.8us
I20260709 13:02:03.317842  6874 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:02:03.256803  7063 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:03.307507  6877 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.373631  6889 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:03.374203  6838 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:03.362459  7073 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:03.366581  6762 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:03.368129  6810 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:02:03.370277  6990 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.438004  6838 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:03.408578  6938 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
W20260709 13:02:03.501722  7080 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 13:02:03.506959  7063 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:03.508242  6990 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.509459  7073 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:03.554368  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.557111  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:03.560285  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:03.561038  1508 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:03.553184  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:03.564769  1508 leader_election.cc:400] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15.
I20260709 13:02:03.565004  1534 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:03.567970  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:03.570495  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:03.572496  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:03.565286  1508 leader_election.cc:403] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:02:03.573494  1609 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:03.574898  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:03.576933  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:03.380890  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:03.581534  1612 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
W20260709 13:02:03.582378  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 201433us
I20260709 13:02:03.583272  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:03.584282  1612 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:02:03.584460  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:03.586388  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 491915us
I20260709 13:02:03.587163  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:03.588317  1615 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:03.591034  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:03.591200  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:03.593837  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:03.594231  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:03.594758  1692 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:03.416879  2203 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:03.598076  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:03.598984  2231 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.020s	sys 0.036s
W20260709 13:02:03.599749  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.600205  2230 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.430096  2237 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:03.425325  6574 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.604195  6748 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.641219  1536 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:03.644378  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:03.646703  2231 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.020s	sys 0.036s
W20260709 13:02:03.624002  1542 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.002s	sys 0.013s
I20260709 13:02:03.650681  1687 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:02:03.650980  6812 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:03.651527  6812 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.652031  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:03.655400  6574 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.656641  1609 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:03.657881  1691 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:03.658237  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:03.658103  6835 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:03.652094  6812 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:03.659885  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:03.661743  6666 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.661970  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:03.662691  1686 raft_consensus.cc:2410] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 15 because replica has last-logged OpId of term: 14 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 13:02:03.664001  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:03.662966  1612 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
W20260709 13:02:03.666728  1542 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.003s	sys 0.014s
I20260709 13:02:03.658103  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:03.673894  7063 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:03.674216  7042 consensus_queue.cc:1059] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:03.675117  7066 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:03.675012  7063 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.675693  7066 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:03.676677  6748 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15.
I20260709 13:02:03.676134  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:03.677613  6990 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.677860  6748 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:03.678530  6990 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.681926  6556 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.682905  6556 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:03.686154  6810 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:03.686887  6810 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:03.688390  6810 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:03.689652  6840 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.690481  6840 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:03.676162  7066 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:03.693706  1615 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
W20260709 13:02:03.670619  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 106293us
I20260709 13:02:03.706199  1534 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
W20260709 13:02:03.705063  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 114344us
I20260709 13:02:03.707013  1536 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 11.
I20260709 13:02:03.714775  6914 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:03.712970  1618 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:03.716480  6877 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:03.717124  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:03.717576  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.718050  1618 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:03.721330  6829 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:03.721752  6653 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:03.729460  6653 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.722065  6996 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:03.734032  6835 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:03.736443  6835 raft_consensus.cc:697] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:03.736120  6996 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.738041  1582 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:03.731796  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1248) took 2434 ms. Trace:
I20260709 13:02:03.723194  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:03.662714  6666 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.751968  6666 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.728453  6824 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:03.752677  6824 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:03.754784  7048 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:03.754457  6824 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.755812  7062 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.755738  7048 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.757053  6824 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:03.758306  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 324599us
I20260709 13:02:03.752050  1614 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:03.734566  6914 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:03.727224  7042 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:03.726312  7045 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:03.721346  7068 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.934s	user 0.002s	sys 0.000s
I20260709 13:02:03.738247  6835 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.764856  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:03.768626  7045 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.769474  6855 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:03.746217  2232 rpcz_store.cc:276] 0709 13:02:01.296810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.341954 (+ 45144us) service_pool.cc:224] Handling call
0709 13:02:01.342275 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.128377 (+786102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.731144 (+1602767us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b5400630e54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:03.731570 (+   426us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1794908416}
I20260709 13:02:03.772944  1542 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:03.774045  1609 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:03.728416  6829 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.763904  7042 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:03.781052  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:03.774780  6556 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:03.764636  6938 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:03.810966  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 44833.9us
I20260709 13:02:03.819756  6840 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:03.814107  6601 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:03.821933  6601 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:03.822007  1657 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.614s (SENT) [suppressed 11 similar messages]
I20260709 13:02:03.823511  6996 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.823428  6601 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.824221  6996 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.824915  6601 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:03.825667  6938 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:03.826512  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1125) took 1762 ms. Trace:
I20260709 13:02:03.827430  6974 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:03.827894  7048 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:03.828461  2230 raft_consensus.cc:2410] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:02:03.828560  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2464) took 2290 ms. Trace:
I20260709 13:02:03.824024  6969 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.821446  1610 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:03.843114  7063 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.794767  6990 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.824970  6996 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:03.842229  1684 rpcz_store.cc:276] 0709 13:02:02.064179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.064308 (+   129us) service_pool.cc:224] Handling call
0709 13:02:02.097460 (+ 33152us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.256358 (+1158898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.825957 (+569599us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400058f54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:03.826286 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2653362048}
I20260709 13:02:03.855535  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2489) took 1027 ms. Trace:
I20260709 13:02:03.856151  6914 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.857955  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:03.858786  6976 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:02:03.859251  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 117580us
I20260709 13:02:03.858852  7042 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.859694  1535 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:02:03.864117  6671 raft_consensus.cc:3055] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:03.857975  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 2965) took 1039 ms. Trace:
W20260709 13:02:03.872434  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 295962us
I20260709 13:02:03.844655  6974 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:03.869602  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2969) took 1012 ms. Trace:
I20260709 13:02:03.846900  1541 rpcz_store.cc:276] 0709 13:02:01.537861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.538860 (+   999us) service_pool.cc:224] Handling call
0709 13:02:01.539224 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:01.788054 (+248830us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:01.788365 (+   311us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.035960 (+247595us) log.cc:844] Serialized 14 byte log entry
0709 13:02:02.122759 (+ 86799us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:02.122810 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:02.122854 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.687985 (+565131us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b480030c858. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:02:03.828026 (+1140041us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54003e2954. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:03.828114 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.828329 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12958,"prepare.queue_time_us":1426,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":3134,"raft.queue_time_us":182896,"raft.run_cpu_time_us":13266,"raft.run_wall_time_us":584564,"spinlock_wait_cycles":580365568,"thread_start_us":320569,"threads_started":2,"wal-append.queue_time_us":362177,"wal-append.run_cpu_time_us":1697,"wal-append.run_wall_time_us":1110794}
I20260709 13:02:03.880589  1542 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:03.885591  1535 rpcz_store.cc:276] 0709 13:02:02.857317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.858626 (+1001309us) service_pool.cc:224] Handling call
0709 13:02:03.859164 (+   538us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.869305 (+ 10141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.869543 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:03.886384  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2967) took 1029 ms. Trace:
I20260709 13:02:03.886828  1542 rpcz_store.cc:276] 0709 13:02:02.856753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.810332 (+953579us) service_pool.cc:224] Handling call
0709 13:02:03.876862 (+ 66530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.886053 (+  9191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.886298 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:03.888211  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1046) took 1030 ms. Trace:
I20260709 13:02:03.888576  1542 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:03.872243  6671 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:03.890588  1541 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:03.862159  1617 rpcz_store.cc:276] 0709 13:02:02.814577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.816159 (+  1582us) service_pool.cc:224] Handling call
0709 13:02:02.823608 (+  7449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.180952 (+357344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.748699 (+567747us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b54003c87d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:03.841960 (+ 93261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":820186880}
I20260709 13:02:03.891237  6938 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:03.890959  6671 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.892613  6671 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:03.880191  6653 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.898456  1541 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:03.888784  1535 rpcz_store.cc:276] 0709 13:02:02.857891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.886765 (+1028874us) service_pool.cc:224] Handling call
0709 13:02:03.887368 (+   603us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.887727 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.887767 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.887864 (+    97us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:03.887912 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.887990 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.888134 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:03.910113  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1418) took 1069 ms. Trace:
I20260709 13:02:03.910805  2231 rpcz_store.cc:276] 0709 13:02:02.840854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.841036 (+   182us) service_pool.cc:224] Handling call
0709 13:02:02.841368 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:02.841640 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:03.739727 (+898087us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.870831 (+131104us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.877164 (+  6333us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:03.909616 (+ 32452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.909770 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.910001 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3723,"prepare.queue_time_us":214140,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":3609,"raft.queue_time_us":16810,"thread_start_us":867112,"threads_started":2}
I20260709 13:02:03.914837  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1037) took 2096 ms. Trace:
I20260709 13:02:03.915329  1538 rpcz_store.cc:276] 0709 13:02:01.818051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.819099 (+  1048us) service_pool.cc:224] Handling call
0709 13:02:01.819555 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.133292 (+313737us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:02.133323 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:02.133827 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 13:02:02.450650 (+316823us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:02.450696 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:02.450774 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.720176 (+1269402us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:03.914259 (+194083us) raft_consensus.cc:1685] finished
0709 13:02:03.914288 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.914626 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":219835,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":16477,"spinlock_wait_cycles":7096576,"thread_start_us":611715,"threads_started":2,"wal-append.queue_time_us":521060}
I20260709 13:02:03.923293  7045 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.860548  7073 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:03.879657  6923 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.012s	sys 0.010s
W20260709 13:02:03.930048  6923 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.012s	sys 0.010s
I20260709 13:02:03.932991  6974 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:03.934559  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.934922  6974 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:03.935985  6993 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:03.936674  6993 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.880146  6889 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:03.938796  6829 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.939556  6829 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:02:03.880373  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.939805  6993 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:03.879165  6800 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:03.879911  1614 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:03.951535  6556 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:03.872344  1540 rpcz_store.cc:276] 0709 13:02:02.803004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.934857 (+131853us) service_pool.cc:224] Handling call
0709 13:02:03.783391 (+848534us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b540030c0d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:03.842383 (+ 58992us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610338048}
W20260709 13:02:03.953960  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 96224.3us
I20260709 13:02:03.954608  6762 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:03.955736  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2476) took 1135 ms. Trace:
W20260709 13:02:03.940933  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.956350  1543 rpcz_store.cc:276] 0709 13:02:02.820015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.747777 (+927762us) service_pool.cc:224] Handling call
0709 13:02:03.838878 (+ 91101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:03.876942 (+ 38064us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.876967 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:03.955240 (+ 78273us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:03.955304 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.955374 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.955475 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:03.901758  7112 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:03.962002  7112 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:03.964399  7112 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:03.955848  1582 leader_election.cc:400] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13.
I20260709 13:02:03.973694  7042 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:03.981866  1609 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:03.939836  6889 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:03.939961  6829 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:03.997582  7109 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:02:03.999930  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2488) took 1188 ms. Trace:
I20260709 13:02:04.000329  1611 rpcz_store.cc:276] 0709 13:02:02.811683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.813297 (+  1614us) service_pool.cc:224] Handling call
0709 13:02:02.813768 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:02.820857 (+  7089us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:03.258018 (+437161us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.841557 (+583539us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.855646 (+ 14089us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.855825 (+   179us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:03.856159 (+   334us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.859868 (+  3709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.859978 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:03.999540 (+139562us) raft_consensus.cc:1685] finished
0709 13:02:03.999560 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:03.999734 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25013,"prepare.queue_time_us":637102,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":1898,"raft.queue_time_us":24427,"raft.run_cpu_time_us":428,"raft.run_wall_time_us":53742,"thread_start_us":845143,"threads_started":2,"wal-append.queue_time_us":516439}
I20260709 13:02:04.002995  6996 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:04.007882  1618 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:04.011801  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:04.012760  1543 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:04.012920  1657 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:04.020947  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1421) took 1179 ms. Trace:
I20260709 13:02:04.021509  2236 rpcz_store.cc:276] 0709 13:02:02.841952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.842063 (+   111us) service_pool.cc:224] Handling call
0709 13:02:04.020537 (+1178474us) spinlock_profiling.cc:243] Waited 65.1 ms on lock 0x7b54007404d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:04.020712 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149632896}
I20260709 13:02:04.024189  6936 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:04.025555  7103 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.339s	user 0.000s	sys 0.002s
I20260709 13:02:04.026137  1540 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:04.024976  6936 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:04.028646  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 73015us
I20260709 13:02:04.030439  1614 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:03.865013  6855 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:04.004141  6829 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:04.039994  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.040295  2234 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:02:04.024730  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.031524  6976 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:02:04.021073  6859 consensus_meta.cc:287] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.774s	user 0.014s	sys 0.002s
W20260709 13:02:04.048444  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 65420.2us
I20260709 13:02:04.052985  6798 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 13:02:04.056265  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.057555  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.058658  1538 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:02:04.061288  6675 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.848s	user 0.012s	sys 0.006s
W20260709 13:02:04.061749  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 271473us
W20260709 13:02:04.064926  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.066180  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:04.066102  6812 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:04.067467  1509 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:04.068542  1509 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:04.069619  7035 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.070187  7035 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.072445  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.073868  6748 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:04.074374  6748 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:04.074759  6748 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:04.075523  6762 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:04.076066  6762 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:04.076409  6762 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:04.077206  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2964) took 1274 ms. Trace:
I20260709 13:02:04.077904  6653 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:04.078053  7063 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:04.079006  6666 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:04.078624  6653 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.079646  6666 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.081612  6938 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.077669  6984 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:04.079219  7107 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.078565  7088 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:04.072685  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:04.096701  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.090683  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.043467  6936 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:04.077710  1539 rpcz_store.cc:276] 0709 13:02:02.802705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.741555 (+938850us) service_pool.cc:224] Handling call
0709 13:02:03.768875 (+ 27320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.771489 (+  2614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.771522 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.772260 (+   738us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.849623 (+ 77363us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:03.869782 (+ 20159us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.076532 (+206750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.076640 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.076706 (+    66us) raft_consensus.cc:1685] finished
0709 13:02:04.076725 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.076916 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":208322,"prepare.queue_time_us":20651,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":213757,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":310,"thread_start_us":76079,"threads_started":1,"wal-append.queue_time_us":105351}
I20260709 13:02:04.084905  6653 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:04.117388  2230 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:04.120442  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 107633us
I20260709 13:02:04.123010  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.124704  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.109218  6824 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:04.125941  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.126753  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.128588  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.129099  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2963) took 1326 ms. Trace:
I20260709 13:02:04.130623  1690 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:04.129565  1537 rpcz_store.cc:276] 0709 13:02:02.802379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:03.214595 (+412216us) service_pool.cc:224] Handling call
0709 13:02:03.214962 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:03.227564 (+ 12602us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:03.227617 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:03.254883 (+ 27266us) log.cc:844] Serialized 26 byte log entry
0709 13:02:03.690724 (+435841us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:03.732646 (+ 41922us) log.cc:844] Serialized 14 byte log entry
0709 13:02:03.761585 (+ 28939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:03.774001 (+ 12416us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:04.126564 (+352563us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b18001dce60. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:02:04.128644 (+  2080us) raft_consensus.cc:1685] finished
0709 13:02:04.128749 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:04.128929 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":288585,"prepare.queue_time_us":12065,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":45268,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":20395,"spinlock_wait_cycles":6403840,"thread_start_us":105349,"threads_started":1,"wal-append.queue_time_us":125175}
I20260709 13:02:04.133654  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:04.134361  1614 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:04.136193  6996 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:04.136971  6996 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.139400  7073 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:04.139514  6996 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:04.140341  6812 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:04.128334  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.116206  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:04.142671  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.142185  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:04.146507  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.147595  1692 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:04.147706  1613 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:04.148532  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.149075  6653 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:04.149784  6653 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:04.150452  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:04.151230  1614 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:02:04.150991  6653 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:04.152052  6824 consensus_queue.cc:1059] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:04.152446  6835 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:04.153786  6835 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:04.154189  6835 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:04.154505  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.155154  6835 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:04.156010  1612 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:02:04.156559  6835 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:04.089807  6984 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:04.157303  6984 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:04.157836  1685 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:04.158232  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:04.158977  1615 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:04.159977  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.160470  6601 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.161119  6601 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.159001  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:04.163282  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.078785  7063 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:04.166033  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 50527.6us
I20260709 13:02:04.168190  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:04.169163  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:04.170087  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 101689us
I20260709 13:02:04.170738  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:04.180440  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.180878  1543 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:04.181656  1534 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:02:04.093405  6938 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:04.185861  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.186486  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:04.182091  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:04.187702  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:04.188651  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:04.189347  1613 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:04.190438  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:04.141746  7035 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:04.213990  1543 raft_consensus.cc:2393] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:04.199581  2203 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:04.215148  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:04.201486  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.205763  1538 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:04.213636  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.217263  2231 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:04.209596  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.191597  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:04.227944  1617 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:02:04.198460  6601 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:04.228832  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:04.229544  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.157755  7107 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:04.191310  6838 consensus_meta.cc:287] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.724s	user 0.008s	sys 0.004s
I20260709 13:02:04.281513  6838 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.200425  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.201927  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.282987  2235 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:04.229226  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.284090  1689 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:04.284562  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:04.285594  1686 raft_consensus.cc:2368] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15.
I20260709 13:02:04.286509  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.287629  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.194909  7116 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.288757  7116 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:04.289611  7042 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:04.290383  7042 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.291447  7116 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:04.293147  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 2966) took 1489 ms. Trace:
I20260709 13:02:04.293756  1534 rpcz_store.cc:276] 0709 13:02:02.803292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.019738 (+1216446us) service_pool.cc:224] Handling call
0709 13:02:04.181392 (+161654us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.182048 (+   656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.292857 (+110809us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54001e5850. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:04.293066 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370746112}
I20260709 13:02:04.297919  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1419) took 1456 ms. Trace:
I20260709 13:02:04.298491  2234 rpcz_store.cc:276] 0709 13:02:02.841187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:02.841438 (+   251us) service_pool.cc:224] Handling call
0709 13:02:02.841742 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.068609 (+1226867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.297552 (+228943us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400475254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:04.297840 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2754966272}
I20260709 13:02:04.166720  6829 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:04.307875  6829 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:04.308790  1683 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:04.209684  6996 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.310191  6996 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.255620  6984 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:04.312722  6984 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:04.195178  7063 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:04.274724  1540 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
W20260709 13:02:04.255010  7066 consensus_meta.cc:287] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.511s	user 0.004s	sys 0.006s
I20260709 13:02:04.259614  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:04.261848  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:04.317232  1609 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:02:04.262624  1614 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:04.200086  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.318574  1611 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:04.209352  6666 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:04.202757  6956 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:04.193573  1583 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:04.276207  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.119273  7088 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.278666  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.287734  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:04.248952  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:04.310626  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 81914.8us
I20260709 13:02:04.256973  6938 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:04.214995  6812 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:04.318318  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.272714  1543 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:02:04.267112  6762 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.326725  6956 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:04.326999  6855 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:04.328102  1540 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:02:04.329104  6872 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:04.329289  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 150260us
I20260709 13:02:04.205330  6800 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.125s
I20260709 13:02:04.319593  1685 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:04.293030  7042 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:04.313390  6984 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:02:04.361768  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 88078us
W20260709 13:02:04.363268  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 103812us
I20260709 13:02:04.367127  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.369880  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.371167  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.372304  1691 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:04.371074  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.384016  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.385314  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.392988  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.397243  1686 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:04.397756  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:04.317291  6996 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:04.400135  1685 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:04.415944  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.416836  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:04.417287  1536 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:04.331278  7066 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.389518  6601 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:04.413475  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:04.419559  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 220064us
W20260709 13:02:04.419832  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 87030.5us
I20260709 13:02:04.425850  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.420028  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.429024  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.397927  6938 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:04.430671  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.434001  1583 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:04.438378  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.439364  1688 raft_consensus.cc:2410] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:02:04.440389  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:04.440970  1688 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:04.441522  7064 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.018s	sys 0.027s
W20260709 13:02:04.441931  7064 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.018s	sys 0.027s
I20260709 13:02:04.430011  1686 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
W20260709 13:02:04.447055  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:04.448167  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 44040us
I20260709 13:02:04.449025  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:04.451097  1535 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:04.453197  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:04.453687  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:04.454316  6973 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:04.454464  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:04.455117  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 313003us
I20260709 13:02:04.455027  6973 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:04.457973  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.458770  1691 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 12: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:04.455050  1536 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:04.465533  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.466272  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.402787  6812 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.469602  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:04.469059  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 67939.4us
I20260709 13:02:04.473487  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:04.435495  6748 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:04.469650  6938 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:04.476157  6923 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:04.478330  1509 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:04.477641  6973 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:04.478494  6956 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.480967  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.482194  1684 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:02:04.499719  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:03.996989   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:04.508272  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 74379.3us
I20260709 13:02:04.488605  7042 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:04.469745  6556 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:04.488474  6872 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:04.506000  6855 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.556275  1685 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:04.558772  1683 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:04.554971  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:04.513602  6800 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:04.563025  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:04.573313  6889 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:04.573815  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.574544  2234 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:04.515787  7042 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:04.618260  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.624737  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:04.558689  6666 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:04.550691  6976 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:04.616909  7042 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:04.631567  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.562430  6859 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.562389  6800 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.574862  6973 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:04.639503  2230 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:04.602322  6860 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:04.574712  7074 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:04.624495  6812 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:04.680287  1683 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 13:02:04.680891  1683 pending_rounds.cc:85] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 10
I20260709 13:02:04.630064  6666 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:04.657639  6800 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:04.645383  7107 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:04.671748  6859 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.703424  1609 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:02:04.704620  7073 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:04.705678  7123 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:02:04.705278  7074 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.706035  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:04.677816  7110 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:04.682639  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:04.708705  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:04.701649  6973 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:04.714730  1683 pending_rounds.cc:107] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 13:02:04.620402  6872 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:04.628335  6938 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.816049  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.713788  6666 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:04.840138  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 328244us
I20260709 13:02:04.707748  6976 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:04.653872  6855 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.841341  7066 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:04.895900  6835 consensus_meta.cc:287] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.739s	user 0.019s	sys 0.000s
W20260709 13:02:04.881188  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.760605  2230 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:04.818881  6877 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:04.631670  7112 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.082s
W20260709 13:02:04.913053  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 251158us
I20260709 13:02:04.880617  1683 raft_consensus.cc:2889] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:04.915345  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 365658us
W20260709 13:02:04.953245  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.954926  1509 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:04.955829  1689 raft_consensus.cc:2410] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:02:04.956115  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:04.965735  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:04.969141  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:04.951049  7107 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:04.934296  6748 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:04.841813  7110 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:04.892069  7073 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:04.973323  1617 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:02:04.980742  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:04.949393  6923 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:04.981192  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:04.912035  7064 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:05.013561  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:05.020566  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:05.028729  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:05.013123  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:05.035542  1537 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:05.040753  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.045136  1617 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:05.046905  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:05.058471  7128 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 13:02:05.063253  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.064199  1616 raft_consensus.cc:2393] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:05.065912  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.066351  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.066927  1541 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:05.067276  1612 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 13:02:05.073182  2236 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:05.066802  1613 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:05.065086  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.094575  1612 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:05.099383  6996 raft_consensus.cc:3055] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:05.099951  6996 raft_consensus.cc:740] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:05.105412  1535 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:05.108114  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:05.108772  1617 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:05.045061  7107 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:05.109287  1616 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:05.115468  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:05.125957  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:05.120572  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:05.131426  1610 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:05.028023  7064 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.133364  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:05.141669  7126 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.324s	user 0.001s	sys 0.000s
I20260709 13:02:05.107892  6984 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:05.141764  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 186915us
I20260709 13:02:05.142321  6984 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.142961  6976 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:05.143441  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:05.130270  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:05.155570  1536 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.671s	user 0.007s	sys 0.002s
I20260709 13:02:05.156221  1536 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:02:05.161716  6984 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.162181  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:05.162989  6984 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:05.163543  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:05.164423  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:05.165724  6973 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:05.166515  1538 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:05.165123  6984 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:05.167380  6984 raft_consensus.cc:697] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:05.168439  6984 consensus_queue.cc:237] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:05.068151  6923 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.171137  6923 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:05.171342  7064 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.172282  7074 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.172791  7073 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:05.152114  7042 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.175230  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:05.175938  7042 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.134014  6859 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.101078  6996 consensus_queue.cc:260] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:05.304428  1617 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:02:05.178573  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:05.193746  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:05.248123  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:05.198853  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 1962) took 1005 ms. Trace:
I20260709 13:02:05.177922  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:05.190348  1688 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.744s	user 0.008s	sys 0.004s
I20260709 13:02:05.309875  1688 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:05.310479  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1953) took 1120 ms. Trace:
I20260709 13:02:05.288352  1535 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:02:05.141283  6860 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:05.316777  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:05.230863  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 754338us
I20260709 13:02:05.307825  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:05.178944  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 113594us
I20260709 13:02:05.310863  1688 rpcz_store.cc:276] 0709 13:02:04.190200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.440085 (+249885us) service_pool.cc:224] Handling call
0709 13:02:05.310419 (+870334us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:05.211381  7149 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 13:02:05.234402  7132 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 13:02:05.221109  7135 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:02:05.338637  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:05.215940  6855 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.353192  1541 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:02:05.222047  7107 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:05.362247  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 168577us
I20260709 13:02:05.237164  6973 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.372282  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:05.306686  1686 rpcz_store.cc:276] 0709 13:02:04.192792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.492800 (+300008us) service_pool.cc:224] Handling call
0709 13:02:04.552337 (+ 59537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.627975 (+ 75638us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:04.678536 (+ 50561us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.197316 (+518780us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:05.197395 (+    79us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:05.197480 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.197572 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.197739 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58931,"prepare.queue_time_us":70831,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":133001,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":144,"thread_start_us":181947,"threads_started":1}
I20260709 13:02:05.267779  6762 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.285931  6872 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:05.313504  7066 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.367647  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:05.332864  6956 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:05.366979  1584 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:05.209668  6783 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.155s
I20260709 13:02:05.222890  6838 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.363626  1537 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:05.366986  2235 raft_consensus.cc:2376] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14. Re-sending same reply.
I20260709 13:02:05.332759  6923 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:02:05.366272  6936 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:05.266551  6666 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:05.467640  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 81876.3us
W20260709 13:02:05.467967  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 151307us
I20260709 13:02:05.474246  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:05.471803  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 99681.2us
I20260709 13:02:05.374253  6796 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 13:02:05.370864  7107 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:05.470240  7152 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:05.476678  7148 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.477669  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:05.467583  6855 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:05.476006  1657 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.685s (SENT)
W20260709 13:02:05.481350  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 338055us
I20260709 13:02:05.483129  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:05.468029  7074 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:05.261081  7064 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.217s
I20260709 13:02:05.492657  1539 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:05.480304  7107 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:05.266577  7150 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.216s
W20260709 13:02:05.506718  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 139819us
I20260709 13:02:05.485383  6855 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:05.486989  6675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.037s	sys 0.087s
I20260709 13:02:05.468990  7066 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:05.517009  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:05.517611  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:05.502580  1507 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:05.503783  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:05.386471  6973 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.508585  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:05.265928  7112 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.217s
I20260709 13:02:05.485810  6872 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:05.533219  1686 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:02:05.511296  6675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.037s	sys 0.088s
I20260709 13:02:05.507927  6936 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:05.510738  6855 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:05.541796  7152 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:05.548133  7154 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:05.548792  7154 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:05.549233  7154 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:05.550967  7129 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:05.552390  2228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1269) took 1431 ms. Trace:
W20260709 13:02:05.552614  7136 log.cc:927] Time spent T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 13:02:05.553157  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 77210.3us
I20260709 13:02:05.553857  7141 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:05.555706  6973 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.530061  6872 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:05.556669  6872 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:05.556957  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:05.559525  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:05.560988  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:05.561681  6762 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:05.562135  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:05.562096  7144 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:05.562157  7163 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:05.563720  1584 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:05.557636  1686 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:05.564966  7144 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:05.565644  7144 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:05.565061  6889 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:05.567052  7144 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:05.568673  7144 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:05.564850  7141 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:05.552850  2228 rpcz_store.cc:276] 0709 13:02:04.121314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.127129 (+  5815us) service_pool.cc:224] Handling call
0709 13:02:04.127609 (+   480us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.203026 (+ 75417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.203074 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:04.203801 (+   727us) log.cc:844] Serialized 26 byte log entry
0709 13:02:04.514766 (+310965us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:04.514807 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.514937 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.551952 (+1037015us) raft_consensus.cc:1685] finished
0709 13:02:05.551980 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.552199 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74769,"raft.queue_time_us":79404,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":91,"thread_start_us":310514,"threads_started":1,"wal-append.queue_time_us":1265939}
I20260709 13:02:05.572098  6872 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.575093  6872 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.564878  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:05.581655  1584 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:05.581991  6872 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:05.583237  7157 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:05.493570  7074 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:05.589136  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 47594.5us
I20260709 13:02:05.593477  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:05.586412  1657 leader_election.cc:336] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.614s (SENT)
I20260709 13:02:05.598591  1657 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:05.485940  7148 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.598279  7141 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:05.610582  7160 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:05.604780  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.562325  7152 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:05.606945  6666 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.607566  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2854) took 1243 ms. Trace:
I20260709 13:02:05.608666  6556 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:05.614670  1618 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
W20260709 13:02:05.618135  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 422276us
I20260709 13:02:05.562294  6762 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:05.625408  1685 rpcz_store.cc:276] 0709 13:02:04.364396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.639798 (+275402us) service_pool.cc:224] Handling call
0709 13:02:04.666335 (+ 26537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:04.895483 (+229148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.895533 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:04.936726 (+ 41193us) log.cc:844] Serialized 26 byte log entry
0709 13:02:05.485131 (+548405us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:05.485181 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.485251 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.607339 (+122088us) raft_consensus.cc:1685] finished
0709 13:02:05.607361 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.607504 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":944,"raft.queue_time_us":177238,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":30801,"thread_start_us":445660,"threads_started":1,"wal-append.queue_time_us":586938}
I20260709 13:02:05.633283  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:05.642113  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 167945us
I20260709 13:02:05.597930  7157 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:05.614017  6872 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:05.647454  1537 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:05.624015  7152 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.627350  6556 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:05.613193  7154 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:05.664558  2203 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:05.641587  6984 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:05.676959  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 116045us
W20260709 13:02:05.678555  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 84437.7us
I20260709 13:02:05.654587  7152 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:05.664258  6653 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.664209  6824 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.654053  6666 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:02:05.683877  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:05.648209  7163 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:02:05.686693  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:05.681635  6800 consensus_meta.cc:287] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.840s	user 0.010s	sys 0.003s
I20260709 13:02:05.654760  6872 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:05.698191  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 153441us
I20260709 13:02:05.678217  6914 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.706687  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:05.707645  1688 raft_consensus.cc:2410] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:02:05.714696  1685 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:05.709075  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:05.689400  6653 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.702246  6800 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.701802  6762 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:05.762257  6800 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:05.763312  2229 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:05.764139  7148 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.765028  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1427) took 1133 ms. Trace:
I20260709 13:02:05.702538  7174 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:02:05.765187  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:05.765604  2229 rpcz_store.cc:276] 0709 13:02:04.631279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.631483 (+   204us) service_pool.cc:224] Handling call
0709 13:02:04.658708 (+ 27225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:05.763774 (+1105066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.764784 (+  1010us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400599cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:05.764967 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2539889408}
I20260709 13:02:05.769153  1537 raft_consensus.cc:2410] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 16 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:02:05.719379  1688 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:05.707125  7144 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:05.721465  7169 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:05.716112  1509 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:05.720970  7140 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:05.774924  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:05.722067  7170 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:05.685140  6984 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:02:05.713860  6556 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:05.753733  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 69905.2us
W20260709 13:02:05.775951  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 59950.4us
I20260709 13:02:05.695256  6860 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:02:05.776729  7163 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:05.690632  6824 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:05.717519  7157 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:05.777532  7177 raft_consensus.cc:2764] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:02:05.721176  6914 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.778946  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:05.778453  7157 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:05.778379  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 69433.5us
W20260709 13:02:05.781868  7162 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
W20260709 13:02:05.782869  7133 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.444s	user 0.002s	sys 0.000s
I20260709 13:02:05.783939  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2499) took 1233 ms. Trace:
I20260709 13:02:05.784013  7074 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.784595  6914 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:05.770025  6859 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.784518  1611 rpcz_store.cc:276] 0709 13:02:04.550994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.559116 (+  8122us) service_pool.cc:224] Handling call
0709 13:02:04.574064 (+ 14948us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:04.930798 (+356734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.930915 (+   117us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:04.959314 (+ 28399us) log.cc:844] Serialized 52 byte log entry
0709 13:02:05.485589 (+526275us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:05.636294 (+150705us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.643698 (+  7404us) log.cc:844] Serialized 14 byte log entry
0709 13:02:05.643930 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:05.644014 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.783712 (+139698us) raft_consensus.cc:1685] finished
0709 13:02:05.783733 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.783882 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17670,"prepare.queue_time_us":65497,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":130775,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":1207,"spinlock_wait_cycles":28058624,"thread_start_us":921819,"threads_started":3,"wal-append.queue_time_us":643843}
I20260709 13:02:05.786118  6824 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:05.786108  6859 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.788425  6859 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.789103  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.790256  1611 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:05.790531  6653 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.794865  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:05.800174  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:05.800812  1613 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:05.802917  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:05.805111  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:05.805981  1688 raft_consensus.cc:2368] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15.
I20260709 13:02:05.808010  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.808791  1614 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:02:05.808144  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.810235  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:05.817325  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:05.817745  1614 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:05.817958  7154 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:05.818568  7154 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:05.819162  7154 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.811906  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:05.815450  7139 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:05.826757  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.827359  1609 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:05.827553  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.828287  1616 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:05.814340  6556 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:05.822216  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:05.830729  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:05.819584  7134 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.117s	user 0.002s	sys 0.000s
I20260709 13:02:05.831313  1612 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:05.813977  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.838099  1618 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:05.831400  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:05.805969  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:05.837195  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.839277  1611 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:05.837126  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:05.773593  7144 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.841432  2232 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:05.842368  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:05.842545  1613 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:05.842522  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:05.843580  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:05.845242  2230 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:05.865962  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:05.848697  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:05.848217  1688 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:05.848385  7144 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:05.874236  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:05.822427  6872 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:05.865908  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 101250us
I20260709 13:02:05.867906  1539 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:05.821518  6855 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.824849  7110 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:05.810777  7170 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:05.847918  7139 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:05.821412  7154 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:05.887109  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:05.891216  7154 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:05.891757  7154 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:05.892196  7154 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:05.899849  7154 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.901211  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:05.910409  1685 raft_consensus.cc:2410] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 17 because replica has last-logged OpId of term: 16 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:02:05.904807  7178 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:05.905830  1539 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:05.906383  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:05.907209  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:05.915282  2235 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:05.911255  1543 raft_consensus.cc:2410] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:02:05.915692  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:05.912776  1688 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:05.901517  7139 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.883370  6762 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.922202  7178 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.917670  7139 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:05.924772  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:05.926335  1614 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:05.928836  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:05.929775  1611 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:05.930488  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:05.931927  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.932605  1614 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:05.933437  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:05.934109  2236 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:02:05.934211  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.934777  1617 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
W20260709 13:02:05.935087  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 91542.6us
I20260709 13:02:05.936394  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:05.937693  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:05.938287  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:05.938354  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:05.939476  1539 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 12.
I20260709 13:02:05.942317  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:05.938297  2235 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:05.933665  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:05.945735  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:05.919080  7110 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:05.947834  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:05.944944  2228 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:05.956554  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:05.874083  7167 consensus_queue.cc:1175] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" responder_term: 10 status { last_received { term: 10 index: 6 } last_committed_idx: 6 last_received_current_leader { term: 10 index: 6 } }
I20260709 13:02:05.962273  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:05.958262  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:05.964380  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:05.965183  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:05.968879  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2852) took 1833 ms. Trace:
I20260709 13:02:05.969309  1691 rpcz_store.cc:276] 0709 13:02:04.135750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.459391 (+323641us) service_pool.cc:224] Handling call
0709 13:02:04.459779 (+   388us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:04.460216 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:04.460245 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:04.460845 (+   600us) log.cc:844] Serialized 52 byte log entry
0709 13:02:04.550116 (+ 89271us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:04.627079 (+ 76963us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.628080 (+  1001us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.946956 (+318876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:04.947035 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:05.224560 (+277525us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b18003663a0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:02:05.231763 (+  7203us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b18003663a0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:02:05.818859 (+587096us) spinlock_profiling.cc:243] Waited 5.53 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:05.968652 (+149793us) raft_consensus.cc:1685] finished
0709 13:02:05.968676 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:05.968820 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":78562,"prepare.queue_time_us":163609,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":162973,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":325,"spinlock_wait_cycles":123187072,"thread_start_us":407164,"threads_started":2,"wal-append.queue_time_us":196889}
I20260709 13:02:05.973592  6922 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:05.974224  6922 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:05.964192  6653 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:02:05.974653  6922 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:05.975705  1612 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:05.976500  1618 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:02:05.977804  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.978751  1612 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:02:05.955792  7148 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:02:05.979691  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:05.980522  1612 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:05.980906  6796 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:05.949120  7179 raft_consensus.cc:2764] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:02:05.981313  7170 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:05.984720  1539 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:05.986703  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:05.994920  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:05.999867  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 193958us
I20260709 13:02:05.998872  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:06.000735  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:06.001070  1509 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:06.001578  2228 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:05.929593  6762 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:06.007402  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50942.3us
I20260709 13:02:05.960595  7112 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:06.010263  1507 leader_election.cc:400] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15.
I20260709 13:02:06.010874  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:05.966619  6872 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:05.989869  7150 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:06.020841  1583 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:05.990085  7177 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:06.026161  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:06.026459  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:05.998726  6824 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:06.028733  1583 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:06.029790  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:06.030076  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.030642  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:06.033278  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:06.032503  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:06.033926  1685 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:06.007052  6653 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:06.036914  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:06.038566  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:06.039112  1539 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:02:06.041062  1613 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:02:06.011754  7112 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:06.017551  6675 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:06.043679  1540 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:06.032362  7141 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:06.048210  7141 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.049503  6984 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:06.050356  6984 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:05.998196  7110 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.038420  6859 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:06.049031  7176 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:06.036592  7140 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:06.055399  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 47123.9us
I20260709 13:02:06.037420  6653 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.056295  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:06.072710  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:06.074385  1507 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:06.047209  6973 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:06.078698  2230 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:06.045734  6675 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.046921  7112 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:06.032814  6796 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:02:06.048614  7141 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:06.050755  6984 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.048319  7189 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:06.080273  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:06.057317  7140 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.041347  6872 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.036437  7179 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:02:06.057740  7176 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:06.104553  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:06.079435  6973 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.096168  7140 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:06.085901  6675 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:06.115626  1685 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:06.074579  6653 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:06.106465  6984 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:06.101337  6855 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.114477  6973 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.106252  7176 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:02:06.151345  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 123512us
I20260709 13:02:06.162061  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:06.162753  2235 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
W20260709 13:02:06.165249  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 134682us
I20260709 13:02:06.165853  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:06.166335  1583 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:06.166623  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:06.135581  6601 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:06.136637  7154 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.171813  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:06.172326  2235 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:06.141759  7181 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:06.186023  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 110798us
I20260709 13:02:06.166965  7179 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:06.190080  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:06.185844  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:06.191618  2231 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:06.134972  7150 raft_consensus.cc:697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:06.170969  7178 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:06.189010  7107 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.677s	user 0.006s	sys 0.006s
I20260709 13:02:06.228047  6922 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.229458  2228 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 14: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:06.190871  7154 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:06.190449  7179 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:06.196499  7184 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 13:02:06.237757  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 47757.6us
I20260709 13:02:06.236620  7195 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:06.239063  7182 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:06.254647  2235 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:06.200287  6855 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:06.221207  6762 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.231679  6922 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:06.260558  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 204328us
I20260709 13:02:06.263082  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:06.197261  6601 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:02:06.261845  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1136) took 2187 ms. Trace:
I20260709 13:02:06.274856  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:06.196090  7181 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.257566  7182 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:06.284276  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:06.261052  7154 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.288177  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.240219  7107 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.279253  7178 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:06.276839  1687 rpcz_store.cc:276] 0709 13:02:04.071661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.082606 (+ 10945us) service_pool.cc:224] Handling call
0709 13:02:04.294569 (+211963us) spinlock_profiling.cc:243] Waited 5.25 ms on lock 0x7b5800010e98. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:02:04.294846 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:04.483150 (+188304us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:04.483486 (+   336us) log.cc:844] Serialized 14 byte log entry
0709 13:02:04.911947 (+428461us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:04.912232 (+   285us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:04.912289 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.258379 (+1346090us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54000749d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:06.259294 (+   915us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.259410 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9006,"prepare.queue_time_us":648,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":3986,"raft.queue_time_us":86937,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":19673,"spinlock_wait_cycles":444576000,"thread_start_us":395672,"threads_started":1,"wal-append.queue_time_us":527985}
I20260709 13:02:06.291054  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.291352  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.302737  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.303215  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.302897  7179 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.303647  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:06.300697  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:06.305079  7196 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:06.304993  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.305824  7196 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.307199  7107 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.307965  7196 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:06.309267  7192 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 13:02:06.304335  7179 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:06.310425  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:06.309569  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:06.311844  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:06.313189  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.285212  7182 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:06.314302  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.314821  7181 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:06.315492  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.317001  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.302853  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:06.284889  6855 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:06.304298  7191 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:02:06.287213  6956 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:06.327003  1508 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:06.281514  7110 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:06.322443  1540 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
W20260709 13:02:06.305018  7190 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:02:06.299293  7178 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.318719  1534 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:02:06.286554  6872 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:06.324553  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.280473  7150 consensus_queue.cc:237] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.329501  6855 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.346249  6675 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.364997  6855 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:06.348802  7152 consensus_meta.cc:287] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.654s	user 0.008s	sys 0.003s
I20260709 13:02:06.365518  6675 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:06.365852  7152 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.367187  1537 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:02:06.366946  6984 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.353298  1508 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:06.341764  7188 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:02:06.353953  2229 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:06.359257  7139 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.360226  1536 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:06.329963  6956 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:06.361157  7178 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:06.369201  6984 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.290920  7154 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.377364  1541 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:02:06.338124  7186 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.184s	user 0.001s	sys 0.001s
I20260709 13:02:06.374015  2235 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:06.382109  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:06.349349  7205 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:06.369261  7152 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.381245  6956 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:06.395546  7204 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:06.396075  7204 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.396695  7204 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:06.397586  6783 raft_consensus.cc:3055] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:06.398027  6783 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:06.398111  6653 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.399128  6783 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.400508  6653 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:06.401177  7181 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:06.403219  1688 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:02:06.395999  6824 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:06.406067  6824 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.407176  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:06.408439  1584 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:06.409709  7200 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:06.400547  6783 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:06.410501  7200 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:02:06.412528  1584 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:06.416759  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:06.420552  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:06.422343  1584 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:06.388643  7205 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:06.429742  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 100720us
I20260709 13:02:06.433588  7199 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:06.436973  7139 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:06.419124  7181 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:06.422693  6872 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:06.423475  7200 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:06.446020  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:06.394626  7112 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.449812  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:06.426069  7205 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:06.414558  7182 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.452898  6601 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:06.477618  6601 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.478070  6601 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:06.452098  7170 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 13:02:06.478802  7170 raft_consensus.cc:3055] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:06.479136  7170 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:06.480052  7170 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.453095  7202 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:06.460260  6973 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.462451  6956 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.462872  7112 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:06.465891  7144 consensus_meta.cc:287] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.579s	user 0.009s	sys 0.003s
I20260709 13:02:06.467020  7154 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.471156  7204 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.476748  6824 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:06.482251  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 56001.2us
W20260709 13:02:06.488173  1689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1966) took 2290 ms (client timeout 2852 ms). Trace:
I20260709 13:02:06.492883  7144 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:06.519050  1689 rpcz_store.cc:269] 0709 13:02:04.193944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.442444 (+248500us) service_pool.cc:224] Handling call
0709 13:02:06.484090 (+2041646us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b540041e6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:06.484233 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1144827776}
I20260709 13:02:06.500098  6601 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.500869  7141 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.493125  7170 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:06.505148  7182 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.507201  6973 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.531750  7140 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.540880  7140 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.511109  6956 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:06.275586  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 238765us
I20260709 13:02:06.543390  1509 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:06.497786  6835 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:06.521275  7205 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.523178  1610 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:06.524827  6824 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:06.453541  7176 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.529660  7144 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.545882  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:06.544210  6835 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:06.547286  6835 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:06.548507  6835 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:06.528283  7066 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:06.509073  7204 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.551589  1509 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:06.554085  7088 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:06.554147  6748 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:06.556491  1509 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:06.554819  6748 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:06.557063  7204 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:06.559718  6748 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:06.560084  7204 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:06.560303  7066 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:06.560801  7066 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:06.554874  6666 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:06.561708  6666 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:06.562944  7176 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:06.562620  6666 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.556437  6601 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:06.569070  7141 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.574124  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:06.575623  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:06.575914  2228 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:06.578231  7154 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.578806  7154 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.579084  6748 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.582957  7154 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.584425  6877 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:06.584923  6877 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.585577  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:06.587313  7066 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:06.590087  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.591063  7205 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.594514  6877 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.561116  7204 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.597496  7144 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:06.597798  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.581225  6748 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.599011  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:06.600284  6889 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:06.600736  7140 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:06.602486  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:06.603914  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:06.605249  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.630115  1616 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:06.605579  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.630270  1613 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:06.638729  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.639977  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:06.630906  6922 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:06.642125  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:06.643252  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:06.644718  1609 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:06.641287  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.652133  1613 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:06.650247  7177 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:06.658567  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:06.659058  1613 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:02:06.660983  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.662946  1613 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:06.662854  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:06.665126  1618 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:06.665966  7074 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:06.666433  7074 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:06.667316  7074 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.668607  7074 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.670701  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:06.667889  6877 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:06.671581  6748 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:06.671871  1507 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:06.672058  6748 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:06.672706  6748 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.674220  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:06.675037  2234 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:06.675225  6762 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:06.675463  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1438) took 1501 ms. Trace:
I20260709 13:02:06.675812  2234 rpcz_store.cc:276] 0709 13:02:05.174072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.192641 (+ 18569us) service_pool.cc:224] Handling call
0709 13:02:06.674054 (+1481413us) spinlock_profiling.cc:243] Waited 6.14 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ab20a0 00007fe094ab1984 00007fe08e02c6d4 00007fe08e02c662
0709 13:02:06.675388 (+  1334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14123904}
I20260709 13:02:06.677882  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.663681  1615 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:06.682812  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.683656  1611 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:06.658601  7177 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:06.616622  7144 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:06.708773  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.709849  1615 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:06.631366  6889 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.638841  7140 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.714444  6938 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:06.715134  6938 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.717245  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:06.717805  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 133397us
I20260709 13:02:06.650532  7178 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.720777  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.721451  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:06.726644  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:06.727617  1686 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:06.713604  6835 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:06.734205  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:06.735242  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:06.684826  7189 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:02:06.736472  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:06.742702  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:06.743338  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.743510  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:06.744064  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:06.745436  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:06.722226  6877 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:06.742697  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.751854  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.768893  1691 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 13:02:06.767752  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.743528  1688 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:06.769668  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:06.770581  1687 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:02:06.728189  7176 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:06.775487  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:06.781548  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:06.868775  1686 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.782410  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:06.783444  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.870275  1541 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:06.728531  6938 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.785275  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:06.774674  6835 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.789613  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.789271  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:06.790231  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:06.789889  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:06.789155  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:06.798238  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 53807.4us
I20260709 13:02:06.816386  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.776875  7176 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:06.753367  6855 consensus_queue.cc:1059] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:02:06.891335  1538 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:02:06.866475  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 66264us
I20260709 13:02:06.777530  1690 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:06.729400  6824 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:06.777089  1539 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:02:06.772449  6838 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.068s
I20260709 13:02:06.768496  6922 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:06.781066  7155 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:06.875131  6829 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:02:06.908776  7155 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:06.908985  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:06.909802  7155 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.912365  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:06.913184  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:06.918462  7196 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:06.919075  7196 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.919466  7152 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:06.919919  7152 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:06.922287  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1064) took 1035 ms. Trace:
I20260709 13:02:06.922797  1537 rpcz_store.cc:276] 0709 13:02:05.886997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.887194 (+   197us) service_pool.cc:224] Handling call
0709 13:02:05.907901 (+ 20707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:06.367637 (+459736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.921956 (+554319us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b540030a7d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:06.922210 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1055658880}
W20260709 13:02:06.927248  7164 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.256s	user 0.000s	sys 0.004s
I20260709 13:02:06.918488  7178 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:06.930998  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:06.931581  7220 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:02:06.868189  1534 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:06.873741  1536 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:06.931049  1683 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:02:06.936472  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 937) took 1141 ms. Trace:
I20260709 13:02:06.936848  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1985) took 1135 ms. Trace:
I20260709 13:02:06.875602  1614 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:06.781015  7088 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 13:02:06.937352  1683 rpcz_store.cc:276] 0709 13:02:05.801677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.821927 (+ 20250us) service_pool.cc:224] Handling call
0709 13:02:06.930857 (+1108930us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54000030a0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa63dc0 00007fe08faefe18 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6
0709 13:02:06.936791 (+  5934us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":311487872}
I20260709 13:02:06.868227  1688 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:06.938370  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.938596  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:06.939152  1614 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:06.939280  6889 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.939394  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:06.939975  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:06.785478  7210 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.122s
I20260709 13:02:06.940889  1535 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:06.905470  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:06.941219  7144 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:06.910279  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:06.913789  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:06.914090  2232 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1428) took 2256 ms (client timeout 1908 ms). Trace:
I20260709 13:02:06.942020  7140 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:06.942196  2232 rpcz_store.cc:269] 0709 13:02:04.658106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:04.658333 (+   227us) service_pool.cc:224] Handling call
0709 13:02:06.913880 (+2255547us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400700154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:06.913995 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2404035968}
I20260709 13:02:06.938381  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:06.915599  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:06.945142  1691 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:02:06.916303  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2520) took 1034 ms. Trace:
I20260709 13:02:06.945951  1612 rpcz_store.cc:276] 0709 13:02:05.881716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.096789 (+215073us) service_pool.cc:224] Handling call
0709 13:02:06.097612 (+   823us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:06.107652 (+ 10040us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:06.107690 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:06.599962 (+492272us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b2c0005b3b8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:02:06.600404 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 13:02:06.683535 (+ 83131us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:06.683586 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.683686 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:06.916001 (+232315us) raft_consensus.cc:1685] finished
0709 13:02:06.916023 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:06.916229 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":817,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":25725,"spinlock_wait_cycles":6201088,"thread_start_us":82703,"threads_started":1,"wal-append.queue_time_us":129170}
I20260709 13:02:06.916983  1582 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:06.940047  1688 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.951442  1613 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:02:06.914714  6855 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:02:06.933152  6835 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:06.905788  6824 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.936892  1610 rpcz_store.cc:276] 0709 13:02:05.794838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.795076 (+   238us) service_pool.cc:224] Handling call
0709 13:02:05.795454 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:06.545077 (+749623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:06.936257 (+391180us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b54001a63d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:06.936414 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1673040128}
I20260709 13:02:06.955302  6824 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:06.955955  6872 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.956501  7141 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.956490  6872 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:06.958235  7214 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:06.959127  6872 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:06.893884  7176 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:06.961824  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:06.962296  6996 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:06.962970  7163 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.963526  7163 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:06.965644  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 72050.1us
I20260709 13:02:06.966454  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:06.967638  1657 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:06.920856  7152 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.968638  7152 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:06.970532  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.970717  1657 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:06.971395  1542 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:06.971840  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:06.972463  1535 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:06.958177  1582 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:06.974673  6976 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:06.975020  1582 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:06.975062  6976 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:06.976343  6976 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:06.976701  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:06.977624  7200 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:06.978300  6556 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:06.978729  6556 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:06.979101  6556 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:06.981667  7144 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:06.957083  7141 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.984642  7141 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.944429  1543 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:06.985165  1688 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:06.985931  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:06.989948  7188 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.597s	user 0.007s	sys 0.000s
I20260709 13:02:06.940203  1614 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:06.995349  7163 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:06.933307  6748 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:06.997253  1539 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:06.998153  6748 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:06.999966  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:07.003146  1584 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:07.004714  2235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "12adfb7855f340329f110249c207f409"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:07.005523  2235 raft_consensus.cc:606] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:02:07.005618  7074 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:07.005892  2235 raft_consensus.cc:614] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:07.006022  7074 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.006397  7074 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:06.944332  7196 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:07.009577  7218 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.225s	user 0.000s	sys 0.009s
I20260709 13:02:07.008832  1684 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:06.982257  7144 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.011435  7160 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.012154  7214 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:07.011960  7160 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:07.019523  7193 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.083s	user 0.002s	sys 0.000s
I20260709 13:02:07.013118  7144 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:07.022003  7225 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.091s	user 0.000s	sys 0.002s
I20260709 13:02:07.022982  7160 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:07.023375  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:07.024120  1683 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
W20260709 13:02:07.025151  7221 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.236s	user 0.000s	sys 0.016s
I20260709 13:02:07.025183  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:07.027199  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:07.027290  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:07.028045  1691 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:02:07.028492  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:07.028610  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:06.874236  1507 leader_election.cc:336] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.681s (SENT)
I20260709 13:02:07.028887  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:07.029655  1507 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:07.029932  7214 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:07.030412  1691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "532fefba367345759887da9fcc9872e3"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:02:07.031000  1691 raft_consensus.cc:606] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:07.030447  7214 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:07.031383  1691 raft_consensus.cc:614] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:07.031980  1692 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
W20260709 13:02:07.018555  7224 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.089s	user 0.003s	sys 0.000s
I20260709 13:02:07.029341  1687 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:07.032895  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:06.867772  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 84401.2us
I20260709 13:02:06.867120  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:06.989584  7176 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:07.035792  1615 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:07.036800  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:07.038643  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.039109  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.042920  1686 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
W20260709 13:02:07.048182  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 181132us
W20260709 13:02:07.050750  1684 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1978) took 1858 ms (client timeout 1681 ms). Trace:
W20260709 13:02:07.051250  1684 rpcz_store.cc:269] 0709 13:02:05.192699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:05.196703 (+  4004us) service_pool.cc:224] Handling call
0709 13:02:07.050601 (+1853898us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54002cd654. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:07.050690 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3898416000}
I20260709 13:02:07.053237  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:07.049966  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:07.055332  1683 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:07.057765  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.060487  2230 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:07.064011  1540 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:02:07.065100  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:07.059177  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.066840  2231 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:07.066699  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.067653  2237 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:07.058557  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:07.068220  2234 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:07.066471  2229 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:07.069388  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:07.071029  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:07.071493  1615 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:07.071659  2230 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:07.073884  1543 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:07.075344  1617 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:07.075654  2233 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:07.035615  6835 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:07.035447  6812 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:07.035742  6824 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:07.077728  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 107074us
I20260709 13:02:07.031653  7214 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:07.049348  7144 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:07.054841  7154 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:07.085193  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:07.085579  1542 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:07.086313  1535 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:07.056332  7167 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:07.089556  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:07.091403  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:07.091620  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 217394us
I20260709 13:02:07.117574  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:07.118338  2233 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:07.120028  1683 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:02:07.120637  1683 raft_consensus.cc:606] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:02:07.121006  1683 raft_consensus.cc:614] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:07.122823  2234 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:07.083280  6748 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:07.117564  1541 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:07.113348  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:07.126029  2231 raft_consensus.cc:2410] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 14 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:02:07.077607  7170 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:02:07.128831  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.129487  1583 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.119083  7167 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:07.134177  7167 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:07.134758  7176 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:07.135537  6877 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:07.136618  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:07.137050  6976 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:07.138583  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.139374  6976 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:07.111367  7197 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.140369  7197 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.140826  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.139374  2237 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:07.142194  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.067937  6796 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:02:07.142578  1657 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.144523  1692 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:07.144726  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:07.142683  1583 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.066434  7143 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 13:02:07.143560  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:07.144068  2236 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:07.089035  7160 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:07.129655  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:07.104058  7154 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.147151  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.149559  1691 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:07.150090  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:07.092468  6872 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.112042  7144 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:07.163290  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.159452  1657 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.159654  6973 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:07.157666  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:07.153288  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.155938  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:07.165051  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.173671  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:07.159452  1583 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.151818  7141 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:07.173931  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:07.188768  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:07.147833  6976 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:07.162545  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.185540  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.161082  7154 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:07.189636  1583 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.189684  2230 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 15 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:02:07.191772  2233 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:07.192466  1507 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:07.162366  6835 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:07.160992  6860 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:07.171554  6973 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:07.194432  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:07.175232  6838 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:07.155311  7177 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:07.147508  6812 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:02:07.195801  2234 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:02:07.224630  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 59678.1us
I20260709 13:02:07.229775  1657 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.197592  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.206844  1507 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:07.195091  1658 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:07.166582  6872 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:07.204768  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:07.191109  7182 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:07.162536  7064 consensus_queue.cc:1059] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:02:07.175860  7169 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 13:02:07.235867  1584 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:07.239110  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.239415  1507 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:07.218539  6976 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:07.250118  7174 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:07.251302  7174 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:07.251700  7174 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:07.251894  6556 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:07.205621  6984 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:02:07.227941  6973 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:07.235232  7112 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:07.248342  7230 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:07.277096  2231 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:07.252533  7174 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:07.253708  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:07.254719  7177 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:07.193944  6889 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:02:07.228343  7199 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.257346  1541 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:02:07.222817  6860 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.261248  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:07.260778  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 65800us
I20260709 13:02:07.259963  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:07.234818  7140 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:07.247314  7205 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:07.270265  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:07.265563  7141 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:07.196847  7196 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:02:07.220367  7195 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:02:07.255857  7074 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:07.248335  6829 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:02:07.290278  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 558114us
I20260709 13:02:07.313273  6860 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:07.238831  7066 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:07.322839  1543 raft_consensus.cc:2393] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:07.283202  2201 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:07.236819  7144 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.312090  7230 consensus_queue.cc:1243] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:07.335577  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:07.354840  6796 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:07.356343  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:07.285269  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:07.339931  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:07.346411  1612 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:02:07.278949  7160 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:07.322827  7112 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 13:02:07.330546  7174 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:07.331244  7140 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:07.351805  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68684.3us
I20260709 13:02:07.352918  1617 raft_consensus.cc:2368] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14.
I20260709 13:02:07.322006  7210 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:07.283126  7197 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:07.357220  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:07.373220  1618 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:07.373740  2201 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:07.375118  6601 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:07.375541  6601 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:07.375674  7210 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:07.376003  6601 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:07.311244  7199 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.376750  7210 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:07.357331  7228 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
W20260709 13:02:07.380434  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 95274.6us
W20260709 13:02:07.347453  7204 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.014s	sys 0.024s
I20260709 13:02:07.375953  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:07.397761  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 264248us
W20260709 13:02:07.399425  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:07.393358  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 163666us
I20260709 13:02:07.393052  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:07.401867  1657 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.362125  7160 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:07.364601  7177 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:07.404328  1618 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:07.414125  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.420137  1508 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:07.398770  7204 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.015s	sys 0.025s
I20260709 13:02:07.385035  7176 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:07.387729  7232 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:07.433552  1657 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:07.482039  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:07.421308  7177 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:07.382778  7140 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:07.426606  6762 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:07.485216  2231 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:07.421787  7150 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:07.422055  6796 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:07.482972  1543 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:07.490068  1618 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:02:07.432216  7231 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:07.477455  6835 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:07.484529  7195 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:02:07.516885  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 189266us
I20260709 13:02:07.425005  7112 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:02:07.486405  6829 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:07.425359  7170 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:02:07.478933  6783 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:02:07.489675  7232 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:07.527938  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:07.499147  6762 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:07.525126  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:07.477588  6922 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 13:02:07.508081  6872 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:07.479110  6993 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:02:07.504966  7200 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:07.522631  7152 consensus_meta.cc:287] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.554s	user 0.009s	sys 0.004s
I20260709 13:02:07.483502  6877 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:02:07.594873  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:07.584509  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 151067us
W20260709 13:02:07.586694  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 58870.7us
I20260709 13:02:07.598250  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:07.595604  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 172232us
I20260709 13:02:07.485015  7167 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.603055  1583 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.604019  7182 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:07.604512  7182 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:07.606561  7227 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.447s	user 0.005s	sys 0.006s
I20260709 13:02:07.608779  7182 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.617080  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:07.577842  6829 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:07.583653  7107 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:07.582902  6762 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:07.532042  6796 consensus_queue.cc:1059] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 13:02:07.633054  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:07.593065  6996 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:07.636715  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:07.637823  1692 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:02:07.648623  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:02:07.601945  7155 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.005s	sys 0.024s
I20260709 13:02:07.658172  7195 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
W20260709 13:02:07.675658  7229 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.211s	user 0.001s	sys 0.000s
W20260709 13:02:07.677265  7234 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 13:02:07.678757  7233 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
W20260709 13:02:07.680181  7235 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:02:07.632194  7107 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:02:07.681692  7239 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:02:07.682240  7107 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:07.702702  7107 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:07.702798  7248 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:02:07.534603  7160 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.685009  6872 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:07.678795  7167 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:07.704648  7160 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:07.707661  6835 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:07.708313  6835 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:07.718720  7252 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:07.729439  7088 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:07.712745  7232 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.737061  6829 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.738745  6556 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:07.743211  1690 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:49352
W20260709 13:02:07.671391  7155 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.006s	sys 0.025s
I20260709 13:02:07.672529  7244 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:07.751039  7244 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.752143  7178 raft_consensus.cc:3055] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:07.752776  7178 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:07.656522  7243 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:07.597275  7200 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.775301  7178 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:07.785903  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 187753us
W20260709 13:02:07.788339  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 368265us
I20260709 13:02:07.789384  1508 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:07.787199  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 138677us
I20260709 13:02:07.793356  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:07.786811  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:07.797588  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.797013  7243 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:02:07.807813  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.806233  7257 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.803082  1687 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:07.808965  7261 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.809809  7261 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:07.807662  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 282607us
W20260709 13:02:07.812429  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 209487us
I20260709 13:02:07.817687  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:07.818316  1614 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:02:07.820314  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:07.820674  1684 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:02:07.822734  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:07.825277  6973 consensus_meta.cc:287] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.546s	user 0.003s	sys 0.015s
I20260709 13:02:07.825809  6973 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.826779  1537 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:07.827651  7140 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:07.828081  6973 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:07.828281  6601 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.829484  2235 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:07.808929  7257 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.833139  7257 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:07.800648  7152 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:07.834545  1691 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:07.811954  7261 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:07.791052  7178 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:07.839658  2229 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:02:07.837715  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44875.3us
I20260709 13:02:07.839150  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:07.838553  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:07.838181  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.847152  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:07.848191  1687 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:07.852684  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.853116  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:07.845409  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:07.857381  1508 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.858982  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:07.859664  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:07.837409  6976 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.588s	user 0.013s	sys 0.007s
I20260709 13:02:07.860404  1688 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:07.859687  2230 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:07.862303  1614 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:07.833994  6601 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:07.872444  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:07.875128  1508 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.872054  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:07.872983  1609 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:07.838255  7167 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:07.844165  6872 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:07.862978  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:07.839030  7200 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:07.838063  7152 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:07.876374  7253 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:07.855611  7140 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.872349  6996 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:07.880223  1684 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:07.893772  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:07.893586  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:07.876986  6976 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:07.842898  7155 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:02:07.838605  7251 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:07.878453  1508 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.861550  7252 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:07.860728  1658 leader_election.cc:400] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14.
I20260709 13:02:07.876794  7189 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:07.880992  7246 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:07.856781  7144 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:07.900854  1658 leader_election.cc:403] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:07.906575  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:07.923130  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:07.928368  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:07.929283  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.930855  1657 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:07.931039  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:07.934779  7163 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:02:07.936586  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:07.938293  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:07.938661  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:07.942618  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:07.943395  1537 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 11: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:07.945767  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:07.897698  6996 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:07.952620  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 92014.7us
I20260709 13:02:07.967363  1618 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:07.970010  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:07.970793  1618 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:07.982849  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:07.984254  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 90750.3us
I20260709 13:02:07.922583  7253 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:07.988456  1540 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:07.892009  7231 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:02:07.893695  7140 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:07.948415  7200 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:07.952263  6556 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:08.005210  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:07.948534  6976 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:02:07.954671  6922 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:08.019518  6922 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.019985  6922 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:08.023291  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:07.989435  7150 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 13:02:07.989177  7230 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:08.004458  7251 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:02:08.026228  6976 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:08.026546  7214 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.902s	user 0.008s	sys 0.008s
W20260709 13:02:08.004837  6860 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.651s	user 0.007s	sys 0.006s
I20260709 13:02:08.027098  7214 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:08.027215  6860 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.027509  6976 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.010615  7262 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.011446  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:08.012130  1614 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:02:08.017736  6996 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.030905  1613 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:07.949986  6877 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 13:02:07.957409  7112 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:07.957231  6601 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:07.951532  7144 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.012117  7140 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:07.958933  6829 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:02:08.044189  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:08.050592  7255 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.174s
I20260709 13:02:08.052404  1538 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:08.053535  7244 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:07.928053  7252 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.054685  6938 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:02:08.059829  1540 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:02:08.033514  7154 consensus_meta.cc:287] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.806s	user 0.012s	sys 0.003s
I20260709 13:02:08.065757  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:08.066787  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:07.928766  7246 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.029307  6860 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:08.036131  6601 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.029307  7214 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:08.045938  7262 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.087815  7262 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:08.043305  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 57807.2us
I20260709 13:02:08.096000  1507 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:08.096989  7088 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:08.055183  6938 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:08.052758  7170 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:08.101260  1534 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 10. Current term is 11. Ops: []
I20260709 13:02:08.110762  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:08.112665  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:08.113638  1537 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:08.114248  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:08.085407  6601 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:08.069720  7155 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:08.143761  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:08.066033  7243 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:08.146870  2237 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:08.149912  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:08.150897  1688 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:08.103713  7088 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:08.097332  6984 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:08.156589  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:08.036734  7152 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.112656  7170 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:08.072656  7244 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:02:08.169718  2201 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:08.145370  6666 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:08.147181  7066 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:08.175725  7282 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:08.180368  7285 consensus_queue.cc:1059] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:02:08.182705  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 249634us
I20260709 13:02:08.184314  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:08.074810  7154 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:08.185552  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 232346us
I20260709 13:02:08.170763  6860 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:08.186265  7205 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.189119  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:08.190140  6666 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:08.201409  1657 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:08.198428  2230 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:08.166080  7254 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.201737  7282 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.207185  7254 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.204352  7205 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.166543  7232 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:02:08.209707  7205 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:08.209896  7214 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:08.210532  7214 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.196704  7280 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:02:08.206760  6860 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.211937  6922 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:08.213496  7174 consensus_meta.cc:287] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.833s	user 0.019s	sys 0.009s
I20260709 13:02:08.214211  7140 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.215008  7140 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.214282  1541 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
W20260709 13:02:08.217012  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.204114  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 60436.5us
I20260709 13:02:08.214425  7197 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.220180  1507 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:08.199220  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:08.221403  1657 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:08.223747  1657 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:08.197767  7277 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:08.225215  1540 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 13:02:08.200536  7244 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:08.230108  2230 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:08.214267  6877 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:08.234591  1583 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:08.155997  7243 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.235441  2237 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:08.272360  2237 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:08.273202  2237 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.273804  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:08.274844  2232 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:08.236523  1687 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:08.237214  7252 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:08.224932  7088 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.229373  7205 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:08.234457  6877 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:08.271215  7271 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:02:08.238888  6666 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.239658  1536 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:08.241595  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:08.245401  7140 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:08.247169  7154 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:08.247720  7141 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:08.248411  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:08.250406  1691 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:02:08.251971  6835 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.543s	user 0.011s	sys 0.007s
I20260709 13:02:08.252574  7182 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:08.253721  7170 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.255549  7246 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:08.256253  6748 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:02:08.257678  6762 consensus_meta.cc:287] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.618s	user 0.007s	sys 0.007s
I20260709 13:02:08.258085  7144 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:08.235458  7244 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:08.225634  7197 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.324744  7197 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:08.260555  7238 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.625s	user 0.000s	sys 0.002s
I20260709 13:02:08.261503  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:08.262359  6973 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:08.263186  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:08.331455  2235 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:08.264724  1683 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:02:08.261570  2228 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 13:02:08.266428  7107 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.563s	user 0.016s	sys 0.001s
I20260709 13:02:08.200948  6938 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.267047  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:08.343889  1692 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:08.267740  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:08.271117  6993 raft_consensus.cc:3055] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:02:08.271802  7160 consensus_meta.cc:287] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.566s	user 0.018s	sys 0.002s
I20260709 13:02:08.276767  1684 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:08.280821  2230 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:08.273764  7243 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:08.287058  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:08.349757  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:08.287340  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:08.350540  1618 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:08.288865  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:08.350904  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:08.351437  1611 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
W20260709 13:02:08.291240  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 65317.4us
I20260709 13:02:08.267741  6922 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:08.352243  1692 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:08.291934  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 85503.1us
I20260709 13:02:08.353183  2203 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:08.354697  7262 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:08.249234  6556 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:08.355098  7262 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.355486  7262 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:08.357126  7264 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:08.294549  1610 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:02:08.294570  1688 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:02:08.286841  6877 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:08.318467  2232 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:08.320891  6762 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.360909  6762 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:08.294127  7285 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:08.361723  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:08.322270  7244 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.362426  1610 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:08.362856  7244 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:08.366030  1543 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:08.366631  6877 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.367954  2233 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:08.369490  7169 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:02:08.302541  6666 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:08.298789  7150 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:08.300343  7276 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:08.301914  6838 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:08.338162  7152 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.334779  1683 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:02:08.370697  1537 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:08.370671  7169 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:08.299733  6835 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:08.385036  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 161352us
I20260709 13:02:08.342108  2201 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:08.418849  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 76860.2us
I20260709 13:02:08.419356  1657 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:08.387684  6877 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:08.388800  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1157) took 1404 ms. Trace:
I20260709 13:02:08.395247  6829 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14.
I20260709 13:02:08.397217  6748 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.398978  7173 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:08.346850  6973 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:08.346171  6993 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:08.417574  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 65128us
I20260709 13:02:08.420529  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:08.418545  7169 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:08.438856  1686 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:08.342854  6601 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.386229  6938 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:08.386559  7276 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:08.431847  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.430207  6800 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:08.424953  1689 rpcz_store.cc:276] 0709 13:02:06.984668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.989597 (+  4929us) service_pool.cc:224] Handling call
0709 13:02:07.025116 (+ 35519us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b5800010e98. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:02:07.025322 (+   206us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:07.046617 (+ 21295us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:07.046647 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:07.055478 (+  8831us) log.cc:844] Serialized 26 byte log entry
0709 13:02:07.474041 (+418563us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:07.474134 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:07.475788 (+  1654us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:08.259974 (+784186us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:08.388561 (+128587us) raft_consensus.cc:1685] finished
0709 13:02:08.388588 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:08.388740 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20762,"raft.queue_time_us":21432,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":10800,"spinlock_wait_cycles":76848256,"thread_start_us":379837,"threads_started":1,"wal-append.queue_time_us":449426}
I20260709 13:02:08.421689  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:08.455502  1683 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:08.419487  7152 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.437562  7174 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:08.443131  7144 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:08.444701  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.432266  1692 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:08.456266  1657 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:08.466398  1610 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:08.468441  6838 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:02:08.470253  6838 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:08.471903  7144 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.474334  7144 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:08.475721  7232 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:08.477547  7174 raft_consensus.cc:697] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:08.478142  7074 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:08.434008  7173 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:08.478538  7174 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.481508  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:08.435653  7244 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:08.462809  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:08.487174  6993 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.488011  7199 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:08.491886  2232 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:08.432678  7141 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:02:08.494477  1537 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:08.505450  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:08.436237  6748 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:08.508692  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:08.509461  2230 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:08.483875  7244 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:08.545279  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:08.547331  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:08.548211  7293 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 13:02:08.511261  7237 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:02:08.483654  6800 raft_consensus.cc:697] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:08.510303  7231 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:08.540800  7197 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:08.573925  1688 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:02:08.558580  7244 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:08.539491  7195 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 13:02:08.489131  7276 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.541504  7232 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:02:08.566833  7231 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:08.578630  6800 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:08.594530  7283 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:08.595494  7276 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:08.479820  7173 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.588704  7205 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:08.598505  7231 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:08.616039  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.553776  6973 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.602741  7173 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:08.583930  7197 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.619971  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:08.623570  1610 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:02:08.625519  7200 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:08.626045  7200 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:08.628830  7154 raft_consensus.cc:3055] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:08.629292  7154 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:08.630177  7154 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.631933  7154 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:08.632412  6993 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:08.554347  7169 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:08.636600  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:08.640997  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.621105  6835 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:08.650274  6748 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:08.618606  7205 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.688349  7152 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:08.691025  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:08.750732  1610 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:08.763724  1683 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:08.741065  7281 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:08.763309  6835 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:08.813884  1685 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 1999) took 2079 ms (client timeout 1725 ms). Trace:
W20260709 13:02:08.817970  1685 rpcz_store.cc:269] 0709 13:02:06.734518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.734634 (+   116us) service_pool.cc:224] Handling call
0709 13:02:08.813547 (+2078913us) spinlock_profiling.cc:243] Waited 984 ms on lock 0x7b540028e6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:08.813771 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2262941440}
I20260709 13:02:08.762977  7074 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:08.762789  6877 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:08.819362  1536 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2522) took 1551 ms (client timeout 1724 ms). Trace:
I20260709 13:02:08.799431  7169 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:08.819837  1536 rpcz_store.cc:269] 0709 13:02:07.259475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.259621 (+   146us) service_pool.cc:224] Handling call
0709 13:02:08.810181 (+1550560us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b540012e3d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:08.810368 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2132084480}
I20260709 13:02:08.823634  7144 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.815014  7281 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.830133  6877 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.825374  6835 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.829955  7074 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.853752  6838 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:08.858204  7278 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.890239  7254 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:08.892360  7282 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:08.896158  7281 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:08.854460  7144 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.871446  7197 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:08.914244  7177 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.878777  6838 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:08.920578  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.915813  1691 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:02:08.879678  6835 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:08.919253  7262 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.563s	user 0.006s	sys 0.006s
I20260709 13:02:08.931196  7152 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:08.903718  7278 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.986169  7144 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:08.948098  7200 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.990926  7200 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:08.942759  7277 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.667s	user 0.012s	sys 0.000s
I20260709 13:02:08.992169  7277 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:02:08.942292  7182 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.640s	user 0.011s	sys 0.001s
I20260709 13:02:08.948434  7262 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:08.996373  7262 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.001803  6877 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:08.962208  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:08.936821  7288 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:08.947003  6855 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:08.944518  7205 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:08.979034  6838 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:08.987409  7177 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:08.943583  7152 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:09.026082  7278 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:09.030405  6838 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:09.021569  7288 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:02:09.049299  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.049713  7074 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.052002  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2054) took 1207 ms. Trace:
I20260709 13:02:09.052450  1613 rpcz_store.cc:276] 0709 13:02:07.844969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.845123 (+   154us) service_pool.cc:224] Handling call
0709 13:02:07.872060 (+ 26937us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.031329 (+159269us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.051736 (+1020407us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400389854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:09.051937 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364442496}
I20260709 13:02:09.029685  6855 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:09.057168  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 702074us
I20260709 13:02:09.059687  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:09.060374  1612 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
W20260709 13:02:09.062374  1538 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 1075) took 2322 ms (client timeout 2185 ms). Trace:
W20260709 13:02:09.062808  1538 rpcz_store.cc:269] 0709 13:02:06.740224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:06.909993 (+169769us) service_pool.cc:224] Handling call
0709 13:02:09.062142 (+2152149us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b54000e92d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:09.062252 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1515926912}
W20260709 13:02:09.063684  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.064127  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.065480  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:09.066105  1612 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:09.067906  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2007) took 1109 ms. Trace:
I20260709 13:02:09.068311  1684 rpcz_store.cc:276] 0709 13:02:07.958443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.959885 (+  1442us) service_pool.cc:224] Handling call
0709 13:02:08.276514 (+316629us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.348937 (+ 72423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.067717 (+718780us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54000f5850. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:09.067820 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":727374592}
I20260709 13:02:09.070278  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:09.070874  1618 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:09.072978  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.073943  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 3016) took 1130 ms. Trace:
I20260709 13:02:09.074342  1541 rpcz_store.cc:276] 0709 13:02:07.943318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.945542 (+  2224us) service_pool.cc:224] Handling call
0709 13:02:09.073777 (+1128235us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54001e4e54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:09.073889 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":615793408}
I20260709 13:02:09.075057  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:09.075554  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.073814  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:09.076623  1612 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:09.076901  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:09.078243  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:09.078974  1617 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:09.075888  1610 raft_consensus.cc:2410] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:02:09.081713  7214 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.089109  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:09.068729  1614 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:09.093681  6855 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:09.094475  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.096913  7286 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.097545  7286 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.100965  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:09.049877  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.103199  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:09.104044  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:09.104204  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:09.104919  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:09.105167  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.106173  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:09.106823  1684 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:02:09.101692  2233 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:09.114991  1541 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:02:09.118660  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:09.119022  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.125725  1536 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:09.126428  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:09.128052  2230 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:09.128733  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:09.129588  1685 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:09.131498  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.042469  7277 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.127511  2201 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:09.138653  2201 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:09.140318  2201 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:09.136547  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:09.142397  1613 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.145738  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.148623  2229 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:02:09.150703  1614 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 13:02:09.155005  2232 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:02:09.156524  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:09.157207  2230 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:09.108206  7263 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:09.158222  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:09.170269  1542 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:09.118101  7214 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:09.176170  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:09.147728  7244 consensus_meta.cc:287] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.549s	user 0.011s	sys 0.000s
I20260709 13:02:09.130586  6835 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:09.182204  1618 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:09.163672  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.144647  7173 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:09.154503  7197 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:09.198689  1539 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:09.158335  6783 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:09.132198  7205 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 13:02:09.172197  7258 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:02:09.212095  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.248072  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:09.212736  7152 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.153970  7277 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.222287  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1070) took 1119 ms. Trace:
I20260709 13:02:09.226807  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2017) took 1265 ms. Trace:
I20260709 13:02:09.231416  6601 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:09.232573  7308 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:02:09.237542  7281 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:09.239262  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2009) took 1280 ms. Trace:
I20260709 13:02:09.209075  7173 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:09.280190  7173 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:09.242326  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.287503  7112 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.287657  7170 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:09.288192  7112 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:09.288684  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:09.290304  1684 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:02:09.290473  7112 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:09.291514  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:09.292827  6860 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.197916  6835 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.296159  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:09.296882  1684 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:09.299000  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:09.301349  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.302057  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.248772  1618 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:09.195631  6996 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:09.221577  2237 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:09.302354  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.308652  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.181627  7214 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:09.241755  7231 consensus_meta.cc:287] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.618s	user 0.012s	sys 0.000s
I20260709 13:02:09.314973  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:09.244920  7258 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:09.267293  1686 rpcz_store.cc:276] 0709 13:02:07.958942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.241181 (+282239us) service_pool.cc:224] Handling call
0709 13:02:09.239063 (+997882us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b5400374e54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:09.239190 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":349692160}
W20260709 13:02:09.240288  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.249626  6748 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:09.189483  6938 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 13:02:09.252270  7232 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.192816  7244 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.352399  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3015) took 1409 ms. Trace:
I20260709 13:02:09.352815  1535 rpcz_store.cc:276] 0709 13:02:07.942966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.945050 (+  2084us) service_pool.cc:224] Handling call
0709 13:02:07.945347 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.056913 (+111566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.056966 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:08.057021 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:08.057065 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.352165 (+1295100us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400419bd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:09.352214 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.352340 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256004480}
I20260709 13:02:09.249747  7295 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:09.302170  1537 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:09.307056  2234 rpcz_store.cc:276] 0709 13:02:08.102483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.102892 (+   409us) service_pool.cc:224] Handling call
0709 13:02:08.143995 (+ 41103us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:08.152864 (+  8869us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:08.152910 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:08.168540 (+ 15630us) log.cc:844] Serialized 26 byte log entry
0709 13:02:08.264889 (+ 96349us) spinlock_profiling.cc:243] Waited 2.99 ms on lock 0x7b1800396c40. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08f6944bc 00007fe08c9e248b 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1
0709 13:02:08.265045 (+   156us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:08.265080 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:08.265149 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:09.218091 (+952942us) spinlock_profiling.cc:243] Waited 8.74 ms on lock 0x7b1800396c40. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:02:09.222020 (+  3929us) raft_consensus.cc:1685] finished
0709 13:02:09.222058 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.222222 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16095,"raft.queue_time_us":3811,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":8501,"spinlock_wait_cycles":26980352,"thread_start_us":13358,"threads_started":1,"wal-append.queue_time_us":13709}
I20260709 13:02:09.242183  1613 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:09.357266  1582 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:09.173264  7286 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:09.309312  1536 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:09.359263  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:09.311177  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:09.311278  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.311709  6996 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.315794  1691 rpcz_store.cc:276] 0709 13:02:07.961472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.296266 (+334794us) service_pool.cc:224] Handling call
0709 13:02:08.296598 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:08.924306 (+627708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.226473 (+302167us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b540041e6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:09.226733 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1423119744}
I20260709 13:02:09.187233  7263 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.365082  1541 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:02:09.317027  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.346148  1618 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:09.312036  7309 log.cc:927] Time spent T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.270s	user 0.003s	sys 0.000s
I20260709 13:02:09.310979  1435 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:09.307518  6872 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:09.348320  7258 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.347626  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.354892  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:09.355691  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 55245.2us
I20260709 13:02:09.397756  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.318027  1683 raft_consensus.cc:2410] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:02:09.312609  7214 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.408867  1540 raft_consensus.cc:2393] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:09.362125  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.409507  1537 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:02:09.363595  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:09.410753  1691 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:09.312825  7152 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.412761  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:09.412984  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:09.414114  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:09.363936  1688 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:09.408136  1692 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:09.312566  7284 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.345232  7231 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:09.343828  7200 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.350265  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:09.433077  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2527) took 1411 ms. Trace:
W20260709 13:02:09.359681  7276 consensus_meta.cc:287] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.757s	user 0.014s	sys 0.000s
I20260709 13:02:09.397518  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:09.468739  1535 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.420620  7152 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.273020  7088 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:09.402913  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.471041  1541 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:09.472060  7231 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.362993  6860 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 13:02:09.481227  1690 raft_consensus.cc:606] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:09.481277  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.481777  1690 raft_consensus.cc:614] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:09.402985  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 45807.5us
I20260709 13:02:09.409945  1584 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:09.367218  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.503584  1536 raft_consensus.cc:2410] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 17 because replica has last-logged OpId of term: 16 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 13:02:09.408941  7286 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:09.364710  6996 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:09.368688  6973 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:02:09.420077  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 279850us
I20260709 13:02:09.409082  6872 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:09.423516  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.369684  1689 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:09.351588  7232 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:09.433615  1534 rpcz_store.cc:276] 0709 13:02:08.021718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.021850 (+   132us) service_pool.cc:224] Handling call
0709 13:02:08.022181 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:08.112881 (+ 90700us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.432793 (+1319912us) spinlock_profiling.cc:243] Waited 78 ms on lock 0x7b54000e83d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:09.433004 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179296128}
I20260709 13:02:09.352830  6748 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.428696  7288 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:09.449466  7244 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:09.459540  1543 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:09.460942  7154 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.461886  7260 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.462312  7277 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:09.462409  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2046) took 2129 ms. Trace:
I20260709 13:02:09.466534  6762 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:09.509352  1582 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:09.364319  7295 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:09.583331  1540 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:09.590951  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:09.628273  2233 raft_consensus.cc:2410] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:02:09.477833  6835 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:09.566250  7276 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.566218  7231 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:09.610483  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:09.568217  6976 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:09.622550  6762 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:02:09.623147  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 113924us
W20260709 13:02:09.642903  1685 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.513s	user 0.009s	sys 0.005s
I20260709 13:02:09.580749  6973 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.635272  7295 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:09.611620  1616 rpcz_store.cc:276] 0709 13:02:07.333347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.333549 (+   202us) service_pool.cc:224] Handling call
0709 13:02:07.353492 (+ 19943us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:07.525155 (+171663us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:07.806809 (+281654us) log.cc:844] Serialized 14 byte log entry
0709 13:02:07.888047 (+ 81238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:07.888125 (+    78us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:07.888192 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.462156 (+1573964us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400190fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:09.462206 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:09.462325 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":271755,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":4539,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":2007,"spinlock_wait_cycles":393396480,"thread_start_us":315323,"threads_started":2,"wal-append.queue_time_us":182800,"wal-append.run_cpu_time_us":1682,"wal-append.run_wall_time_us":1088698}
I20260709 13:02:09.433118  7284 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:09.663875  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.671967  1684 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:09.681192  1543 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:09.640661  7286 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:09.641264  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.567315  7200 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.684302  7200 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:09.684769  7286 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:09.647253  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:09.686951  1658 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.724s (SENT) [suppressed 22 similar messages]
W20260709 13:02:09.640447  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 230568us
W20260709 13:02:09.689237  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.654366  1685 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:09.691869  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:09.644054  1507 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:09.700282  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:09.703684  1535 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:09.704178  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:09.637703  7144 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:09.653002  7276 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:09.734148  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.735236  7313 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:09.736008  7177 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.706s	user 0.008s	sys 0.008s
I20260709 13:02:09.735796  7313 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:09.736416  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 430659us
I20260709 13:02:09.737154  2203 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:09.738231  7313 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.738323  7255 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:09.738886  7255 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:09.739250  7255 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.739845  7232 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.740398  7232 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:09.740659  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:09.742239  6938 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.742770  6938 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.744621  6601 raft_consensus.cc:3055] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:09.745069  6601 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:09.746369  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.746670  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:09.747517  2233 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:09.714191  7152 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:09.686229  7231 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:09.718827  7263 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.718958  6748 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:09.639639  7244 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.669914  6973 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:09.740902  7232 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:09.720410  1542 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.540s	user 0.017s	sys 0.000s
I20260709 13:02:09.720876  7230 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:09.721118  6838 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.678s	user 0.015s	sys 0.000s
I20260709 13:02:09.721333  2231 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:09.703791  7295 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.722425  2237 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:02:09.722877  6872 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.745823  6855 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.755626  6860 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:09.773319  7177 raft_consensus.cc:2764] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:02:09.729635  7197 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:09.606267  7154 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.745069  6938 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:09.755066  7176 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:09.768746  7286 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.768646  6601 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.770426  7231 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.771059  7244 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.771452  7263 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:09.774292  1542 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:09.888885  7263 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.890614  7244 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:09.733187  7276 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:09.780541  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1072) took 1563 ms. Trace:
I20260709 13:02:09.891839  7263 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:09.783051  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:09.892632  1684 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:09.893010  7232 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.812150  6855 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:09.894205  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 544009us
I20260709 13:02:09.895180  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:09.791045  1689 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:09.623138  7260 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:09.723702  6762 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:09.781629  2203 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:09.787037  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:09.785679  2201 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:09.724892  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:49352 (request call id 35) took 1982 ms. Trace:
I20260709 13:02:09.832312  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:09.913900  1683 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:09.914148  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2020) took 1468 ms. Trace:
I20260709 13:02:09.914763  1689 rpcz_store.cc:276] 0709 13:02:08.446091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.547045 (+100954us) service_pool.cc:224] Handling call
0709 13:02:09.913415 (+1366370us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b540003f054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:09.914068 (+   653us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1869933440}
W20260709 13:02:09.799839  6796 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:02:09.799697  6877 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:09.927680  6877 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:09.928393  6877 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:09.930210  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 144647us
I20260709 13:02:09.931268  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:09.856065  7286 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:09.861272  2231 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 1307) took 1380 ms (client timeout 1746 ms). Trace:
W20260709 13:02:09.933722  2231 rpcz_store.cc:269] 0709 13:02:08.480983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.481162 (+   179us) service_pool.cc:224] Handling call
0709 13:02:09.861085 (+1379923us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400560c54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:09.861211 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2849693056}
I20260709 13:02:09.853317  7176 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:09.935122  7286 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:09.935807  7173 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:09.936270  6796 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:09.936815  6796 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:09.865914  6601 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:09.866294  7154 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:09.869601  6872 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:09.946568  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:09.870990  7255 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.874986  6938 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:09.879796  1435 catalog_manager.cc:5697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:09.805663  6835 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:09.885973  7284 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.887408  7197 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.901259  1684 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:09.891459  2237 rpcz_store.cc:276] 0709 13:02:08.217120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:08.217936 (+   816us) service_pool.cc:224] Handling call
0709 13:02:08.219151 (+  1215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:09.221135 (+1001984us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b58005a17d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa480aa 00007fe08faca447 00007fe08fae2fe7 00007fe08fae47de 00007fe08faeeb49 00007fe08faef416 00007fe08faebd8f 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4
0709 13:02:09.780278 (+559143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.780487 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1441478016}
W20260709 13:02:09.907393  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.908061  1540 raft_consensus.cc:2410] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:02:09.956795  6996 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.957677  1435 catalog_manager.cc:5697] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 11 to 15, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 15 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:09.959138  6996 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:09.832846  1685 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:02:09.964058  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 277118us
I20260709 13:02:09.915802  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:09.912685  1690 rpcz_store.cc:276] 0709 13:02:07.742916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:07.743065 (+   149us) service_pool.cc:224] Handling call
0709 13:02:08.666491 (+923426us) spinlock_profiling.cc:243] Waited 45 ms on lock 0x7b5800010e98. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac8c40 00007fe094ab4bb9 00007fe094ab447c 00007fe08e03a7d4
0709 13:02:09.724591 (+1058100us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b540028e1d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fae6853 00007fe094ab460a 00007fe08e03a7d4 00007fe08e03a762 00007fe08e03a691 00007fe08e03a60c 00007fe08e0398c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:09.724765 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1976700160}
W20260709 13:02:09.966295  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 322348us
I20260709 13:02:09.921521  7260 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:09.906138  7276 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:09.925887  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 144336us
W20260709 13:02:09.926247  7214 consensus_meta.cc:287] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.506s	user 0.011s	sys 0.006s
I20260709 13:02:09.800222  7295 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:09.926671  7231 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:09.813103  6860 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:09.938409  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:09.976761  2233 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:09.941777  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:09.942962  1689 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:09.945061  2236 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:09.945328  6796 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.945797  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:09.979914  2230 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
W20260709 13:02:09.980247  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64566.9us
I20260709 13:02:09.951020  2201 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:09.952800  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:09.949111  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 295030us
I20260709 13:02:09.799604  7230 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:09.920760  7258 consensus_meta.cc:287] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.010s
I20260709 13:02:09.952637  6938 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:09.983111  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:09.984302  1542 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:09.984509  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:10.005863  1538 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:09.985265  1583 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:09.981184  2229 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:09.949468  6872 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:09.988803  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:09.964383  7200 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.016884  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:09.985908  6860 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:09.955616  6835 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:10.087703  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:09.995191   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
W20260709 13:02:10.086982  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 95812.1us
I20260709 13:02:09.974291  7281 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.015180  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:10.003266  7170 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:10.014035  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:10.016209  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:09.976222  7231 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:09.990373  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 284365us
I20260709 13:02:10.116519  2236 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:10.116922  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:10.094019  7178 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.118104  7178 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.118539  7195 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:10.118983  7199 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:10.119089  7195 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:10.119932  7231 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:10.120324  7178 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:09.995178  7232 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:10.121032  7232 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.121462  7232 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:10.122553  7177 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.123091  7177 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.125118  7177 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.089077  1508 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:10.114302  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 117826us
I20260709 13:02:10.129294  1509 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:10.129745  7232 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:09.995589  7074 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.024631  7320 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.131853  7074 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.094883  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:10.010304  6914 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.123s
I20260709 13:02:10.133715  7205 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:10.134393  7074 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.135129  7074 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:10.135257  7205 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:10.135762  7205 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:10.135852  7232 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:10.136257  7205 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.136176  7321 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 13:02:10.094161  7241 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.138797  7154 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.015427  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:10.119453  7199 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:10.126163  7319 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:10.121317  7195 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:10.132325  7320 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.156225  7320 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:09.990294  7260 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:10.133195  2233 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:10.138679  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:10.138767  1439 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 342) took 54238 ms (client timeout 15000 ms). Trace:
I20260709 13:02:10.157543  2236 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:10.158424  7255 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:10.114285  2230 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 13:02:10.133195  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 119272us
I20260709 13:02:10.162846  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:10.168906  6938 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:10.157547  1439 rpcz_store.cc:269] 0709 13:01:15.757502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:15.841816 (+ 84314us) service_pool.cc:224] Handling call
0709 13:02:09.983544 (+54141728us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:02.232043 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:02.232600 (+   557us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:02.232630 (+    30us) write_op.cc:435] Acquired schema lock
0709 13:02:02.232647 (+    17us) tablet.cc:662] Decoding operations
0709 13:02:02.233467 (+   820us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:02.233522 (+    55us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:02.233554 (+    32us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:02:02.234080 (+   526us) tablet.cc:701] Row locks acquired
0709 13:02:02.234099 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:02:02.234247 (+   148us) write_op.cc:270] Start()
0709 13:02:02.234356 (+   109us) write_op.cc:276] Timestamp: P: 1783602122234197 usec, L: 0
0709 13:02:02.234377 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:02:02.317604 (+ 83227us) log.cc:844] Serialized 2463 byte log entry
0709 13:02:03.472549 (+1154945us) op_driver.cc:464] REPLICATION: finished
0709 13:02:03.894054 (+421505us) write_op.cc:301] APPLY: starting
0709 13:02:03.894258 (+   204us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:03.902692 (+  8434us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:03.902835 (+   143us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:03.917237 (+ 14402us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:03.917961 (+   724us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:02:03.918082 (+   121us) write_op.cc:312] APPLY: finished
0709 13:02:03.918808 (+   726us) log.cc:844] Serialized 65 byte log entry
0709 13:02:03.920884 (+  2076us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:03.921254 (+   370us) write_op.cc:454] Released schema lock
0709 13:02:03.921334 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:03.922615 (+  1281us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:03.922853 (+   238us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":537296,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":421023,"apply.run_cpu_time_us":13147,"apply.run_wall_time_us":28478,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29243,"num_ops":6,"prepare.queue_time_us":219998,"prepare.run_cpu_time_us":20627,"prepare.run_wall_time_us":226038,"raft.queue_time_us":525239,"raft.run_cpu_time_us":18999,"raft.run_wall_time_us":343400,"replication_time_us":1237928,"thread_start_us":855985,"threads_started":4,"wal-append.queue_time_us":222959,"wal-append.run_cpu_time_us":33801,"wal-append.run_wall_time_us":2396064}]]}
I20260709 13:02:10.170550  6783 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:02:10.166227  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 77232.9us
I20260709 13:02:09.981264  7214 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.093658   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:09.953457  6877 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.018121  7141 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:10.147948  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:09.987736  7244 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.109s
I20260709 13:02:10.139518  7154 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:10.158571  7260 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:10.171173  6783 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:10.172224  1583 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:10.172461  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:10.180704  6838 consensus_queue.cc:1059] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:02:10.175625  7288 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:10.020727  6976 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:10.118367  1584 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:09.980041  7276 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.183144  1582 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:09.959824  1435 catalog_manager.cc:5697] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:10.138958  7152 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.005941  7258 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.116140  7170 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:10.018534  7277 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.173s
I20260709 13:02:10.138849  7241 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.202291  7244 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:10.092464  7200 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.200731  7178 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:10.145604  7074 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.152487  7199 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.006227  7286 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.215088  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:10.299036  7141 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:10.216059  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:10.194660  7182 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:10.192071  7195 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:10.300819  6993 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:10.257540  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 74511us
I20260709 13:02:10.195020  7232 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:10.258559  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 140293us
I20260709 13:02:10.327903  1584 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:10.261516  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:10.140002  7322 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:02:10.208772  7260 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:10.222474  7088 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.746s	user 0.012s	sys 0.016s
I20260709 13:02:10.218431  7144 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:10.188935  7176 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.009052  7169 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.282s
I20260709 13:02:10.221726  6976 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.305079  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:10.222019  7327 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:10.319206  2236 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:10.215857  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:10.326870  7195 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.324198  7182 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:10.352596  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 47635.8us
I20260709 13:02:10.303406  7334 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:10.194785  7177 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.324293  2228 raft_consensus.cc:2368] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15.
I20260709 13:02:10.304376  7329 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.139052  7281 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.225498  7288 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.313313  6762 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.331506  1538 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:10.323572  7277 raft_consensus.cc:3055] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:10.318097  7178 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.309897  6914 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:02:10.345150  2235 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:10.298213  7152 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.301165  7170 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:10.538371  7258 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:10.351723  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:10.313736  7244 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.618636  7231 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:10.619801  7244 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:10.620491  6973 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.840s	user 0.007s	sys 0.003s
I20260709 13:02:10.620807  7231 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:10.376968  1539 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 13:02:10.409565  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:10.622449  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.623540  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:10.625057  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:10.625973  1684 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:10.387534  1657 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:10.626890  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 239507us
I20260709 13:02:10.318341  7074 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.627357  7321 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.627583  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:10.627923  7321 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.628906  1657 leader_election.cc:400] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15.
I20260709 13:02:10.629130  7074 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.340540  7144 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.629801  6996 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:10.630038  7144 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:10.630439  6996 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.630987  6996 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:10.618698  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:10.456691  7230 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.427443  7141 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:10.617133  7214 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.634457  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:10.635573  2236 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:10.349777  7088 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.470126  1508 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:10.374759  6835 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:10.490617  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:10.677255  2228 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:10.678581  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:10.679361  2228 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
W20260709 13:02:10.388883  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 371926us
I20260709 13:02:10.357825  7295 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:10.572865  6762 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.684062  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:10.684583  7295 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:10.685475  7214 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:10.685922  7214 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.351946  7284 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:10.495428  7176 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.578819  7263 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.466034  1435 catalog_manager.cc:5697] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 8 to 12, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 12 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:10.347139  7327 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.356056  7334 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:10.346922  6976 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:10.580576  7178 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:10.381637  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 54829.2us
I20260709 13:02:10.581655  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1465) took 1478 ms. Trace:
I20260709 13:02:10.718176  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:10.366680  7197 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:10.596019  7331 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.254s	user 0.000s	sys 0.000s
I20260709 13:02:10.466138  7335 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.726241  7281 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:10.726593  7335 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:10.727598  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:10.728322  1684 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:10.728950  7335 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:10.335822  6783 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:10.598392  6601 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.653s	user 0.009s	sys 0.002s
I20260709 13:02:10.431270  7177 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:10.599422  7332 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.248s	user 0.000s	sys 0.000s
W20260709 13:02:10.600253  7330 log.cc:927] Time spent T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.243s	user 0.000s	sys 0.000s
I20260709 13:02:10.600603  7348 raft_consensus.cc:3055] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:10.779837  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.780512  1610 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:10.600018  7205 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:10.344132  7232 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:10.601138  6872 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.602160  7276 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.603407  7241 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:10.604633  7200 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.792917  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.606474  7154 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:10.609879  7356 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:10.610174  7338 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 13:02:10.610581  7347 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:10.611300  7344 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.610849  7182 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:10.611881  6838 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.629225  1657 leader_election.cc:403] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:02:10.325049  7286 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:10.332218  6993 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.634478  7141 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:10.621127  6973 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.638782  7169 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:10.639496  7244 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:10.640410  7321 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.645195  7144 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.627089  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:10.649070  6860 consensus_meta.cc:287] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.003s
I20260709 13:02:10.447988  7329 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.845999  6860 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:10.847430  2230 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:10.847901  7329 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:10.667791  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 197759us
W20260709 13:02:10.676234  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 348406us
I20260709 13:02:10.850492  1584 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:10.850795  7286 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:10.851213  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:10.852350  1584 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:10.353389  7195 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:10.840224  7244 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.853910  7244 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:10.855068  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:10.856451  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:10.856597  7232 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:10.427989  6855 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:10.587385  7277 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:10.686334  7214 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.830422  7276 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15.
I20260709 13:02:10.717509  2235 rpcz_store.cc:276] 0709 13:02:09.102811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.102919 (+   108us) service_pool.cc:224] Handling call
0709 13:02:10.581395 (+1478476us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54004ba854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:10.581571 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2854652800}
I20260709 13:02:10.698788  7320 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:10.729032  7152 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.729436  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:10.731282  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:10.864681  1692 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:10.733031  7197 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.705475  6835 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:10.734975  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:10.735841  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1078) took 1158 ms. Trace:
I20260709 13:02:10.726301  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:10.739665  7281 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:10.739442  7260 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.711256  7327 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:10.747094  7288 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.749221  7176 consensus_queue.cc:1059] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:02:10.714075  7334 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:10.899494  7260 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.699103  7088 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:10.751389  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 273929us
I20260709 13:02:10.866557  7320 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.753760  1691 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:10.902020  7320 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.757958  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:10.759994  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.903100  7088 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:10.903620  1613 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:10.904708  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.760993  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.905292  1613 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:10.760926  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.905934  1612 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:10.906961  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.907545  1612 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:10.908016  7214 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.908470  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:10.758675  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:10.909226  7205 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:10.860713  7284 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:10.910207  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.910362  7214 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.906353  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.911314  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:10.911767  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:10.760305  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.777590  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:10.787766  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:10.926474  1617 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:10.926743  1536 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 13.
I20260709 13:02:10.927486  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.928108  1617 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:10.929001  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.791489  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.808310  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:10.710516  7263 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:10.811110  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 128933us
I20260709 13:02:10.812661  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:10.776341  7326 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:10.810598  1609 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 10.
I20260709 13:02:10.814867  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:10.815527  1539 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:10.778888  7348 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:10.822268  1535 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:02:10.826660  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 197885us
I20260709 13:02:10.807104  7230 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:10.818571  7347 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:10.845237  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1309) took 1270 ms. Trace:
I20260709 13:02:10.857568  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:10.859222  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:10.858700  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:10.861593  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:10.698949  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:10.597023  7328 log.cc:927] Time spent T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.295s	user 0.000s	sys 0.000s
I20260709 13:02:10.833906  6762 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.865631  2232 rpcz_store.cc:276] 0709 13:02:09.641372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.655360 (+ 13988us) service_pool.cc:224] Handling call
0709 13:02:09.708089 (+ 52729us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:09.736733 (+ 28644us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.736762 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:09.737354 (+   592us) log.cc:844] Serialized 52 byte log entry
0709 13:02:10.087196 (+349842us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:10.197545 (+110349us) log.cc:844] Serialized 14 byte log entry
0709 13:02:10.198116 (+   571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:10.198223 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:10.368664 (+170441us) spinlock_profiling.cc:243] Waited 7.87 ms on lock 0x7b1800006220. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:02:10.735436 (+366772us) raft_consensus.cc:1685] finished
0709 13:02:10.735574 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:10.735761 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68737,"prepare.queue_time_us":109696,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":5130,"raft.queue_time_us":29694,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":191,"spinlock_wait_cycles":18096896,"thread_start_us":349370,"threads_started":1,"wal-append.queue_time_us":565519}
I20260709 13:02:10.893618  1689 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:10.893764  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.004552  1686 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:11.006068  6993 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:10.828709  6838 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.012589  6838 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.897974  1683 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
W20260709 13:02:10.898728  7346 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:02:10.912076  1618 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:11.014940  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.015592  1618 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:11.016433  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.017127  1618 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:10.762511  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.018236  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.018934  1611 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:11.018714  6996 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.019757  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.020474  1611 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:10.915777  7170 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.002921  7214 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:11.022034  7214 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.022442  7214 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:11.023030  7170 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:10.922263  7277 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.023952  7257 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:11.024206  7277 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.024425  7257 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.024895  7257 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:11.025611  2229 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:11.027559  7320 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:10.924793  7199 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.600s	user 0.010s	sys 0.000s
I20260709 13:02:11.029500  7178 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:11.029798  7199 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:11.032063  7199 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:10.828153  7344 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:10.867368  6835 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:10.953799  7326 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.033409  6835 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.034317  7344 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:10.862818  7074 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:11.034610  1615 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:11.034667  6835 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:10.945981  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:10.867573  7197 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:10.933880  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.038192  1612 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:10.942888  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:11.045852  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 103072us
I20260709 13:02:11.048620  1686 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:10.825752  7356 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:10.994836  2228 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:10.995566  2236 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:11.050348  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:10.995734  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 134233us
I20260709 13:02:11.051149  7356 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:10.994413  2237 raft_consensus.cc:2410] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 15 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:02:11.051618  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:11.052930  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 49012.1us
W20260709 13:02:10.994865  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 136231us
W20260709 13:02:10.998337  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 220824us
I20260709 13:02:11.056008  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:11.056126  6973 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:11.056826  6973 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:11.057255  1657 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:11.058285  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.059108  1691 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:11.059971  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.059880  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.060952  1534 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
W20260709 13:02:10.930913  7350 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:02:11.022567  7348 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:10.932633  1435 catalog_manager.cc:5697] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 10 to 16, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 16 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:11.021154  6996 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:11.063581  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:11.064085  6996 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:11.058475  7286 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:11.065065  6996 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:11.065179  1509 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:11.065210  1583 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:11.066473  7295 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:11.066864  7295 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.067207  7295 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:11.068120  1617 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:02:11.068535  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:10.994031  2234 rpcz_store.cc:276] 0709 13:02:09.574956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:09.604759 (+ 29803us) service_pool.cc:224] Handling call
0709 13:02:09.605174 (+   415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:09.743270 (+138096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:09.743305 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:09.743870 (+   565us) log.cc:844] Serialized 52 byte log entry
0709 13:02:09.981300 (+237430us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:09.981338 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:09.981400 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:10.845003 (+863603us) raft_consensus.cc:1685] finished
0709 13:02:10.845025 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:10.845177 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127884,"raft.queue_time_us":129516,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"thread_start_us":237007,"threads_started":1,"wal-append.queue_time_us":566111}
I20260709 13:02:11.070124  7356 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:11.070463  7251 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:11.070667  7356 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:11.070984  7342 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:02:11.071116  7356 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:11.071684  1508 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:11.072549  7322 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:11.072940  7322 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.073287  7322 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:11.057370  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:10.999183  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.064388  6855 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:11.065440  7286 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.103180  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:10.866732  7321 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:10.915777  7195 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:11.064173  7363 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.218s	user 0.001s	sys 0.001s
I20260709 13:02:11.081565  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:11.065587  6996 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:11.122699  2230 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:02:11.123994  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:11.124257  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:11.124359  7197 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.104264  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:11.077216  1508 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:11.002422  7335 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:11.125960  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.126257  7335 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.126660  7335 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.126852  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:11.120414  6855 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:11.127691  2237 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:10.982553  7205 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:11.128404  7205 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.115737  7286 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:11.129786  7347 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:11.130841  7347 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:11.131199  7205 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:11.131807  7205 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:11.119478  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:11.069609  7174 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:11.037024  7326 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:11.077980  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:11.080181  2235 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:11.064845  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.065310  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.144881  1543 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:11.061059  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:11.080108  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:11.079185  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.146020  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.077023  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:11.061724  7088 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:11.105758  1614 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:11.124773  2228 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.071007  7251 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:10.941573  7281 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:11.125962  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:11.125742  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 48634.7us
I20260709 13:02:11.102900  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:11.127105  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:10.944298  6872 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:10.983872  7176 raft_consensus.cc:3055] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:11.082621  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:11.117982  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:11.116084  7321 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:11.144999  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:11.181504  2235 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.147102  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.147783  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:11.182930  2236 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.081404  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:11.064878  7152 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:11.125751  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.155547  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.156946  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:11.171763  1618 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.133196  6976 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.129302  6855 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:11.201927  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:10.984277  7288 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.162s
I20260709 13:02:11.159264  2232 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:11.167078  2237 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:11.115146  7320 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:11.179400  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 61525.8us
I20260709 13:02:11.203732  2203 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:11.159405  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:11.172937  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 70368.1us
I20260709 13:02:11.171309  1690 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:11.188948  7152 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.002995  7230 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:11.211412  2236 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:11.211829  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:11.184351  7169 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:11.196082  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:11.214375  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:11.215255  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:11.198630  7281 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.199050  6976 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:11.201267  1609 raft_consensus.cc:2368] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:11.204782  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.229933  1536 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:02:11.205554  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.159107  7177 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:11.206332  2235 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:11.240865  1686 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:02:11.212285  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.190086  7197 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:11.141640  7174 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:11.203630  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.184592  6762 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:11.215895  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:11.222302  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 141024us
I20260709 13:02:11.150523  7178 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:11.227823  1618 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:11.211035  7152 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:11.231009  1690 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:11.214362  7288 consensus_queue.cc:1243] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:11.223678  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:11.273093  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 49524.3us
I20260709 13:02:11.228925  7205 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:11.238013   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:11.159775  7326 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 13:02:11.159566  7251 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:11.210486  6855 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.170863  7176 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:11.185297  7321 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:11.215852  2236 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:11.250355  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:11.303264  2232 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:11.304632  6783 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:11.289220  7197 raft_consensus.cc:2764] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:02:11.308436  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1474) took 1161 ms. Trace:
I20260709 13:02:11.308828  2229 rpcz_store.cc:276] 0709 13:02:10.147478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.147638 (+   160us) service_pool.cc:224] Handling call
0709 13:02:11.308261 (+1160623us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b540023d7d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:11.308378 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1915948928}
I20260709 13:02:11.309324  2236 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
W20260709 13:02:11.310379  7361 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.459s	user 0.000s	sys 0.003s
I20260709 13:02:11.312397  7170 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:11.313047  6860 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:11.313949  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.314217  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:11.315133  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.315848  1609 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:11.316648  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.252272  7174 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:11.239143  7320 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
W20260709 13:02:11.323099  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.252959  7353 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:11.253878  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.325006  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.326205  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.293598  7074 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:11.326773  1685 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:11.238052  6601 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:02:11.327126  1613 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:11.327628  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:11.328742  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.252355  7345 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:11.329493  7230 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:11.330132  7345 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.254388  6993 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:11.254736  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.332604  7345 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:11.332899  1658 connection.cc:256] RPC call timeout handler was delayed by -1.359899793000011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
I20260709 13:02:11.334817  6993 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:11.335338  6993 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:11.337149  7341 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.493s	user 0.003s	sys 0.001s
I20260709 13:02:11.254805  6835 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.338627  6835 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:11.255299  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:11.260133  2230 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:11.264400  1582 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:11.278169  7205 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.206305  1614 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:11.271720  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.278738  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:11.229269  7352 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:11.177515  7088 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.329299  1685 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:11.291699  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.273383  7244 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.268822  7177 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:02:11.241597  7214 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.206238  7334 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:11.185971  6872 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:11.263885  6762 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.341169  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:11.348621  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:11.345166  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.350819  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:11.448879  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.449513  1685 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 11.
I20260709 13:02:11.345093  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:11.451567  1508 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:11.374755  1615 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:11.454114  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.451637  1609 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:11.359853  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:11.352927  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.354646  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:11.455526  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 95754.5us
I20260709 13:02:11.456076  1536 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:11.451752  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:11.356211  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.344187  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.457891  1614 raft_consensus.cc:2410] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:02:11.443470  2234 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:11.353597  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.457126  1685 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.350514  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.459498  1538 raft_consensus.cc:2410] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 13:02:11.460286  1613 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
W20260709 13:02:11.341902  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 77602.2us
I20260709 13:02:11.353901  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:11.445012  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.357271  1688 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:02:11.442747  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.345585  1618 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:11.464006  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 250932us
I20260709 13:02:11.368959  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:11.353266  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.349385  7088 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:11.466598  7074 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:11.351148  7335 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:11.505900  1614 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:11.507261  7335 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:11.507427  1614 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [11.9-13.10]
W20260709 13:02:11.372610  7367 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.003s
I20260709 13:02:11.063548  1435 catalog_manager.cc:5697] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 11 to 15, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 15 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:11.533402  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.341378  6835 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:11.367050  7214 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:11.342070  7205 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:11.514531  1687 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:11.552388  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.467216  1582 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:11.470063  1583 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:11.509792  1686 raft_consensus.cc:2376] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15. Re-sending same reply.
I20260709 13:02:11.526134  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:11.526352  2201 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:02:11.511955  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 43537us
W20260709 13:02:11.571902  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 45688us
I20260709 13:02:11.572978  2201 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:11.534070  1614 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 5. (index mismatch)
I20260709 13:02:11.534996  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:11.573350  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.573482  1618 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:11.574332  1615 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:11.575517  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:11.576562  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:11.536468  2228 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:11.366940  6973 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:11.350443  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:11.554564  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 87411us
I20260709 13:02:11.585517  1690 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:11.587440  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.554996  2235 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:02:11.554944  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 85002.2us
I20260709 13:02:11.589839  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:11.590054  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.372865  7144 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:11.355469  7195 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:11.454902  6762 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.555848  6860 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:11.452347  6855 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:11.461853  7177 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:11.535039  7322 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:11.563658  7372 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:02:11.564440  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:11.566268  1687 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 13:02:11.674687  1683 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:11.674983  1687 pending_rounds.cc:85] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 7
I20260709 13:02:11.675340  1687 pending_rounds.cc:107] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 13:02:11.675796  1687 raft_consensus.cc:2889] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:11.676925  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 364890us
I20260709 13:02:11.346009  7241 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.197s
I20260709 13:02:11.533896  1685 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:11.678294  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:11.678192  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:11.679178  1685 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:11.679280  1687 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:02:11.343145  7263 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:11.358292  7244 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:11.590898  1614 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:11.587311  1509 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:11.594734  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.681965  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.682452  1615 raft_consensus.cc:2410] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
W20260709 13:02:11.681783  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 94560us
I20260709 13:02:11.683275  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.683588  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:11.684027  1541 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:11.594873  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.352622  7174 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:11.595357  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.598815  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.684986  1538 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
W20260709 13:02:11.600118  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.585069  6973 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.682619  1614 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:11.601402  7169 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:11.592900  7088 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:11.671432  1534 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:11.342799  7152 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:11.579218  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:11.686036  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.543778  7074 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:11.677829  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:11.703325  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:11.704795  1582 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:11.681102  7286 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.003s
I20260709 13:02:11.695827  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:11.685489  1611 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.007s	sys 0.007s
I20260709 13:02:11.696400  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.713161  1614 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:11.713653  1615 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
W20260709 13:02:11.713114  1611 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.007s	sys 0.007s
I20260709 13:02:11.698530  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:02:11.697036  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 253887us
I20260709 13:02:11.669683  7178 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:11.656365  7348 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:11.671429  7177 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:11.694571  1687 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:11.675465  7353 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:02:11.703994  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 459590us
W20260709 13:02:11.693454  7356 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.622s	user 0.011s	sys 0.000s
W20260709 13:02:11.715107  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.700701  7088 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:11.715723  1509 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:11.721251  1542 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:11.721670  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:11.753799  1614 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:11.726176  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.726884  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:11.755848  1541 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:02:11.756418  2231 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:11.728298  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.728963  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.702222  7321 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.730404  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:11.731004  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.761502  2232 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:11.697461  6973 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:11.508632  7176 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:11.682910  6860 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:11.735081  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 2913) took 1001 ms. Trace:
I20260709 13:02:11.728561  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:11.697281  7195 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:11.805816  7195 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:11.807031  7195 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.809042  7321 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:11.740278  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:11.830439  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 981) took 1001 ms. Trace:
I20260709 13:02:11.701402  6872 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:11.757488  1613 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:11.842339  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:11.757138  1615 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:11.751969  1657 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:11.721557  7263 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:11.700310  7375 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:11.829656  7370 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:02:11.727232  7335 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:11.732672  7177 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:11.732187  7199 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:11.752497  7284 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:11.767315  7360 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:11.956629  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:11.794862  1688 rpcz_store.cc:276] 0709 13:02:10.734505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.734621 (+   116us) service_pool.cc:224] Handling call
0709 13:02:11.734862 (+1000241us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b54000749d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:11.735019 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1130226944}
W20260709 13:02:11.798509  7251 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.001s
W20260709 13:02:11.799341  7327 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.865s	user 0.003s	sys 0.007s
I20260709 13:02:11.799779  7322 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:11.801172  7295 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.734s	user 0.011s	sys 0.000s
I20260709 13:02:11.799795  1435 catalog_manager.cc:5697] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 8 to 12, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 12 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:02:11.803340  6996 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.653s	user 0.008s	sys 0.001s
I20260709 13:02:11.751324  7353 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:11.795089  1687 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:02:11.737756  1535 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:11.809742  1540 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:11.727653  7286 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:11.702903  7169 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.811738  7174 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:11.812048  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 217310us
I20260709 13:02:11.745136  7178 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:11.742059  7356 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:11.818078  7152 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:11.762637  7371 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:02:11.819049  7368 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:02:11.763940  7088 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:11.819939  1541 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:02:11.824560  1614 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:11.830482  1538 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:12.046475  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.824128  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:11.839162  1616 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.018s	sys 0.024s
W20260709 13:02:12.048254  1616 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.018s	sys 0.024s
I20260709 13:02:12.048913  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 1110) took 1260 ms. Trace:
W20260709 13:02:11.842743  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 102702us
I20260709 13:02:12.049297  1540 rpcz_store.cc:276] 0709 13:02:10.788512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.788704 (+   192us) service_pool.cc:224] Handling call
0709 13:02:12.048737 (+1260033us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b54003444d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:12.048861 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2003965184}
I20260709 13:02:12.031617  7176 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:12.031704  7286 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:11.862761  2232 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:11.850373  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:11.850513  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 98641us
I20260709 13:02:11.850221  1658 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:11.934782  2235 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:11.832728  7141 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:11.842378  6835 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.891458  7263 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:11.990589  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.995944  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:11.924618  7244 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:11.922196  7347 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:11.927181  7170 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:11.824118  6762 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:11.924144  6872 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:11.882680  7375 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:12.004736  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:12.023168  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:12.088387  1542 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:12.023375  7295 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.032189  1692 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:12.066516  6976 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:11.961148  7335 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:12.042644  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:12.035593  6601 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:11.732231  1683 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:12.099581  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2037) took 1044 ms. Trace:
I20260709 13:02:12.100025  1683 rpcz_store.cc:276] 0709 13:02:11.055528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.564143 (+508615us) service_pool.cc:224] Handling call
0709 13:02:12.099502 (+535359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:11.988822  7251 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:12.054841  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:12.040941  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:12.056936  1618 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:12.057744  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2545) took 1004 ms. Trace:
I20260709 13:02:12.035678  7321 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.031685  7169 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:12.045197  7257 consensus_meta.cc:287] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 1.020s	user 0.009s	sys 0.003s
I20260709 13:02:12.062494  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:11.987833  7334 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:11.847821  1617 rpcz_store.cc:276] 0709 13:02:10.828844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.069066 (+240222us) service_pool.cc:224] Handling call
0709 13:02:11.069341 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.075555 (+  6214us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:02:11.516881 (+441326us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.830084 (+313203us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:11.830143 (+    59us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:11.830198 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.830264 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:11.830380 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81810,"prepare.queue_time_us":226729,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":82702,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":606463,"threads_started":2}
I20260709 13:02:12.118947  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:12.061596  1657 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:12.062793  1615 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 10.
I20260709 13:02:12.002511  7327 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.073654  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:12.120404  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:12.083527  7347 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:12.121129  2232 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:12.074703  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:12.121129  1685 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
W20260709 13:02:12.121789  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 59376.9us
I20260709 13:02:12.123442  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:12.023936  7381 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.082543  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1093) took 1868 ms. Trace:
I20260709 13:02:12.092433  7375 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:12.058194  7174 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.088896  1613 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:12.075738  7263 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:12.091763  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:11.965389  7360 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:12.096730  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 363272us
I20260709 13:02:12.159005  1543 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:02:12.159473  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.160506  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:12.066783  7141 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:12.075853  7244 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.056677  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2068) took 1298 ms. Trace:
I20260709 13:02:12.107587  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:12.169235  1540 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:02:12.169263  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:12.109589  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2026) took 1061 ms. Trace:
I20260709 13:02:12.112356  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:12.060604  6783 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:02:12.176190  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:12.105086  1609 rpcz_store.cc:276] 0709 13:02:11.053316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.466756 (+413440us) service_pool.cc:224] Handling call
0709 13:02:11.544588 (+ 77832us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.544881 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:11.575304 (+ 30423us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.057399 (+482095us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.057468 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:12.057519 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.057586 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.057694 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21181,"prepare.queue_time_us":56118,"prepare.run_cpu_time_us":1853,"prepare.run_wall_time_us":64007,"raft.queue_time_us":2366,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":11433,"thread_start_us":480148,"threads_started":1}
I20260709 13:02:12.177379  7152 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:12.177661  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:12.178424  1609 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:12.179070  7251 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:12.118999  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1205) took 1259 ms. Trace:
I20260709 13:02:12.179927  1684 rpcz_store.cc:276] 0709 13:02:10.860002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.897316 (+ 37314us) service_pool.cc:224] Handling call
0709 13:02:10.943380 (+ 46064us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:10.943812 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:10.943854 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.003057 (+ 59203us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.277831 (+274774us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:11.277951 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.278094 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.118706 (+840612us) raft_consensus.cc:1685] finished
0709 13:02:12.118734 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.118925 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3729,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":37604,"thread_start_us":242561,"threads_started":1,"wal-append.queue_time_us":563420}
I20260709 13:02:12.181064  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:12.182301  1684 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:02:12.182698  1684 pending_rounds.cc:85] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 7
I20260709 13:02:12.182941  1684 pending_rounds.cc:107] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:02:12.183218  1684 raft_consensus.cc:2889] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:12.184244  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2051) took 1041 ms. Trace:
I20260709 13:02:12.084997  7286 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:12.185173  7244 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.185895  7286 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:12.186614  7286 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:12.187701  7286 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.121069  1615 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:11.966487  6860 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.190851  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2551) took 1115 ms. Trace:
I20260709 13:02:12.124357  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2040) took 1065 ms. Trace:
I20260709 13:02:12.034108  6996 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.191382  1615 rpcz_store.cc:276] 0709 13:02:11.075240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.189928 (+1114688us) service_pool.cc:224] Handling call
0709 13:02:12.190257 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.190439 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.190463 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.190520 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:12.190574 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.190651 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.190783 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.133303  1534 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:12.116976  6855 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:12.076066  7380 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 13:02:12.134881  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:12.220988  6976 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:12.102741  6601 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:12.221491  6976 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:12.222139  6601 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.223214  6601 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:12.080274  7214 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:02:12.225965  1615 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:11.959708  7074 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.226850  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 991) took 1146 ms. Trace:
I20260709 13:02:12.227293  1615 rpcz_store.cc:276] 0709 13:02:11.080132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.210192 (+1130060us) service_pool.cc:224] Handling call
0709 13:02:12.225536 (+ 15344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.226508 (+   972us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.226780 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.077724  6835 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.126808  2233 rpcz_store.cc:276] 0709 13:02:10.214267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.451901 (+237634us) service_pool.cc:224] Handling call
0709 13:02:10.946845 (+494944us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.001015 (+ 54170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.001047 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.002693 (+  1646us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.250403 (+247710us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:11.250446 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.250520 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.038749 (+788229us) raft_consensus.cc:1685] finished
0709 13:02:12.038768 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.074003 (+ 35235us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54003d3bd0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:12.082435 (+  8432us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82954,"raft.queue_time_us":55567,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":1137140480,"thread_start_us":209432,"threads_started":1,"wal-append.queue_time_us":526836}
I20260709 13:02:12.134511  7381 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.228790  1615 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:12.161305  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:12.165068  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:12.124706  7347 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:12.125193  7169 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:12.096292  6872 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.173312  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:12.084597  7178 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.246856  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:12.247970  1540 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:12.174779  7320 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.169063  1616 rpcz_store.cc:276] 0709 13:02:10.757872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:10.761505 (+  3633us) service_pool.cc:224] Handling call
0709 13:02:10.761775 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:10.761994 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:11.116380 (+354386us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.048997 (+932617us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.054013 (+  5016us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.056305 (+  2292us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.056360 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:12.056402 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.056469 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.056600 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9862,"prepare.queue_time_us":455246,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":7746,"raft.queue_time_us":15292,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":18803,"spinlock_wait_cycles":83591808,"thread_start_us":1238376,"threads_started":2}
W20260709 13:02:12.175529  7205 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.619s	user 0.010s	sys 0.000s
I20260709 13:02:12.170357  1692 rpcz_store.cc:276] 0709 13:02:11.048126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.086947 (+ 38821us) service_pool.cc:224] Handling call
0709 13:02:12.109289 (+1022342us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b54000f5354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:12.109509 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1948574080}
I20260709 13:02:12.184129  1507 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:12.192411  1687 rpcz_store.cc:276] 0709 13:02:11.058796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.111772 (+1052976us) service_pool.cc:224] Handling call
0709 13:02:12.123904 (+ 12132us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.124074 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.124096 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.124140 (+    44us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:12.124181 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.124230 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.124310 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:12.250517  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 66455.4us
I20260709 13:02:12.250808  1692 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:12.251449  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1209) took 1105 ms. Trace:
I20260709 13:02:12.251741  1692 rpcz_store.cc:276] 0709 13:02:11.146599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.250140 (+1103541us) service_pool.cc:224] Handling call
0709 13:02:12.250472 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.251207 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.251398 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.214996  7152 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:12.252771  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:12.184667  1684 rpcz_store.cc:276] 0709 13:02:11.142603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.181638 (+1039035us) service_pool.cc:224] Handling call
0709 13:02:12.181966 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.184007 (+  2041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.184193 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.253624  6855 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:12.254289  6855 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:12.255646  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:12.183037  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2550) took 1108 ms. Trace:
I20260709 13:02:12.256191  1614 rpcz_store.cc:276] 0709 13:02:11.074992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.182205 (+1107213us) service_pool.cc:224] Handling call
0709 13:02:12.182515 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.182702 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.182732 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.182773 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:12.182810 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.182877 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.182981 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:11.999558  7284 raft_consensus.cc:697] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:12.257608  7232 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:12.257925  7284 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.260212  1539 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:12.260524  7232 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:12.261281  1614 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:12.179529  1609 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:12.261974  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2554) took 1109 ms. Trace:
I20260709 13:02:12.216401  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2035) took 1161 ms. Trace:
I20260709 13:02:12.219837  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:12.262578  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2548) took 1188 ms. Trace:
W20260709 13:02:12.209120  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.229943  1542 raft_consensus.cc:2368] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14.
I20260709 13:02:12.229413  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2553) took 1091 ms. Trace:
I20260709 13:02:12.165186  7360 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:12.240411  1617 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:12.119555  7321 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.121505  7257 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.255770  1684 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:02:12.135386  7281 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:12.229557  1618 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:12.249207  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:12.264904  1687 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:12.260286  1507 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:12.299491  1618 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 11. Current term is 13. Ops: []
I20260709 13:02:12.299690  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2552) took 1188 ms. Trace:
I20260709 13:02:12.300137  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2556) took 1093 ms. Trace:
I20260709 13:02:12.263315  1691 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:12.262825  1535 raft_consensus.cc:2368] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:12.298213  1609 rpcz_store.cc:276] 0709 13:02:11.074458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.179028 (+1104570us) service_pool.cc:224] Handling call
0709 13:02:12.179330 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.262326 (+ 82996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.262525 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.303741  1684 pending_rounds.cc:85] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 9
I20260709 13:02:12.262320  1614 rpcz_store.cc:276] 0709 13:02:11.153151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.257115 (+1103964us) service_pool.cc:224] Handling call
0709 13:02:12.257412 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.261708 (+  4296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.261910 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.304231  1684 pending_rounds.cc:107] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:02:12.293149  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2078) took 1532 ms. Trace:
I20260709 13:02:12.304661  1684 raft_consensus.cc:2889] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:12.301249  1612 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.015s	sys 0.050s
W20260709 13:02:12.305991  1612 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.016s	sys 0.050s
I20260709 13:02:12.300755  1540 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:12.300302  1617 rpcz_store.cc:276] 0709 13:02:11.110941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.221314 (+1110373us) service_pool.cc:224] Handling call
0709 13:02:12.229705 (+  8391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.299476 (+ 69771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.299639 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.305124  1611 rpcz_store.cc:276] 0709 13:02:10.760828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.020975 (+260147us) service_pool.cc:224] Handling call
0709 13:02:11.021260 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.084427 (+ 63167us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:11.722132 (+637705us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.208845 (+486713us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.208927 (+    82us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:12.208995 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.209089 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.293062 (+ 83973us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96767,"prepare.queue_time_us":218032,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":40855,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":14600,"thread_start_us":997878,"threads_started":2}
I20260709 13:02:12.298273  1615 rpcz_store.cc:276] 0709 13:02:11.137980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.228308 (+1090328us) service_pool.cc:224] Handling call
0709 13:02:12.228634 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.229180 (+   546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.229350 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:12.262450  1685 rpcz_store.cc:276] 0709 13:02:11.055002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.692389 (+637387us) service_pool.cc:224] Handling call
0709 13:02:11.694155 (+  1766us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.122272 (+428117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.216152 (+ 93880us) spinlock_profiling.cc:243] Waited 427 ms on lock 0x7b5400374454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:12.216343 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":981360896}
I20260709 13:02:12.297443  1613 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:12.215250  7230 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:12.262282  7295 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:12.244796  7335 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:12.305985  7205 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:12.321388  7251 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:12.321651  7170 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:02:12.317744  7169 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:02:12.334159  7295 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:12.337074  7295 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:12.337925  6993 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 1.002s	user 0.011s	sys 0.000s
I20260709 13:02:12.338469  7327 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:12.163398  7174 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:12.339326  7295 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:12.278582  6762 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:12.328642  2236 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:02:12.340178  6973 consensus_meta.cc:287] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.000s
I20260709 13:02:12.340691  7174 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:12.249269  7320 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:12.342952  7320 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:12.343156  7257 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:12.343688  7383 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.185s	user 0.002s	sys 0.000s
I20260709 13:02:12.345029  7321 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:12.277524  7141 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.323385  6872 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:12.345820  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:12.346261  7141 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.346580  1542 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:12.347441  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:12.348021  1542 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:12.155426  7353 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.348750  6996 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:12.348876  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:12.350318  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2056) took 1158 ms. Trace:
I20260709 13:02:12.350741  1691 rpcz_store.cc:276] 0709 13:02:11.192075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.344882 (+152807us) service_pool.cc:224] Handling call
0709 13:02:12.350149 (+1005267us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b5400375354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:12.350259 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1873544960}
W20260709 13:02:12.351263  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.265720  7381 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:12.360734  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:12.318380  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 58162.5us
I20260709 13:02:12.338399  6993 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.394550  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2079) took 1633 ms. Trace:
W20260709 13:02:12.242169  7348 consensus_meta.cc:287] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
I20260709 13:02:12.277050  1435 catalog_manager.cc:5697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 11 to 15, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 15 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:12.359264  1609 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:12.360150  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:12.361528  1685 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:12.300562  1618 rpcz_store.cc:276] 0709 13:02:11.207122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.298971 (+1091849us) service_pool.cc:224] Handling call
0709 13:02:12.299323 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.299922 (+   599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.300084 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40448}
I20260709 13:02:12.435562  1543 raft_consensus.cc:2368] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:12.366436  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2085) took 1596 ms. Trace:
I20260709 13:02:12.395033  1610 rpcz_store.cc:276] 0709 13:02:10.761072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.038034 (+276962us) service_pool.cc:224] Handling call
0709 13:02:11.038370 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:11.038741 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:11.242349 (+203608us) log.cc:844] Serialized 14 byte log entry
0709 13:02:11.895155 (+652806us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:11.895812 (+   657us) log.cc:844] Serialized 26 byte log entry
0709 13:02:11.895979 (+   167us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:11.896021 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:11.896185 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.343093 (+446908us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:12.357415 (+ 14322us) raft_consensus.cc:1685] finished
0709 13:02:12.394277 (+ 36862us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.394476 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":250825,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":34607,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":21505,"spinlock_wait_cycles":28963072,"thread_start_us":597056,"threads_started":2,"wal-append.queue_time_us":863022}
I20260709 13:02:12.396565  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:12.398969  1582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.021s	sys 0.027s
W20260709 13:02:12.438242  1582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.021s	sys 0.028s
W20260709 13:02:12.438839  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 734061us
I20260709 13:02:12.440900  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:12.423538  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:12.441973  1540 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:12.442907  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:02:12.323602  7382 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
I20260709 13:02:12.445709  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:12.447263  2201 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:12.447474  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2558) took 1196 ms. Trace:
I20260709 13:02:12.436734  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:12.448105  1609 rpcz_store.cc:276] 0709 13:02:11.251045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.302908 (+1051863us) service_pool.cc:224] Handling call
0709 13:02:12.359022 (+ 56114us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.447144 (+ 88122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.447406 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:12.429003  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 58586.2us
I20260709 13:02:12.423452  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:12.450084  1535 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.430810  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:12.453405  2228 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:12.463632  1534 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:12.324620  6835 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:12.471992  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:12.475291  1534 raft_consensus.cc:2368] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12.
I20260709 13:02:12.327484  7356 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:12.477555  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 627367us
I20260709 13:02:12.478632  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:12.482072  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2554) took 1130 ms. Trace:
I20260709 13:02:12.482671  1539 rpcz_store.cc:276] 0709 13:02:11.351400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.353332 (+  1932us) service_pool.cc:224] Handling call
0709 13:02:12.481841 (+1128509us) spinlock_profiling.cc:243] Waited 801 ms on lock 0x7b54000eddd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:12.481992 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1843193728}
W20260709 13:02:12.485795  2230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.003s	sys 0.027s
W20260709 13:02:12.486227  2230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.003s	sys 0.027s
I20260709 13:02:12.487679  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:12.488818  1543 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 13: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:12.488951  1509 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:12.491674  1534 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:12.493384  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:12.506970  1535 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:12.508608  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:12.519482  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:12.445905  6996 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.340762  6973 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:12.534628  1542 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:12.466032  7199 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:02:12.467060  7174 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:12.544613  1584 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:12.427647  7321 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
W20260709 13:02:12.473917  7352 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.018s	sys 0.044s
I20260709 13:02:12.520797  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:12.550911  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:12.476892  7241 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:12.515810  6835 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:12.464046  7326 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.076s
I20260709 13:02:12.518981  6762 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:12.529552  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:12.542233  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:12.535004  7371 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:12.554378  1542 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:12.516439  7376 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:12.556937  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:12.538429  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:12.633996  2236 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.571702  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:12.634753  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 63145.9us
I20260709 13:02:12.571844  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:12.549633  7368 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:02:12.550021  7385 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.450s	user 0.000s	sys 0.001s
I20260709 13:02:12.612660  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:12.549796  7352 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.019s	sys 0.044s
I20260709 13:02:12.648195  1613 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:12.552666  6872 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:12.543380  7251 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:12.544003  7199 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:12.629315  2229 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.007s	sys 0.027s
W20260709 13:02:12.692942  2229 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.008s	sys 0.027s
I20260709 13:02:12.634081  1610 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.571146  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:12.639384  1657 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.016s	sys 0.020s
W20260709 13:02:12.694402  1657 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.016s	sys 0.020s
W20260709 13:02:12.694795  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 633222us
I20260709 13:02:12.549291  7174 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:12.532737  7327 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:12.641775  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:12.646723  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:12.643157  7154 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.011s	sys 0.055s
I20260709 13:02:12.618973  7263 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.569110  7241 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.557813  7295 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:12.650218  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:12.621147  7375 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.653672  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:12.666018  2236 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:12.540567  6996 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:12.531889  7170 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.100s
W20260709 13:02:12.689100  7088 consensus_meta.cc:287] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.000s
I20260709 13:02:12.689826  1435 catalog_manager.cc:5697] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 8 to 14, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:12.611431  7144 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 13:02:12.639650  7390 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:12.734534  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 190006us
I20260709 13:02:12.627208  6762 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:12.739636  2231 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:12.703959  1614 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:02:12.741066  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.741228  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:12.713059  7174 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.742764  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:12.744167  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:12.553088  7205 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:12.703974  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 62265.7us
W20260709 13:02:12.704176  1686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.015s	sys 0.029s
I20260709 13:02:12.690759  6973 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:12.715315  1613 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:12.748997  1534 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:12.711077  1543 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:12.695955  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:12.652343  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:12.749269  1613 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:12.722890  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:12.750274  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:12.750511  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 54646.7us
I20260709 13:02:12.752658  1614 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:12.684141  7152 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:12.703519  7199 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:02:12.719208  7241 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:12.730710  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.765300  1689 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:12.691582  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:12.717787  7154 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.011s	sys 0.056s
W20260709 13:02:12.746994  1686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.015s	sys 0.029s
I20260709 13:02:12.712219  7334 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:12.749867  1611 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.657869  7348 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:12.724910  1610 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:12.756924  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:12.757905  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:12.758823  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:12.759673  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:12.760847  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:12.750619  1617 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
W20260709 13:02:12.717382  7197 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.005s	sys 0.033s
I20260709 13:02:12.693317  6872 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:12.732533  7074 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.013s	sys 0.010s
I20260709 13:02:12.728626  7327 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:02:12.752041  7178 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.006s	sys 0.028s
I20260709 13:02:12.749467  6762 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:12.796936  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:12.749867  7400 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:12.800976  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:12.803956  1615 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:12.796937  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2555) took 1619 ms. Trace:
I20260709 13:02:12.820135  1616 rpcz_store.cc:276] 0709 13:02:11.177248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.298966 (+1121718us) service_pool.cc:224] Handling call
0709 13:02:12.299323 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.299540 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:02:12.352493 (+ 52953us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.796407 (+443914us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.796567 (+   160us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.796609 (+    42us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:12.796704 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.796769 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.796882 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7387,"prepare.queue_time_us":60658,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":4528,"spinlock_wait_cycles":42880,"thread_start_us":430540,"threads_started":1}
W20260709 13:02:12.791672  7197 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.006s	sys 0.033s
I20260709 13:02:12.782238  7353 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:12.806703  7178 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.006s	sys 0.033s
I20260709 13:02:12.786123  7348 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:12.802806  7074 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.014s	sys 0.010s
I20260709 13:02:12.799773  7154 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:12.739033  7088 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.791464  7334 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.873175  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:12.855103  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2036) took 1799 ms. Trace:
W20260709 13:02:12.875540  7387 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.341s	user 0.000s	sys 0.001s
I20260709 13:02:12.833137  7404 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:12.871780  6993 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:12.878541  7348 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:12.878456  7353 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.876777  7154 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:12.890622  1616 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:12.891788  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:12.848192  7329 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:12.906426  1435 catalog_manager.cc:5697] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 8 to 15, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 15 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:02:12.909948  7399 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
W20260709 13:02:12.910732  7401 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 13:02:12.911499  7244 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.726s	user 0.001s	sys 0.008s
I20260709 13:02:12.872081  7295 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:12.872434  7400 consensus_queue.cc:1059] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:02:12.974850  2234 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:12.890884  1686 rpcz_store.cc:276] 0709 13:02:11.055266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.692954 (+637688us) service_pool.cc:224] Handling call
0709 13:02:11.693249 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.693498 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:02:12.095661 (+402163us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.854575 (+758914us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:12.854660 (+    85us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:12.854739 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.854836 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.855015 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8541,"prepare.queue_time_us":532330,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":30,"thread_start_us":1137014,"threads_started":3,"wal-append.queue_time_us":823372}
I20260709 13:02:12.932155  1610 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:12.940681  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1208) took 1875 ms. Trace:
I20260709 13:02:12.974779  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:13.050140  1683 rpcz_store.cc:276] 0709 13:02:11.064811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.110420 (+1045609us) service_pool.cc:224] Handling call
0709 13:02:12.110872 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.111285 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.111322 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.118589 (+  7267us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.654284 (+535695us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:12.654346 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.654445 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.940447 (+286002us) raft_consensus.cc:1685] finished
0709 13:02:12.940470 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.940616 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":74346,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":488412,"threads_started":1,"wal-append.queue_time_us":673289}
W20260709 13:02:13.022303  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 270471us
I20260709 13:02:12.969668  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2027) took 1921 ms. Trace:
W20260709 13:02:13.022207  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 48074.7us
I20260709 13:02:13.054437  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:12.938373  7281 consensus_meta.cc:287] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.637s	user 0.009s	sys 0.000s
W20260709 13:02:12.913108  6601 consensus_meta.cc:287] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.000s
I20260709 13:02:12.928947  7353 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:12.961192  7174 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:12.927280  6976 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.705s	user 0.006s	sys 0.002s
I20260709 13:02:13.025414  7244 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.057278  1689 rpcz_store.cc:276] 0709 13:02:11.048422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.050084 (+  1662us) service_pool.cc:224] Handling call
0709 13:02:12.969453 (+1919369us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54000f6754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:12.969592 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3942231424}
W20260709 13:02:12.918527  7347 consensus_meta.cc:287] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.000s
I20260709 13:02:12.951634  1611 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:13.062505  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:12.932504  7360 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.615s	user 0.006s	sys 0.004s
W20260709 13:02:12.924153  7141 consensus_meta.cc:287] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.577s	user 0.009s	sys 0.000s
W20260709 13:02:12.971834  7177 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.977s	user 0.006s	sys 0.002s
I20260709 13:02:12.929039  7412 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.066784  7281 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:13.067353  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1329) took 1492 ms. Trace:
I20260709 13:02:13.068779  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:13.069733  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.101039  1534 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:12.916230  7375 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:12.940099  7334 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:13.027064  7295 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:12.935253  7088 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:12.913347  7263 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:12.964236  6996 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:12.914562  7404 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:12.937314  1435 catalog_manager.cc:5697] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:13.109444  1435 catalog_manager.cc:5697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:13.110903  1435 catalog_manager.cc:5697] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 9 to 13, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 13 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:13.072023  6976 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.093312  7347 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.113852  1688 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:13.114292  6976 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.114233  1534 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:13.102274  7412 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.117204  7412 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:13.118428  7244 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:13.120147  7174 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.126951  7244 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:13.127754  7244 consensus_queue.cc:1243] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:13.128263  7430 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:13.128661  7430 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:12.931151  7154 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.129536  7430 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.130223  7154 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:13.101769  7177 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.134656  1536 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:13.135974  7177 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:13.138012  7404 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.090310  1584 leader_election.cc:400] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14.
W20260709 13:02:13.144501  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 54430.7us
I20260709 13:02:13.145679  1584 leader_election.cc:400] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:13.047075  6783 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:13.064805  7169 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.066540  7205 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:12.938524  7327 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:12.914834  6872 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:12.933787  7329 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.964396  6762 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:12.971344  7199 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:12.966351  7241 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:12.971308  7348 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.077533  7284 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.101590  2229 rpcz_store.cc:276] 0709 13:02:11.574454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.574575 (+   121us) service_pool.cc:224] Handling call
0709 13:02:11.574882 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.575100 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:12.124889 (+549789us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.703480 (+578591us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.970529 (+267049us) spinlock_profiling.cc:243] Waited 5.55 ms on lock 0x7b18003ef500. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff522 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:02:13.061805 (+ 91276us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.061861 (+    56us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:13.061917 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.061987 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.063944 (+  1957us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28561,"prepare.queue_time_us":539396,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":3310,"raft.queue_time_us":60863,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":30360,"spinlock_wait_cycles":35955840,"thread_start_us":1373381,"threads_started":3,"wal-append.queue_time_us":714427}
I20260709 13:02:13.092442  7353 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:13.148823  1584 leader_election.cc:400] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:13.176767  1584 leader_election.cc:403] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:02:13.104743  7295 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:13.157748  1509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.014s	sys 0.006s
W20260709 13:02:13.179167  1509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.015s	sys 0.007s
W20260709 13:02:13.177321  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 124210us
I20260709 13:02:13.183152  7434 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:13.184437  7435 raft_consensus.cc:3055] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:13.184844  7435 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:13.185703  7435 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.187103  7435 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:13.194545  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:13.195547  6996 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:13.196179  6996 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:13.196563  6996 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:13.196542  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:13.197548  1534 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:13.096784  7360 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:13.179504  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 632793us
I20260709 13:02:13.207288  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 1121) took 1212 ms. Trace:
I20260709 13:02:13.199375  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2034) took 2145 ms. Trace:
I20260709 13:02:13.207672  1536 rpcz_store.cc:276] 0709 13:02:11.994961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.995616 (+   655us) service_pool.cc:224] Handling call
0709 13:02:13.207110 (+1211494us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54003449d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:13.207234 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2445015424}
I20260709 13:02:13.209995  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.170043  7205 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:13.212163  1618 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:02:13.217842  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 69217.3us
W20260709 13:02:13.218973  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.210786  1685 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:13.212076  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:13.219036  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:13.219213  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:13.151360  7169 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:13.213851  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 112358us
I20260709 13:02:13.214780  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:13.216849  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:13.227607  1536 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:13.201407  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:13.212715  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:13.227937  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:13.223059  1614 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:13.229228  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:13.207831  1690 rpcz_store.cc:276] 0709 13:02:11.054402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.601326 (+546924us) service_pool.cc:224] Handling call
0709 13:02:11.665821 (+ 64495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:11.666046 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:12.007487 (+341441us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.792576 (+785089us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.006615 (+214039us) log.cc:844] Serialized 14 byte log entry
0709 13:02:13.199011 (+192396us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:13.199098 (+    87us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:13.199143 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.199214 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.199322 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67029,"prepare.queue_time_us":739340,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":14466,"raft.queue_time_us":446908,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":1682,"thread_start_us":1461848,"threads_started":5,"wal-append.queue_time_us":707012}
I20260709 13:02:13.224258  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:13.224124  1686 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12. Ops: []
I20260709 13:02:13.226101  1611 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:13.153147  6783 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.237682  1538 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:13.238982  1618 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:13.231161  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:13.232926  1509 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:13.234179  2236 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:13.231659  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:13.236236  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:13.225759  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.239347  1584 leader_election.cc:400] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 11. Current term is 12.
I20260709 13:02:13.240712  2234 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:13.205974  6835 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:13.211530  7263 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.244815  1584 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:13.242055  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:13.261087  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.257716  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:13.253046  1543 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:13.180023  7284 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.255977  7381 consensus_queue.cc:1059] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 13:02:13.310966  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.311676  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.312785  7169 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.313270  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2916) took 2220 ms. Trace:
I20260709 13:02:13.250313  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:13.313694  1688 rpcz_store.cc:276] 0709 13:02:11.092978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.111852 (+1018874us) service_pool.cc:224] Handling call
0709 13:02:12.128935 (+ 17083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.114256 (+985321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.313020 (+198764us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b54001dd3d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:13.313211 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2264600960}
I20260709 13:02:13.237962  7335 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:13.318614  2233 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:13.254419  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.205540  7088 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:13.323071  1536 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:13.248903  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:13.323884  1535 raft_consensus.cc:2368] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:13.258070  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:13.291054  2232 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:13.265772  1509 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:13.264154  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:13.267153  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:13.326263  1542 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:13.263275  7197 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:02:13.298969  7199 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.263566  7433 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.299924  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:13.266682  7404 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:13.256462  6976 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:13.306470  1686 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:13.241796  7437 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:13.307129  7430 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:13.331699  7169 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:13.346607  7169 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:13.347590  7169 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.348878  7169 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:13.306643  7205 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.351549  7335 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:13.352244  7335 raft_consensus.cc:697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:13.353219  7335 consensus_queue.cc:237] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.354643  2228 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:13.307548  1690 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:13.308274  7439 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:13.308736  1685 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:02:13.309670  7425 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:02:13.309866  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:13.360939  1687 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:13.310418  1683 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:13.315232  1539 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:13.315618  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.255095  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:13.315835  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:13.316082  2234 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:13.241853  7390 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:13.220933  7251 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 13:02:13.242807  6601 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:13.339860  7381 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:13.324585  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 79849.4us
I20260709 13:02:13.328061  7444 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:13.225653  7375 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:02:13.299280  6783 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:13.334350  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 76446.5us
I20260709 13:02:13.391707  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:13.339404  1691 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
W20260709 13:02:13.341876  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 76163.7us
W20260709 13:02:13.368629  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.372699  1542 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:13.375006  7178 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:13.336354  7141 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 13:02:13.242509  7426 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:13.384183  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68422.5us
I20260709 13:02:13.387346  7390 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:13.340160  7428 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:13.396435  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.397179  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.333879  7088 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.277788  7263 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.336045  7334 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:02:13.383370  7404 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:13.405827  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.399462  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:13.406454  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:13.398938  1685 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:02:13.393477  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:13.381589  1692 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 11.
I20260709 13:02:13.343206  7433 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.345918  1435 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 11 to 15, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 15 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:13.388816  7445 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:12.436226  1612 rpcz_store.cc:276] 0709 13:02:10.770231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.038825 (+268594us) service_pool.cc:224] Handling call
0709 13:02:11.039173 (+   348us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:11.039604 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:11.039634 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:11.040206 (+   572us) log.cc:844] Serialized 52 byte log entry
0709 13:02:11.213990 (+173784us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:11.685909 (+471919us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.365569 (+679660us) log.cc:844] Serialized 14 byte log entry
0709 13:02:12.366017 (+   448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.366122 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:12.366178 (+    56us) raft_consensus.cc:1685] finished
0709 13:02:12.366198 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:12.366366 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48099,"prepare.queue_time_us":136653,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":17317,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":61885,"thread_start_us":1220691,"threads_started":3,"wal-append.queue_time_us":468393}
W20260709 13:02:13.457866  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.441455  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:13.407605  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:13.439752  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:13.523494  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1125) took 1460 ms. Trace:
I20260709 13:02:13.469692  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1337) took 1120 ms. Trace:
I20260709 13:02:13.427553  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:13.553412  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 160014us
I20260709 13:02:13.407729  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.559657  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:13.560465  1690 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:13.397575  7347 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:13.462077  7436 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:02:13.561606  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:13.468226  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:13.572453  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 164918us
W20260709 13:02:13.573830  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 134206us
I20260709 13:02:13.563289  1541 rpcz_store.cc:276] 0709 13:02:12.063330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.063467 (+   137us) service_pool.cc:224] Handling call
0709 13:02:12.063851 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.330160 (+266309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.330191 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.330651 (+   460us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.792787 (+462136us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:12.792830 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.513305 (+720475us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b540012cad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:13.513378 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:13.513461 (+    83us) raft_consensus.cc:1685] finished
0709 13:02:13.513481 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:13.523395 (+  9914us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1118,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"spinlock_wait_cycles":611527936,"thread_start_us":461749,"threads_started":1,"wal-append.queue_time_us":602168}
I20260709 13:02:13.563742  2231 rpcz_store.cc:276] 0709 13:02:12.348834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.348964 (+   130us) service_pool.cc:224] Handling call
0709 13:02:12.349235 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:12.740168 (+390933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.469330 (+729162us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54005b0254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:13.469604 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":897205120}
W20260709 13:02:13.564076  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 157692us
I20260709 13:02:13.564127  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.583737  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.564173  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:13.585060  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:13.564417  1684 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:13.587153  1688 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:13.571419  7353 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:13.562912  2235 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:13.587697  1683 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:13.590150  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.590693  7380 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:13.590889  1683 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:13.591475  7380 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.594313  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:13.594731  7380 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:13.597705  1688 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12.
I20260709 13:02:13.390285  6601 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:13.599290  2236 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:13.400291  7443 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.600673  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:13.564046  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:13.601807  1584 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:02:13.644800  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 80889.4us
I20260709 13:02:13.558410  7381 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.564972  1692 raft_consensus.cc:2368] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
W20260709 13:02:13.653105  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 51036.3us
I20260709 13:02:13.559345  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.653587  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:13.432582  6762 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.432214  7241 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.427901  7352 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.462620  7348 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:13.623222  1683 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:13.427976  7141 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:13.461879  7360 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:13.622206  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 154048us
I20260709 13:02:13.575667  7144 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:13.465389  6996 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.604029  1657 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:13.649186  1684 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:13.398507  7174 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.204s
I20260709 13:02:13.606876  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.691568  1686 raft_consensus.cc:2376] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 12. Re-sending same reply.
I20260709 13:02:13.410809  6976 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.236s
I20260709 13:02:13.645653  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:13.655301  1692 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:02:13.561836  7321 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:13.610765  7281 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:13.671573  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:13.409646  7197 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:13.647265  7430 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:13.679425  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.679672  1684 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:13.716279  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:13.680270  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 76373.3us
I20260709 13:02:13.655050  7205 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:13.684409  2236 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
W20260709 13:02:13.644218  7326 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.952s	user 0.016s	sys 0.032s
W20260709 13:02:13.699993  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.722543  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.697232  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2569) took 1700 ms. Trace:
I20260709 13:02:13.725598  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:13.725904  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.726351  1691 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:13.725256  1538 rpcz_store.cc:276] 0709 13:02:11.996441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.046699 (+ 50258us) service_pool.cc:224] Handling call
0709 13:02:12.047084 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.241709 (+1194625us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.696945 (+455236us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54000ec4d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:13.697164 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2737153920}
I20260709 13:02:13.727011  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:13.563078  7295 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.701789  7348 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:13.711532  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.574826  7445 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:13.731127  7326 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.047s	user 0.016s	sys 0.033s
I20260709 13:02:13.671976  7141 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:13.731655  7257 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:13.657326  7381 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.643361  7444 consensus_queue.cc:1059] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:02:13.746291  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:13.710632  7281 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:13.742307  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 140616us
I20260709 13:02:13.744215  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:13.467716  7441 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.267s
I20260709 13:02:13.726569  1692 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:13.563156  7426 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.705374  7144 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:13.723276  7286 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:13.720049  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 74512.1us
I20260709 13:02:13.764598  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:13.721117  7447 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.299s	user 0.000s	sys 0.001s
I20260709 13:02:13.745141  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:13.769240  1540 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:13.745266  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:13.731725  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:13.736980  7348 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:13.724105  7154 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.003s
I20260709 13:02:13.710456  7197 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:13.753122  1584 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:13.741907  1689 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:13.742179  1684 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:02:13.739106  7295 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:13.713336  7321 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:13.713404  7430 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:13.731375  7205 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:13.736058  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:13.787056  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:13.786051  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:13.735113  7327 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:02:13.741302  7454 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:02:13.658052  7437 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:02:13.747962  1688 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:13.645272  7390 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.106s
I20260709 13:02:13.654685  7428 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 13:02:13.772006  1658 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:13.824483  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 52601.7us
I20260709 13:02:13.826861  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:13.653206  7443 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.675335  7352 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.776877  2229 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:13.741482  6835 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:02:13.778613  1538 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:13.776013  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:13.738385  7457 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.733227  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:13.739917  7445 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:13.789458  7205 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:13.741163  7257 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.760588  7263 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.740211  7326 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:02:13.771128  7244 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:13.788329  1540 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:13.830821  7286 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:13.778246  7435 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:02:13.779384  7176 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:13.720332  7404 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.828877  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:13.773162  7320 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:02:13.740624  7141 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.742762  6601 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.774866  7348 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:13.841310  7381 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.842272  7244 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:13.852103  7169 consensus_meta.cc:287] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.000s
I20260709 13:02:13.852735  7244 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:13.789871  7334 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:13.836941  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 155713us
I20260709 13:02:13.837936  1690 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:13.823159  7327 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:13.780601  7197 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:13.822274  6996 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:13.833150  7449 log.cc:927] Time spent T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 13:02:13.830251  7295 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:13.830076  7088 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.788331  7430 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:13.831060  7144 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:13.886049  7144 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.887611  7329 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.840891  7458 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:02:13.842962  7435 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:13.889441  7435 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:13.838397  7360 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:13.878486  2231 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:02:13.832756  7457 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.839951  7433 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:13.844724  7460 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:13.855671  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:13.880317  7445 raft_consensus.cc:2764] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:02:13.783867  7453 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.107s
W20260709 13:02:13.904838  7451 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:02:13.906347  7352 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:13.896808  7456 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
W20260709 13:02:13.893981  7450 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:02:13.835150  7281 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.883154  7334 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.919042  7348 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.844069  7321 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:13.920272  7402 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.233s	user 0.007s	sys 0.000s
I20260709 13:02:13.923230  7257 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.924619  7426 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.920693  7457 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.927723  7465 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:13.928267  7465 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:13.930778  7443 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:13.932932  7348 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:13.933212  7334 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:13.933849  7465 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.934273  7444 raft_consensus.cc:3055] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:13.920290  7327 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.936374  7286 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:13.942806  7460 raft_consensus.cc:697] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:13.944973  7444 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:13.938062  7435 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.940594  7430 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:13.913585  7360 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:13.954452  7431 raft_consensus.cc:3055] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:13.958215  7431 raft_consensus.cc:740] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:13.965515  7431 consensus_queue.cc:260] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.965971  7430 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:13.961414  7444 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.966691  7431 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:13.969681  7465 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:13.970069  7460 consensus_queue.cc:237] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:13.970849  7444 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:13.973913  7435 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:13.707139  1435 catalog_manager.cc:5697] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 8 to 11, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:02:13.834012  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 58077us
I20260709 13:02:14.014698  1435 catalog_manager.cc:5697] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 12 to 15, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 15 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:14.030800  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1120) took 2036 ms. Trace:
I20260709 13:02:14.032115  1435 catalog_manager.cc:5697] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:14.031652  1537 rpcz_store.cc:276] 0709 13:02:11.994575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:11.994716 (+   141us) service_pool.cc:224] Handling call
0709 13:02:11.995013 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.008049 (+ 13036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.008097 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:12.026277 (+ 18180us) log.cc:844] Serialized 26 byte log entry
0709 13:02:12.496550 (+470273us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:12.496599 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:12.496676 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:13.919652 (+1422976us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:14.029781 (+110129us) raft_consensus.cc:1685] finished
0709 13:02:14.029894 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.030516 (+   622us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12594,"raft.queue_time_us":13249,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":8367,"spinlock_wait_cycles":108353408,"thread_start_us":437806,"threads_started":1,"wal-append.queue_time_us":644310}
W20260709 13:02:14.054308  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 301110us
I20260709 13:02:14.060516  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:14.038137  1435 catalog_manager.cc:5697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:14.065047  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:14.064090  1541 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:14.065977  1582 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:14.062562  1542 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: [12.11-14.12]
I20260709 13:02:14.067675  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:14.068194  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:14.068415  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:14.067494  1435 catalog_manager.cc:5697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 10 to 13, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 13 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:14.069638  1535 raft_consensus.cc:2368] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14.
I20260709 13:02:14.070129  1539 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:14.071023  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:14.071967  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.073055  1537 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:14.074237  1539 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:14.075102  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:14.075209  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.076458  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:14.077126  1613 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:14.077512  1615 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:14.078255  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.080437  1538 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:14.081187  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:14.082106  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.083150  1612 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:14.084478  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:14.085457  1614 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:14.086364  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 999) took 1289 ms. Trace:
I20260709 13:02:14.086803  1617 rpcz_store.cc:276] 0709 13:02:12.796575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:12.796726 (+   151us) service_pool.cc:224] Handling call
0709 13:02:12.797071 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:12.797401 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:12.797460 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:13.486351 (+688891us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b2c0005b3b8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:02:13.486836 (+   485us) log.cc:844] Serialized 26 byte log entry
0709 13:02:13.562443 (+ 75607us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:13.562479 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:13.562550 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:14.085723 (+523173us) raft_consensus.cc:1685] finished
0709 13:02:14.085766 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:14.086153 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92695,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"spinlock_wait_cycles":28541568,"thread_start_us":75197,"threads_started":1,"wal-append.queue_time_us":168537}
I20260709 13:02:14.085337  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:14.070461  1435 catalog_manager.cc:5697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:14.089511  1616 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:14.090963  1435 catalog_manager.cc:5697] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 6 to 11, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 11 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:14.102339  1616 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:14.104168  1435 catalog_manager.cc:5697] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:14.110417  1615 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:14.112571  1435 catalog_manager.cc:5697] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 9 to 15, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 15 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:14.116681  1435 catalog_manager.cc:5697] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 5 to 13, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 13 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:14.120234  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.120611  1614 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
W20260709 13:02:14.121037  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 247538us
I20260709 13:02:14.125228  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:14.126372  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.127271  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.129139  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.130779  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:14.132094  1509 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:14.132893  1542 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:14.133657  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.134620  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.134967  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.135941  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:14.136222  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.136881  1613 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:14.137907  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:14.139132  1616 raft_consensus.cc:2368] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:14.140717  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:14.141624  1538 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:14.141912  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.134670  1612 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:14.143205  1507 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:14.138033  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:14.137096  1584 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:13.878829  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.155684  1690 raft_consensus.cc:2410] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:02:14.156172  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.158453  1691 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:14.158515  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:13.936213  7205 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.177975  1617 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:14.158239  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.214552  1692 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:14.190397  7284 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:14.193949  7174 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.215951  7174 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.216666  7284 raft_consensus.cc:2764] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 13:02:14.217687  7334 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.217865  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:14.218302  7334 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.221233  7390 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.221638  7421 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:14.222525  7421 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:14.199523  7335 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:14.200256  6796 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.202929  7360 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.203712  7376 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:14.227793  7356 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:02:14.228554  7356 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:14.227916  7335 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.229866  7356 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.231639  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:14.231937  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:14.232095  6796 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.232870  1537 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:14.233053  7426 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:14.233666  7426 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:14.234231  7426 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:14.235673  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 74174.3us
I20260709 13:02:14.236021  6796 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:14.236850  1584 leader_election.cc:400] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14.
I20260709 13:02:14.236939  7335 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:14.204614  7375 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:02:14.238035  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 53147.1us
I20260709 13:02:14.238124  7375 raft_consensus.cc:697] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:14.239086  2203 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:14.205899  7393 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.579s	user 0.004s	sys 0.002s
I20260709 13:02:14.214929  7205 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:14.215097  7182 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:14.218636  7174 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:14.196363  7232 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:14.269788  7232 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.271744  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1339) took 1028 ms. Trace:
I20260709 13:02:14.222157  7390 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.272225  2229 rpcz_store.cc:276] 0709 13:02:13.243575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.243934 (+   359us) service_pool.cc:224] Handling call
0709 13:02:13.244235 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:13.831977 (+587742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:14.271238 (+439261us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b540059a954. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:14.271531 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1217146752}
I20260709 13:02:14.274077  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.274679  7390 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:14.274973  1692 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:14.198506  6860 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.239869  7334 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:14.199536  1539 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:14.241986  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:14.242853  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.242950  7197 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.245380  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.281898  1616 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:14.248113  7445 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:14.249140  7423 consensus_queue.cc:1059] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:14.258016  7468 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:14.260736  1584 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:14.270272  1689 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:02:14.239567  7375 consensus_queue.cc:237] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.282295  1613 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:14.224858  7284 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:14.317977  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 79022.5us
I20260709 13:02:14.318176  6796 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:14.318846  6796 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.319396  6796 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:14.319857  2203 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:14.275772  6860 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.284294  7356 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:02:14.325302  7354 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:14.326211  6860 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:14.326037  7354 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.326835  7284 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.327706  7423 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:14.328510  7423 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:14.328953  7423 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:14.317108  7182 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:14.333698  1615 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:14.329209  1582 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:14.336236  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.346347  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.350490  1582 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:14.334280  7284 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:14.356015  1538 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:14.278407  7421 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:02:14.374763  1509 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:14.379137  1692 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:14.379181  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.381201  1689 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:14.381711  7353 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:14.379307  7176 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.382223  7353 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.382659  7353 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:14.382599  7176 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.383737  7421 consensus_queue.cc:1243] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:02:14.384625  1542 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:14.389343  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 134676us
I20260709 13:02:14.391151  7334 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:14.413578  7178 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:14.414377   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:14.416401  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 46818.1us
I20260709 13:02:14.416145  7423 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.417834  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:14.418074  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:14.419123  1582 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:14.423377  7354 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:14.424244  7354 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:14.425861  7182 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.402280  7205 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:14.435288  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.417842  7423 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:14.459538  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 69044.6us
I20260709 13:02:14.434676  7205 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.462993  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.430543  7178 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:14.436404  7421 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:02:14.475067  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:14.475083  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:14.469635  7205 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:14.489566  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:14.490568  2234 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:02:14.492004  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 349148us
I20260709 13:02:14.471328  7421 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.510767  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.535147  1508 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:14.535360  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:14.537572  7433 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:02:14.536232  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:14.539027  1538 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:14.536337  7354 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:14.541548  1692 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:14.547777  7178 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.550803  1582 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:14.478708  7428 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:14.510982  7197 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:14.590339  7428 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.578797  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:14.592031  1686 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:14.549727  1689 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
W20260709 13:02:14.562942  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 85688.1us
I20260709 13:02:14.569957  1692 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:02:14.583952  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:14.553903  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 81638.8us
W20260709 13:02:14.554680  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 72098.3us
I20260709 13:02:14.585901  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.600406  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:14.563405  7281 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.498467  7421 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:14.586472  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.558677  6976 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:14.608206  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.608654  6976 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:14.609107  1689 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:14.579247  6783 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:14.610390  6783 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.611593  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:14.612946  6783 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:14.613788  7214 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:14.626914  7214 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:14.615574  7390 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:14.628242  7390 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:14.629727  7390 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.582173  7176 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:14.616870  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.632081  1692 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:14.617969  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.591005  7428 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:14.613790  1690 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:14.567026  7439 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:14.623644  2203 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:14.554206  7205 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.625862  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:14.609143  6976 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:14.630242  2236 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1492) took 1203 ms (client timeout 1570 ms). Trace:
I20260709 13:02:14.620613  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:14.647302  1538 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:14.640059  7199 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:14.640705  7334 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:14.646435  2236 rpcz_store.cc:269] 0709 13:02:13.427008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:13.427203 (+   195us) service_pool.cc:224] Handling call
0709 13:02:14.629908 (+1202705us) spinlock_profiling.cc:243] Waited 35.6 ms on lock 0x7b54002f49d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:14.630077 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":81831168}
I20260709 13:02:14.633680  7390 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:14.648487  1686 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:14.653892  7205 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:14.655216  7334 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:02:14.661798  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 62907.9us
I20260709 13:02:14.666055  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:14.666989  6796 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.683063  6783 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:14.683647  6783 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:14.684046  6783 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:14.684790  6796 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:14.686394  7430 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.687127  7430 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:14.673919  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 66707.7us
I20260709 13:02:14.498594  7468 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.153s
I20260709 13:02:14.707764  2229 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:14.711349  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:14.713443  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 66706.1us
W20260709 13:02:14.737126  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 71193.3us
I20260709 13:02:14.607537  7426 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.762846  7232 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:14.763886  7205 consensus_queue.cc:1059] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:14.711212  7281 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.765998  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:14.766350  7471 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.442s	user 0.000s	sys 0.002s
I20260709 13:02:14.767069  7353 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:14.767688  7480 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.448s	user 0.000s	sys 0.002s
I20260709 13:02:14.771082  6796 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.771813  7354 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:14.781469  7421 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.782985  6855 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:14.786142  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:14.787416  7489 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.467s	user 0.000s	sys 0.002s
I20260709 13:02:14.789927  6783 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:14.793712  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 49614.6us
I20260709 13:02:14.743343  7439 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:14.744400  7430 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:14.798112  7197 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:14.807165  7426 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:14.829825  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 95648.4us
I20260709 13:02:14.830263  7295 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:14.830705  7460 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:02:14.831810  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 56023.6us
I20260709 13:02:14.840250  7141 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:14.850797  7428 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:14.855902  7487 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.537s	user 0.003s	sys 0.000s
W20260709 13:02:14.865387  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:14.872236  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 295533us
W20260709 13:02:14.874738  7375 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.006s	sys 0.041s
W20260709 13:02:14.880275  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 234503us
I20260709 13:02:14.925284  1689 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:02:14.902736  7197 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:14.888713  1542 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:14.792936  6976 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.892895  1540 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:14.922796  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:14.923413  1657 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.570s (SENT) [suppressed 15 similar messages]
I20260709 13:02:14.893392  6796 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:14.925449  7375 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.007s	sys 0.042s
I20260709 13:02:14.937314  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:14.938273  1685 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:14.898562  1539 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
W20260709 13:02:14.900609  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 134786us
I20260709 13:02:14.901302  7404 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:14.901598  7421 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:14.889463  6855 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:14.943069  1542 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:14.911055  7431 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.943984  6855 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.944681  7431 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:14.911926  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:14.912984  7177 raft_consensus.cc:3055] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:14.872421  7214 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:14.931015  7197 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:14.922114  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:14.886094  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:14.955426  1507 leader_election.cc:400] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
I20260709 13:02:14.967881  1507 leader_election.cc:403] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:14.955299  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.947549  7431 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:14.945899  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:14.954610  7439 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:14.970741  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:14.971309  1609 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:14.972046  1686 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:14.973488  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:14.974107  1535 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:14.971792  1611 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:14.948024  6976 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:14.945986  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:14.971401  1536 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:14.945400  6855 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:14.956274  7426 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:14.980051  7353 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:14.954869  7281 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:14.986924  1507 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:14.988088  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:14.988299  1658 leader_election.cc:400] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:14.963572  7177 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:15.010537  1613 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 12. Current term is 13. Ops: [12.5-12.5]
I20260709 13:02:15.010366  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:15.011190  1612 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:02:15.019150  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:15.021311  1611 raft_consensus.cc:2368] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14.
I20260709 13:02:15.040395  1507 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:15.040670  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:15.041385  1609 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:15.041656  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:15.042162  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:15.042438  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:15.043267  1615 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 12: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:15.042192  1613 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.045086  7257 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.047973  1615 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:15.011355  7281 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.017287  7426 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.051898  1685 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:15.051015  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:15.056243  7176 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:15.056473  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:15.065783  1616 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:02:15.057960  1617 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:15.057951  1615 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:15.058620  1613 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:15.070072  6601 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.070600  6601 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.073494  7453 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:02:15.074191  7176 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:15.075294  7176 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.075706  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:15.077086  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:15.077824  1610 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.077858  7232 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:15.078833  7232 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:15.068017  7281 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.001250  7404 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:02:15.082448  6783 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:15.071871  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 83757.2us
I20260709 13:02:15.062354  7257 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.088840  6601 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:15.098110  1609 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:15.107420  1616 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:15.095450  7281 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:15.109256  6796 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:15.109915  6796 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:15.110468  6796 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.114099  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:15.114763  7257 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:15.115511  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:15.117323  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:15.120355  1617 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:15.121487  6762 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:15.116789  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:15.122334  7214 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:15.123368  1613 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:15.126526  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:15.085587  7453 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:15.155371  6796 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:15.134002  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:15.172955  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:15.173784  1685 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:15.169094  1610 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:15.177940  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:15.179865  7497 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.134s	user 0.002s	sys 0.000s
I20260709 13:02:15.182529  7426 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:15.189651  7326 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:15.190510  7326 raft_consensus.cc:697] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:15.190915  7334 consensus_meta.cc:287] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.002s
I20260709 13:02:15.191584  7334 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.191880  7326 consensus_queue.cc:237] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.193089  1618 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:15.171478  7428 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:15.197790  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:15.097450  7353 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:02:15.198696  1686 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:15.199390  7353 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:15.199882  7353 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:15.204540  1509 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:15.214663  6855 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:15.215222  6855 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.215664  6855 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:15.217710  7177 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.219519  7177 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:15.204766  7430 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:15.221768  7430 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:15.222111  7453 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.205621  7375 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.223719  7375 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.225672  7295 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:15.206467  7404 consensus_queue.cc:1059] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:15.226333  7295 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.227111  7404 raft_consensus.cc:3055] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:15.227526  7404 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:15.231261  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.211169  7197 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.212596  6762 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.213101  1614 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 13:02:15.239281  1614 pending_rounds.cc:85] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 11
I20260709 13:02:15.235901  1535 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:15.243220  1610 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:15.248314  7174 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:02:15.249565  1685 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:15.250231  1543 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:15.251349  7174 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:15.239769  1614 pending_rounds.cc:107] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 13:02:15.253299  1614 raft_consensus.cc:2889] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:15.254776  7251 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:15.255445  7251 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.256505  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:15.257121  1684 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.257668  7195 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:15.260462  7232 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:02:15.260594  7154 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:02:15.228508  7404 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.226365  7375 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:15.265545  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 397910us
I20260709 13:02:15.229089  7295 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:15.276047  1686 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:15.201020  7353 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.223145  7430 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.295251  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:15.278766  7205 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.278255  7251 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:15.293000  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 281191us
I20260709 13:02:15.292362  7431 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:15.356010  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 112540us
I20260709 13:02:15.353631  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:15.295636  7182 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:02:15.362092  7230 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14.
I20260709 13:02:15.361770  7205 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.312601  7154 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:15.364755  7154 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.367565  7154 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:15.292327  6976 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 13:02:15.369719  2201 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:15.370709  6783 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:15.374297  7197 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:15.374378  7431 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.376912  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:15.365458  7375 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.311415  7295 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:15.311129  7257 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:15.377116  7431 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:15.388387  7431 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:15.353175  7392 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.389097  7392 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.389644  7426 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:15.389889  2201 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:15.390126  7426 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.390504  7431 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:15.390650  7426 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.378968  7404 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.387425  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:15.387300  7375 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.277159  6762 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.388562  7205 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:15.386035  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 76605.5us
I20260709 13:02:15.382258  7197 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:15.396775  1684 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:15.402956  7430 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:15.419371  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.419212  7431 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.426673  7195 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:15.427366  7195 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:15.410089  7334 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:15.399027  7295 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.403301  7257 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.428665  7195 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.409513  7178 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:15.446743  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:15.454900  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.450052  7178 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:15.489089  2201 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:15.466110  7195 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:15.475508  7176 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.507622  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:15.511247  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:15.512408  7353 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:15.512998  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:15.514690  7327 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:15.516081  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:15.517535  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:15.517850  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:15.517180  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:15.517896  7176 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.520952  7508 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 13:02:15.526508  7295 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:15.527642  7375 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:15.528743  7513 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:15.530365  7257 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:15.531226  6796 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:15.574047  6796 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.575457  7176 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:15.531395  7499 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 13:02:15.532433  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:15.577459  2234 raft_consensus.cc:2410] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:02:15.517222  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:15.578431  1537 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:15.499658  7496 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.579391  7496 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.537266  7522 consensus_queue.cc:1059] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:15.544610  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:15.547101  7512 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:02:15.547319  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:15.491452  7178 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:15.556370  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 48867.5us
I20260709 13:02:15.583765  1540 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:15.584021  2236 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:15.558377  2230 raft_consensus.cc:2376] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17. Re-sending same reply.
I20260709 13:02:15.561805  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:15.564391  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:15.570365  7392 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:15.569073  1539 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:15.589936  1534 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
W20260709 13:02:15.592244  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.653143  1537 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:02:15.561896  7505 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:02:15.550596  7327 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:02:15.596776  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 344545us
I20260709 13:02:15.661976  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:15.652808  7392 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.677917  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:15.684326  1617 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:15.590304  6796 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:15.652976  7496 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:15.656862  7404 raft_consensus.cc:2764] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 13:02:15.685680  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:15.686128  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:15.686722  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:15.653497  7354 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:15.695276  1610 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:15.688189  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:15.695832  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:15.696161  1612 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:15.674490  1509 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:15.688930  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:15.690441  1616 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:02:15.692214  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:15.687395  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:15.663208  7353 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:15.698323  1611 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:15.695300  1609 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:15.700172  1686 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:02:15.697446  1616 pending_rounds.cc:85] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 11
I20260709 13:02:15.705483  1616 pending_rounds.cc:107] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:02:15.705858  1616 raft_consensus.cc:2889] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:15.679028  7295 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:15.697779  1690 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
W20260709 13:02:15.711774  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 222764us
I20260709 13:02:15.678601  7214 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:15.715907  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:15.718119  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:15.723372  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 53522us
I20260709 13:02:15.728938  1582 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:15.677456  7375 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:02:15.711189  7431 raft_consensus.cc:3055] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:15.755072  7431 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:15.756147  7431 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.757546  6762 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:15.713440  7353 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.750289  7327 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.762806  7327 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:15.763527  7353 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:15.708189  7392 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.722930  7519 consensus_queue.cc:1059] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:02:15.766569  7392 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:15.728186  7214 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.769321  1542 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:15.772941  1613 raft_consensus.cc:2410] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:02:15.749012  2201 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:15.710958  7230 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:15.775022  1539 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:15.753492  7521 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:15.753822  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.778800  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.754299  1683 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:02:15.753144  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:15.780351  1687 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:15.778790  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.776255  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 47550.3us
I20260709 13:02:15.726876  7453 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:02:15.789765  1684 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:15.796679  1685 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:15.718562  7174 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 13:02:15.800169  7257 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:15.805325  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:15.806022  1692 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.770833  7214 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:15.812065  1687 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:15.718750  7354 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.795038  7182 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:15.792595  7439 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.819979  1582 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:15.825332  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:15.773476  7295 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:15.821841  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:15.826011  1582 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:15.826588  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 78062.5us
I20260709 13:02:15.826748  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:15.828871  1684 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:15.806459  7195 raft_consensus.cc:3055] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:15.831138  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:15.831509  2201 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:15.834909  1687 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:15.824687  7230 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:15.829734  7404 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:15.846561  1685 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:02:15.846627  1683 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:15.830727  7326 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:15.830152  7195 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:15.848728  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:15.835843  7510 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:15.854331  1539 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:15.848322  1685 pending_rounds.cc:85] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 7
I20260709 13:02:15.816895  7521 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:15.861080  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.864322  1688 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 13:02:15.853928  7230 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.866310  7295 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.868187  7404 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.850523  7431 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.869937  7195 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.870605  7404 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:15.871315  7195 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:15.841572  7334 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:15.873824  1685 pending_rounds.cc:107] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:02:15.874225  1685 raft_consensus.cc:2889] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:15.842419  7182 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:15.879653  1542 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:02:15.876715  7326 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:15.834834  7508 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.021s
W20260709 13:02:15.889014  7526 log.cc:927] Time spent T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:02:15.842082  7439 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:15.889307  7326 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.857955  7510 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.932842  7510 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:15.889973  7182 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:15.936589  1685 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:15.936928  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.891203  6976 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:15.895074  7513 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:15.875808  7515 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 13:02:15.895862  7257 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:15.942010  7439 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:15.943482  7257 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:15.896369  1684 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:15.859156  7536 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:02:15.901661  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 70374.1us
I20260709 13:02:15.850530  7460 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:15.902324  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:15.903936  7354 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:15.886765  7334 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:15.910094  7534 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:02:15.914850  1692 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:15.926856  7508 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:15.927878  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:15.931383  6855 consensus_meta.cc:287] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.715s	user 0.008s	sys 0.004s
W20260709 13:02:15.938760  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:15.974293  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 46538.6us
I20260709 13:02:15.843005  7426 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:15.946637  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:15.981115  1684 raft_consensus.cc:2368] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18.
W20260709 13:02:15.945760  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 108044us
I20260709 13:02:15.959174  2201 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:15.960225  1685 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:02:15.961185  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:15.943845  7214 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:02:15.962234  1687 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:02:15.962574  7521 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:15.962888  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:15.963579  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:15.964325  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:15.957785  6976 consensus_queue.cc:1243] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:02:15.965487  7460 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:15.949851  7430 consensus_meta.cc:287] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.006s
W20260709 13:02:15.968343  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 50921.5us
I20260709 13:02:15.974596  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:15.975644  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:16.003562  1692 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:16.004099  1683 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:02:15.946543  7257 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:15.987483  6855 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:16.016021  1684 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:16.016458  1684 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:15.994990  1613 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:16.026525  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:15.966087  7334 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.006938  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:16.007608  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:16.008510  1539 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:16.010874  1687 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:16.043926  1542 raft_consensus.cc:2410] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 13:02:16.019743  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:15.996273  1688 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:16.017283  1684 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.045441  1692 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:02:16.011991  1616 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:16.011242  7178 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.012583  7510 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:16.014010  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 54980.5us
I20260709 13:02:16.014719  7508 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:16.063872  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:15.998510  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:16.064620  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:16.064940  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 66511.5us
I20260709 13:02:16.065270  1613 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:16.066350  1611 raft_consensus.cc:2410] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:02:16.027063  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:16.027473  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:16.020622  7460 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:16.049242  1507 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:16.051554  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:16.068804  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:16.011395  7530 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.218s	user 0.002s	sys 0.000s
I20260709 13:02:16.054840  7430 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.054996  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:16.055604  7521 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:16.056205  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.059486  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:15.995723  7513 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:15.980177  7354 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.061620  1684 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:16.017976  1609 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:16.031522  1582 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:02:16.080648  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 49208.2us
I20260709 13:02:16.002854  7421 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:16.065951  2203 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:16.030105  7426 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:16.069893  1616 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:16.017338  6855 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:16.074914  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:16.076220  1537 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:16.060629  7257 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.066423  7177 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:16.089383  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:16.098004  1692 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:16.092536  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:16.046206  7182 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:02:16.072678  7536 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:02:16.084950  1658 connection.cc:256] RPC call timeout handler was delayed by -1.414456987000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
I20260709 13:02:16.078809  7513 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.103356  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:16.102191  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.102116  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:16.112504  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:16.078994  7230 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:16.112735  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:16.108155  7195 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:16.116945  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:16.094978  7141 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:02:16.119774  1658 connection.cc:256] RPC call timeout handler was delayed by -1.371256987000038s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
W20260709 13:02:16.127599  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2965569870000309s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
W20260709 13:02:16.128068  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2173569870000165s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 13:02:16.128469  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2002569870000457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
W20260709 13:02:16.128899  1658 connection.cc:256] RPC call timeout handler was delayed by -1.1210569870000313s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
W20260709 13:02:16.129397  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0346569870000053s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
I20260709 13:02:16.117625  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:16.125731  1687 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:16.103736  7508 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:16.103927  7460 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.102931  7177 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:16.080248  7430 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:16.146934  2230 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:16.120422  7513 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:16.093577  7354 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.148558  1683 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:16.153957  2236 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 20. Ops: []
I20260709 13:02:16.126724  7182 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:16.154503  7404 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.158015  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.133328  7230 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:16.124639  7536 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:16.108711  7154 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:02:16.084513  7521 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.099395  7421 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.167440  2229 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:16.168025  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.134311  7431 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:16.116890  7232 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:02:16.147493  7177 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:16.166949  7182 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.205173  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:16.189280  1684 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:02:16.157410  7295 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:16.155897  7508 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:16.186862  2234 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:16.207736  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.207351  1684 pending_rounds.cc:85] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 13
I20260709 13:02:16.212882  1684 pending_rounds.cc:107] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:02:16.213584  1684 raft_consensus.cc:2889] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:16.199137  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:16.218724  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:16.219807  1617 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:16.202571  7522 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.230502  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:16.229827  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:16.229971  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:16.152109  7460 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:16.195163  7232 consensus_queue.cc:1243] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:02:16.241855  2230 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:16.231096  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:16.171028  7404 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:16.241523  2229 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:16.240322  1614 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:16.205005  7182 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:16.196689  7141 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:16.246673  1616 raft_consensus.cc:2410] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 16 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 13:02:16.260273  7392 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:16.260641  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.260859  7392 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:16.262050  7392 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:16.262720  2234 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:16.263474  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.198475  7431 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.254659  7178 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:16.293112  7543 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.299s	user 0.000s	sys 0.003s
W20260709 13:02:16.299919  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 107269us
I20260709 13:02:16.222376  7295 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.301896  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.304164  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.317840  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.289654  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.319418  2236 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:02:16.320096  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.258005  7522 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:16.338119  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:16.339346  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:16.344045  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:16.290505  7553 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.345817  7553 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.346256  7496 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:16.301410  7392 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:16.355669  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:16.362394  2203 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:16.360210  7519 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.341913  7533 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:16.377048  1613 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 13:02:16.378087  1658 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:16.370524  6796 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:16.377665  1613 pending_rounds.cc:85] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 6
I20260709 13:02:16.390053  1613 pending_rounds.cc:107] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:02:16.390412  1613 raft_consensus.cc:2889] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:16.347754  7257 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:16.362146  7178 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:16.356165  7554 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:16.395602  7550 log.cc:927] Time spent T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.228s	user 0.002s	sys 0.000s
W20260709 13:02:16.400701  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 52726.9us
W20260709 13:02:16.389459  7547 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 13:02:16.406083  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 86595.8us
I20260709 13:02:16.389750  6796 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:16.390331  7556 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.425493  2229 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:16.370079  7519 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.374902  7197 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:02:16.348886  7141 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:16.399366  7354 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:16.399027  7554 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.449597  7554 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:16.452471  7562 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:02:16.301401  7536 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.405746  7533 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:16.440438  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.443022  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:16.406407  7430 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:16.444154  7551 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 13:02:16.446619  7571 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.447001  7553 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:16.347565  7508 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.447644  7519 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:16.448280  7567 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.426963  7521 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:16.441038  7421 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:16.352809  7230 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:16.454351  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.455839  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:16.456717  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:16.428225  6796 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:16.475831  1610 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:16.497465  2229 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:16.471441  1509 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:16.499714  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:16.476589  7326 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.533458  7326 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:16.544679  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 45053.7us
W20260709 13:02:16.501263  7549 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
I20260709 13:02:16.501487  7521 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:16.503791  7404 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:16.570140  7526 log.cc:927] Time spent T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:02:16.535957  7460 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:16.548703  7232 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:16.578778  7562 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:02:16.590466  2234 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:16.473802  7567 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:16.550026  7327 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:16.599313  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 160488us
W20260709 13:02:16.605414  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 240471us
W20260709 13:02:16.609727  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.613771  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:16.457489  7556 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:16.476475  7571 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:16.584780  7205 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:02:16.623876  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.605414  7232 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.612993  7327 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:16.629036  1613 raft_consensus.cc:2410] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
W20260709 13:02:16.629848  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 97994.5us
I20260709 13:02:16.631335  1610 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:16.641577  2230 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:16.643611  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:16.645684  2231 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:16.622305  7295 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:16.628114  7561 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 13:02:16.655532  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.628437  7232 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:16.625204  7522 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:16.642028  7327 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:16.657178  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 43541us
W20260709 13:02:16.660181  7570 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 13:02:16.655774  7580 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:16.663235  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.663964  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.664780  2228 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:02:16.664944  7569 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 13:02:16.665539  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:16.666698  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:16.666795  7334 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:16.671626  7334 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:16.672165  7334 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:16.672366  7431 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:16.674598  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:16.675436  7178 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:16.678483  7510 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:16.684734  7571 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:16.694877  7556 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:16.650128  7439 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:16.681077  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:16.666617  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:16.678542  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:16.705456  1616 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:16.708557  7513 consensus_meta.cc:287] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.521s	user 0.005s	sys 0.006s
I20260709 13:02:16.712136  1613 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:16.709875  7580 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 13:02:16.714380  7182 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:16.706039  7195 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.578s	user 0.006s	sys 0.006s
I20260709 13:02:16.718034  7195 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:16.717353  7334 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:16.719430  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.713603  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.725543  1618 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 17: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:02:16.660229  7572 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:02:16.732960  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:16.733250  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 107600us
I20260709 13:02:16.739316  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.744364  7334 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:16.709110  7513 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.754185  7326 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:16.755719  7567 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:16.757763  7554 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.758221  7327 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.759316  2229 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:16.761683  7327 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:16.726480  7439 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:16.772188  7439 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:16.774225  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.775966  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:16.779071  2237 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:16.780534  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:16.777889  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.717325  6796 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.881049  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:16.779687  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.780089  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.984973  1616 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:16.831559  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:17.016822  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:16.880496  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:17.020229  1685 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:16.818722  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:17.021351  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:17.021593  1688 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:17.021634  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:16.894356  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:17.018116  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:16.801203  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 63197.1us
I20260709 13:02:16.778479  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:17.016283  1689 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:17.041673  2235 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:17.027354  2234 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:17.032021  1692 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 13:02:16.781284  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:16.780916  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:17.045547  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 256817us
W20260709 13:02:16.776309  7565 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.341s	user 0.002s	sys 0.000s
I20260709 13:02:17.045603  2230 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:17.046875  1683 raft_consensus.cc:2410] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:02:17.053162  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:16.994084  7177 consensus_meta.cc:287] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.785s	user 0.013s	sys 0.000s
I20260709 13:02:17.053982  7177 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:17.023264  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:17.027788  2233 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
W20260709 13:02:16.894227  7496 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.547s	user 0.008s	sys 0.003s
I20260709 13:02:17.050724  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:17.057215  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 276000us
I20260709 13:02:16.782280  7557 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.270s
I20260709 13:02:17.051728  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:17.046334  1583 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:17.057040  2231 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:16.994539  7232 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:16.832000  7281 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:17.058192  1687 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:17.058689  1684 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:02:17.063791  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:17.065861  7580 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:17.065001  7496 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:17.067103  7580 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.067312  1658 leader_election.cc:400] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:17.067656  1658 leader_election.cc:403] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:02:17.069007  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:17.070748  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:17.070073  1689 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:02:17.062522  7564 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:17.073774  7564 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:17.074231  7564 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:17.074261  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:17.070744  7214 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:02:17.077639  7431 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:17.078012  6976 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.078769  1687 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:17.079950  1689 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:17.081038  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:17.082793  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.078555  6976 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:17.082705  2201 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:17.084661  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:17.085342  1687 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:17.086431  2201 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:17.069186  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:17.067509  7557 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:17.087916  2201 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:17.091216  7281 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:17.091696  7281 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:17.092074  7281 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:17.092382  7532 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:17.093596  7544 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:17.089251  7426 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:17.094209  7544 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:17.067634  7580 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:17.096561  7532 consensus_queue.cc:1059] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:17.100709  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:17.102506  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:17.102669  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:17.103259  1687 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:17.104038  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:17.104615  1689 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:17.069878  1692 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:16.869925  7510 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.200s
I20260709 13:02:17.107039  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:17.108376  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:17.109578  1687 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:17.110244  1692 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:17.111362  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:17.112224  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:17.112869  1691 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:02:17.113991  1684 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:17.097210  7544 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:17.116248  1683 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:17.116751  1687 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:17.101749  7508 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:17.098004  7582 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:17.123118  1692 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:17.105952  7426 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.097298  7580 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:17.107906  7178 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:17.131888  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64710.7us
I20260709 13:02:17.114506  6796 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:17.119374  7392 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.758s	user 0.013s	sys 0.000s
I20260709 13:02:17.126081  7576 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:17.117256  7567 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:02:17.124279  7582 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:17.147607  7582 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:17.148744  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.148923  7582 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:17.133548  7205 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.152665  7205 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:17.154806  1509 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:17.155668  7439 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:17.156199  7439 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:17.157097  7439 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:17.159976  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:17.131393  7508 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:17.132967  7177 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:17.155691  7205 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:17.118997  7214 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:17.135677  7195 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:17.137270  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:17.138073  1617 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:02:17.138517  7556 raft_consensus.cc:3055] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:17.139575  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:17.140331  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:17.140419  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:17.192306  1538 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:17.142109  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:17.142280  7182 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:17.143411  7568 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.531s	user 0.005s	sys 0.009s
I20260709 13:02:17.153200  7513 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:17.173679  6976 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:17.165344  7426 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:17.190308  1617 pending_rounds.cc:85] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 8
W20260709 13:02:17.197549  7593 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 13:02:17.197641  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:17.198591  1609 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 13:02:17.199640  7553 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.199095  7508 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:17.199015  7195 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:17.203699  7232 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:17.191560  7556 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:17.208187  7281 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:17.189610  7214 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:17.214186  1543 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:17.125267  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:17.217476  1617 pending_rounds.cc:107] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:02:17.247887  1617 raft_consensus.cc:2889] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:17.248274  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 123089us
W20260709 13:02:17.237165  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 104251us
I20260709 13:02:17.244911  2236 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:17.249460  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:17.251669  1507 leader_election.cc:336] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.647s (SENT)
W20260709 13:02:17.256163  1507 leader_election.cc:336] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.697s (SENT)
I20260709 13:02:17.250874  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:17.286154  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 177840us
I20260709 13:02:17.290980  2201 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:17.253563  7214 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:17.301102  1539 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:17.293263  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:17.253965  6976 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:17.243182  7182 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:02:17.284089  7232 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.285596  7326 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:17.305081  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:17.302559  1583 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:17.242036  7553 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:17.318392  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 168584us
I20260709 13:02:17.306712  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2145) took 1102 ms. Trace:
W20260709 13:02:17.329636  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 144468us
I20260709 13:02:17.307884  2201 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:17.346210  1583 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:17.350013  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 45010.9us
I20260709 13:02:17.351533  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:17.303910  7575 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:17.364058  1538 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:17.319495  1611 rpcz_store.cc:276] 0709 13:02:16.204318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.204435 (+   117us) service_pool.cc:224] Handling call
0709 13:02:16.231899 (+ 27464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:16.292506 (+ 60607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:16.292539 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:16.293103 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 13:02:16.354060 (+ 60957us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:16.354116 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:16.354214 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:16.773278 (+419064us) spinlock_profiling.cc:243] Waited 8.95 ms on lock 0x7b48001a01d8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:02:17.142778 (+369500us) spinlock_profiling.cc:243] Waited 6.43 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:17.306490 (+163712us) raft_consensus.cc:1685] finished
0709 13:02:17.306512 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:17.306635 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6272,"raft.queue_time_us":76471,"raft.run_cpu_time_us":2918,"raft.run_wall_time_us":449363,"spinlock_wait_cycles":35391616,"thread_start_us":104879,"threads_started":2,"wal-append.queue_time_us":186224}
W20260709 13:02:17.371459  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 115359us
W20260709 13:02:17.378762  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 195041us
I20260709 13:02:17.309465  7510 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:17.308817  7567 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:17.315033  7326 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.356294  6976 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:17.356528  7392 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:17.366645  7576 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:17.374222  7575 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.365089  7574 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:02:17.397612  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 110406us
I20260709 13:02:17.305565  7571 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 13:02:17.368490  7360 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:17.393152  7326 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:17.417618  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:17.389740  7510 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.386495  7557 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:17.389550  7567 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.341367  7232 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:17.396021  7513 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:17.421548  7575 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:17.421497  7574 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.308260  7556 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:17.398535  7564 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:17.460914  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.400220  7544 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
W20260709 13:02:17.472764  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 142082us
W20260709 13:02:17.478926  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 99596.8us
I20260709 13:02:17.480435  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:17.442729  7557 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:17.481650  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:17.482467  2237 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:17.483147  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:17.467965  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 104887us
I20260709 13:02:17.442526  7567 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:17.442154  7510 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:17.491534  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.492218  2228 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 13:02:17.502950  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.509096  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:17.509194  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:17.509630  2232 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:17.509742  1609 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:17.509857  2233 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:17.465147  7554 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:17.440541  7360 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:17.519865  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:17.520530  1618 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:17.521528  1616 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:17.520066  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:17.447188  7421 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:02:17.528019  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:17.503716  7353 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:17.504160  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:17.467550  7574 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:17.505046  1685 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:17.499060  1689 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:17.525777  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:17.493212  7556 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:17.493779  7557 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:17.478500  7609 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:17.529537  1687 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:17.533687  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:17.507220  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:17.537228  1692 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:17.537161  1611 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:02:17.542755  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:17.519913  7426 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:17.522825  7195 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:17.544476  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.551395  2203 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:17.519740  7439 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:17.551508  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:17.558727  1688 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
W20260709 13:02:17.554816  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 164203us
W20260709 13:02:17.560591  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.570047  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:17.570931  1616 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:17.579794  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:17.581373  7553 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:17.581627  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:17.545133  7281 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:17.558211  7426 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:17.604969  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:17.605834  1616 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:17.560606  7522 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:17.605863  2231 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:17.548002  7182 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:02:17.540427  7610 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.062s
I20260709 13:02:17.583387  7178 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:17.572136  7195 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:02:17.637872  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:17.638401  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.638768  1692 raft_consensus.cc:2410] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:02:17.564659  7439 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:17.679018  1618 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:17.677109  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:17.699267  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:17.699831  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:17.705937  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 348777us
I20260709 13:02:17.648268  7232 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:17.681813  6855 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:17.646193  7522 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:17.721305  7582 consensus_meta.cc:287] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.570s	user 0.012s	sys 0.002s
W20260709 13:02:17.783466  7596 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
W20260709 13:02:17.784281  7606 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 13:02:17.784363  7612 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:02:17.713240  7564 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:17.783335  7613 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.214s
W20260709 13:02:17.790570  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 68678.4us
I20260709 13:02:17.783874  7621 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:17.784298  7603 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
W20260709 13:02:17.793658  7604 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:02:17.816488  1611 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:17.783402  7619 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:17.798041  7360 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:17.726198  6855 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:17.790009  7618 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:17.945855  2236 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:17.895685  1691 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:02:17.833240  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 267563us
I20260709 13:02:17.967694  2237 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:17.967074  2201 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:18.003582  1609 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:17.965910  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:18.006795  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 67505.3us
I20260709 13:02:18.007896  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:17.969398  1686 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.018s	sys 0.007s
W20260709 13:02:18.010087  1686 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.018s	sys 0.007s
W20260709 13:02:17.910234  7629 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 13:02:18.017834  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.019784  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1250) took 1113 ms. Trace:
I20260709 13:02:18.014813  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:17.964416  7618 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.023682  1686 rpcz_store.cc:276] 0709 13:02:16.897973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:16.898210 (+   237us) service_pool.cc:224] Handling call
0709 13:02:17.049876 (+151666us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.052153 (+  2277us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:17.252047 (+199894us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.010548 (+758501us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.010612 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:18.010662 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.010786 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.010955 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2216,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":199207,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":20907,"thread_start_us":907712,"threads_started":3,"wal-append.queue_time_us":243023}
I20260709 13:02:17.905670  7630 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:02:18.026551  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 60721.6us
I20260709 13:02:17.856436  7564 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:17.868782  7426 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:17.902371  7625 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:18.036587  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:18.037009  1618 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:18.047118  1538 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:17.936650  7353 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:17.962419  7624 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.035687  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:17.867924  7439 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:17.876652  7631 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:17.874588  7633 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:17.882841  7232 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.158s
I20260709 13:02:18.063458  2203 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:17.912431  7522 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:17.867136  7615 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.189s
I20260709 13:02:18.015903  7576 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.014958  7554 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:02:18.055557  7178 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:18.026068  7544 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:18.059139  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:18.050531  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 419973us
I20260709 13:02:18.028728  7618 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:18.040624  7536 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:18.057866  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:18.071978  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:17.890256  7205 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.152s
I20260709 13:02:17.831804  7621 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:18.060497  1543 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:17.877883  7634 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.202s
I20260709 13:02:18.051862  7353 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:18.084086  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:18.049654  7564 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:18.054188  7426 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.080735  7625 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:18.094208  7574 consensus_meta.cc:287] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.000s
I20260709 13:02:18.083024  7544 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.063302  7623 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:02:18.088956  7536 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:18.105825  7575 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.627s	user 0.012s	sys 0.000s
I20260709 13:02:18.059746  7624 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:18.132835  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 48878.1us
I20260709 13:02:18.133906  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:18.133646  7625 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:18.135277  7624 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:18.137223  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:18.137734  6855 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:18.137917  1686 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:18.132293  7536 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:18.138734  7628 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.139264  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.139793  7628 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.140679  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:18.130227  7574 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:18.141712  7637 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:18.142508  7637 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:18.143188  7637 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:02:18.143777  1615 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:02:18.144955  1658 leader_election.cc:400] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14.
I20260709 13:02:18.145777  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:18.146047  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.146775  1686 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:18.126807  7582 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.147485  7645 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:18.147575  7582 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.149147  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.065824  7633 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.149999  7645 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:18.150573  7645 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:18.151474  7633 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:18.151540  7645 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.157328  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.143400  7574 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:18.158303  7633 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:18.158759  7633 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:18.159154  7633 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:18.126924  7544 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:18.164062  7536 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.130653  7636 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.170586  7636 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.174113  7624 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.174892  7624 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:18.179169  1691 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:02:18.179951  1691 raft_consensus.cc:606] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:18.180274  1691 raft_consensus.cc:614] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:18.180462  7214 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.866s	user 0.000s	sys 0.010s
I20260709 13:02:18.181212  7624 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:18.173281  7636 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:18.169554  7431 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:18.182755  7431 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:18.183758  7431 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.181861  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:18.185164  7431 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:18.185062  1543 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 13:02:18.188712  7567 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.651s	user 0.009s	sys 0.000s
I20260709 13:02:18.190107  7567 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:18.193328  7614 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:18.193842  7614 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:18.194679  7614 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.195274  7567 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:18.192687  7557 consensus_meta.cc:287] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.641s	user 0.009s	sys 0.000s
I20260709 13:02:18.195963  7614 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:18.196141  7557 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:18.197403  1613 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:18.198978  7557 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:18.181046  7214 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.200812  1614 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:18.086727  7197 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 13:02:18.203492  1686 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:18.204012  7214 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:18.157239  7326 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.713s	user 0.007s	sys 0.000s
I20260709 13:02:18.170586  7575 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:18.208577  7575 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:18.208936  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.077174  7631 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.105697  7496 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:18.082295  7576 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:18.214197  1509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.003s	sys 0.014s
I20260709 13:02:18.122279  7610 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:18.146034  7638 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:02:18.167810  7633 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:18.268033  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 59189.3us
W20260709 13:02:18.145181  7556 consensus_meta.cc:287] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.604s	user 0.005s	sys 0.004s
I20260709 13:02:18.104045  7178 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:18.288439  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 411206us
W20260709 13:02:18.297492  2234 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.012s	sys 0.018s
W20260709 13:02:18.289165  1509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.003s	sys 0.015s
W20260709 13:02:18.327966  2234 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.012s	sys 0.018s
W20260709 13:02:18.328064  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 690977us
I20260709 13:02:18.316668  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:18.318786  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:18.329041  1692 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:18.320001  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:18.322027  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:18.325268  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:18.204121  7197 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:18.288022  7636 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:18.330060  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 268830us
I20260709 13:02:18.294787  7326 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.332634  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:18.332679  1582 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:18.323467  7327 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.007s	sys 0.028s
I20260709 13:02:18.337816  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.305557  7575 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:18.338219  1691 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:18.339337  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:18.362882  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:18.309299  7496 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:18.362411  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:18.336351  7214 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:18.368340  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.373836  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:18.375268  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:18.376966  1617 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:02:18.378295  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:18.332571  7197 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:18.376211  1543 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
W20260709 13:02:18.341216  7327 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.007s	sys 0.029s
I20260709 13:02:18.387703  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:18.390398  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:18.363778  7392 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:18.367590  7510 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.849s	user 0.010s	sys 0.006s
I20260709 13:02:18.400730  1538 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:18.366858  7431 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:18.372519  7574 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:18.361500  7619 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:18.372303  7556 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:18.398169  7636 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:18.401216  7637 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:18.471979  7392 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.530558  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:18.371687  7575 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.328897  7353 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.397989  7610 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:18.510438  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:18.404489  7536 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:18.542459  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:18.491447  7508 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.471063  7582 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:18.381963  7617 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.130s
W20260709 13:02:18.551542  2231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.023s	sys 0.027s
W20260709 13:02:18.622822  2231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.023s	sys 0.027s
W20260709 13:02:18.553879  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.554204  1538 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:18.569818  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:18.570756  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 507340us
I20260709 13:02:18.572170  1692 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
W20260709 13:02:18.602662  2201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.012s	sys 0.031s
W20260709 13:02:18.625475  2201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.013s	sys 0.031s
W20260709 13:02:18.625855  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 564945us
I20260709 13:02:18.495325  7510 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.546700  7392 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:18.471081  7605 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 13:02:18.507664  7556 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:18.627409  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:18.514645  7574 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:18.530833  7628 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:18.619818  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 284442us
I20260709 13:02:18.663303  1508 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.641987  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:18.663625  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 280729us
I20260709 13:02:18.644894  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:18.664413  1692 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:18.620251  7564 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:18.662273  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:18.665386  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:18.646509  7647 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:02:18.667367  1508 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:18.669833  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 327690us
I20260709 13:02:18.471546  7607 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.141s
I20260709 13:02:18.642094  7426 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:18.669847  1685 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
W20260709 13:02:18.671958  1610 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.017s	sys 0.026s
W20260709 13:02:18.680799  1610 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.017s	sys 0.027s
I20260709 13:02:18.675035  1508 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.669011  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:18.678483  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:18.682925  1507 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:18.682454  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2175) took 1100 ms. Trace:
I20260709 13:02:18.689828  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:18.692231  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:18.699263  1507 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:18.700418  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:18.545130  7178 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.704819  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:18.605301  7496 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.653908  7554 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:02:18.641733  7508 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.712510  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:18.714053  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1367) took 1227 ms. Trace:
W20260709 13:02:18.738382  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 63443.7us
I20260709 13:02:18.714407  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:18.741863  2233 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:18.743125  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:18.744324  2237 raft_consensus.cc:2410] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 18 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 13:02:18.715773  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:18.689599  7536 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:18.724107  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:18.738199  2234 rpcz_store.cc:276] 0709 13:02:17.479577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.479710 (+   133us) service_pool.cc:224] Handling call
0709 13:02:17.480082 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.585125 (+105043us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.605487 (+ 20362us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.616364 (+ 10877us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.328435 (+712071us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:18.340172 (+ 11737us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.340360 (+   188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.340434 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.706671 (+366237us) raft_consensus.cc:1685] finished
0709 13:02:18.706692 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.706834 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16143,"prepare.queue_time_us":11677,"raft.queue_time_us":58086,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":22614,"spinlock_wait_cycles":13705984,"thread_start_us":752941,"threads_started":2,"wal-append.queue_time_us":908994}
I20260709 13:02:18.746694  7627 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.085s
I20260709 13:02:18.747527  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:18.747884  7649 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:18.748833  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:18.749325  7649 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:18.749743  2231 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:18.688484  7576 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:18.749931  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:18.751012  2232 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:18.750787  7649 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:18.751219  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:18.708122  7643 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:18.707405  7633 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:18.753460  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.715106  7634 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:18.663431  7197 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.710227  7628 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:18.759151  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.762749  1614 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:18.711325  7230 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:18.771664  7651 log.cc:927] Time spent T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
W20260709 13:02:18.772051  7655 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:02:18.772445  7426 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:18.773131  7599 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.110s
I20260709 13:02:18.751880  7574 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:18.721884  7582 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:18.795657  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:18.722721  7154 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:02:18.796357  1691 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:18.798557  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:18.799278  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:18.803933  1614 raft_consensus.cc:2410] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 14 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
I20260709 13:02:18.794044  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:18.785037  7653 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:02:18.753348  7214 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:18.786059  7326 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:18.786131  7610 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:18.785069  7643 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 21 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:18.815110  7633 raft_consensus.cc:2764] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 13:02:18.816320  7654 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 13:02:18.817543  7646 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 13:02:18.818222  7664 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:02:18.801549  7197 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:18.849072  7648 log.cc:927] Time spent T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.349s	user 0.001s	sys 0.000s
I20260709 13:02:18.819720  7657 raft_consensus.cc:3055] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:18.854192  7657 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:18.820564  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:18.823813  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1046) took 1312 ms. Trace:
I20260709 13:02:18.855765  1617 rpcz_store.cc:276] 0709 13:02:17.511651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.511767 (+   116us) service_pool.cc:224] Handling call
0709 13:02:17.524057 (+ 12290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.377641 (+853584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.823497 (+445856us) spinlock_profiling.cc:243] Waited 853 ms on lock 0x7b54002cbd54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:18.823744 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1960771840}
I20260709 13:02:18.814495  1691 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:18.793982  7230 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.821113  7616 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:18.800999  7607 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:18.819693  1685 pending_rounds.cc:85] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 6
I20260709 13:02:18.863241  1685 pending_rounds.cc:107] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:02:18.863646  1685 raft_consensus.cc:2889] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:18.852463  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:18.859495  1612 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:02:18.861338  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 121955us
I20260709 13:02:18.871555  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.847275  7392 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:18.809509  7645 consensus_queue.cc:1059] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:02:18.883404  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:18.879230  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 133603us
I20260709 13:02:18.808244  7426 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.861414  7230 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:18.851537  7575 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:18.858462  7582 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:18.861142  7641 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:18.886648  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.886741  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.883659  1583 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:18.854337  7214 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:18.889921  1616 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:18.874022  7607 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:18.896246  7641 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:18.904917  7678 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:18.901597  1508 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:18.922677  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:18.923089  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2994) took 1286 ms. Trace:
I20260709 13:02:18.923355  1614 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:18.901396  7154 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:18.881261  7633 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:18.888408  7616 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:18.926679  7663 log.cc:927] Time spent T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:02:18.894194  7645 consensus_queue.cc:1243] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:18.854549  7666 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 13:02:18.923470  1688 rpcz_store.cc:276] 0709 13:02:17.636623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.636773 (+   150us) service_pool.cc:224] Handling call
0709 13:02:17.645127 (+  8354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:17.698120 (+ 52993us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.698154 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:17.802871 (+104717us) log.cc:844] Serialized 52 byte log entry
0709 13:02:18.097631 (+294760us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:18.097670 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.097733 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.922862 (+825129us) raft_consensus.cc:1685] finished
0709 13:02:18.922881 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.923029 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":228201,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":156,"thread_start_us":219548,"threads_started":2,"wal-append.queue_time_us":538774}
I20260709 13:02:18.879572  7631 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:18.921383  7618 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.824s	user 0.013s	sys 0.004s
I20260709 13:02:18.896122  7556 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:18.890882  7613 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 13:02:18.889827  7508 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:18.910117  7582 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.975463  7582 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:18.965284  7664 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:18.976342  7664 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:18.976711  7664 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:18.977556  7634 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:18.980476  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2987) took 1474 ms. Trace:
I20260709 13:02:18.980921  1684 rpcz_store.cc:276] 0709 13:02:17.506242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.506363 (+   121us) service_pool.cc:224] Handling call
0709 13:02:17.523501 (+ 17138us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:17.532249 (+  8748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.532281 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:17.537990 (+  5709us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.080636 (+542646us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:18.098108 (+ 17472us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.098410 (+   302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.098505 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:18.848140 (+749635us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:18.980153 (+132013us) raft_consensus.cc:1685] finished
0709 13:02:18.980258 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.980406 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8388,"prepare.queue_time_us":15043,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":2602,"raft.queue_time_us":161427,"raft.run_cpu_time_us":313,"raft.run_wall_time_us":15661,"spinlock_wait_cycles":10300160,"thread_start_us":488075,"threads_started":1,"wal-append.queue_time_us":779254}
I20260709 13:02:18.968454  7618 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:18.910674  7575 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:18.913723  7607 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:18.924127  7544 consensus_meta.cc:287] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.762s	user 0.012s	sys 0.000s
I20260709 13:02:18.954447  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:19.007061  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 52768us
W20260709 13:02:18.967504  7673 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:02:18.967674  1688 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:18.962106  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 60593.3us
W20260709 13:02:18.970098  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:18.986722  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 2166) took 1473 ms. Trace:
I20260709 13:02:18.987351  1689 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:02:19.019371  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:18.895320  7669 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 13:02:18.989650  7513 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:18.991336  7426 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:18.893105  7657 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:18.991997  7392 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:18.992890  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2164) took 1481 ms. Trace:
I20260709 13:02:19.041002  7657 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.040796  1615 rpcz_store.cc:276] 0709 13:02:17.511961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.512656 (+   695us) service_pool.cc:224] Handling call
0709 13:02:17.527646 (+ 14990us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.144180 (+616534us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.992628 (+848448us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b5400105754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:18.992826 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1416345472}
W20260709 13:02:18.954084  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.042502  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:19.011157  1614 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:19.016078  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:18.690558  1610 rpcz_store.cc:276] 0709 13:02:17.582506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.592695 (+ 10189us) service_pool.cc:224] Handling call
0709 13:02:17.611432 (+ 18737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:17.611686 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:18.085604 (+473918us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.681964 (+596360us) log.cc:844] Serialized 14 byte log entry
0709 13:02:18.682110 (+   146us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:18.682151 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:18.682198 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.682262 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:18.682392 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":528,"prepare.queue_time_us":768862,"prepare.run_cpu_time_us":7765,"prepare.run_wall_time_us":80901,"thread_start_us":1056073,"threads_started":2}
I20260709 13:02:19.044771  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:19.018689  1613 rpcz_store.cc:276] 0709 13:02:17.513300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.513409 (+   109us) service_pool.cc:224] Handling call
0709 13:02:18.986517 (+1473108us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b5400322654. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:18.986635 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1537515904}
I20260709 13:02:19.046417  7431 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:19.047075  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:19.016325  7614 consensus_meta.cc:287] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.820s	user 0.013s	sys 0.000s
I20260709 13:02:19.026150  7580 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:18.950125  7643 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.026592  7522 consensus_queue.cc:1059] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:19.029274  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.060428  1539 raft_consensus.cc:2410] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:02:19.032370  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.049796  7637 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.034401  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.037449  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:19.048141  7616 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:19.037813  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.039011  1534 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 13:02:19.065387  1534 pending_rounds.cc:85] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 9
I20260709 13:02:19.065969  1534 pending_rounds.cc:107] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:02:19.066332  1534 raft_consensus.cc:2889] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:19.039673  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:18.964701  7633 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.069875  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.070649  1536 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:19.057017  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:19.071188  7633 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.049901  7645 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:19.075790  7522 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.076452  7522 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.077282  7625 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:19.061357  1542 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:18.950690  1617 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:19.034039  1543 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:19.022536  7513 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:19.084785  7522 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.085554  7232 raft_consensus.cc:3055] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:19.090189  7232 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:19.090389  7513 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.091173  7232 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.091806  7513 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:19.092442  7232 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.057163  7574 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:19.061637  7392 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:19.093072  7574 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.093891  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.095395  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.085847  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.096462  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:19.096732  1543 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:19.087091  1541 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:02:19.097980  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.078299  7625 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:19.021389  7544 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.079066  7618 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:19.102247  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.099269  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:19.100055  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.100106  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:19.104548  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.105957  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:19.105213  1540 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:19.108979  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.107470  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:19.110838  1540 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:19.111215  1611 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:19.098449  7182 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.115531  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.100869  7574 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.102140  7522 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:02:19.119743  7182 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.121757  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:19.093858  7392 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.122020  7360 raft_consensus.cc:3055] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:19.122681  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:19.138309  1542 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:19.123270  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.134644  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:19.144475  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.147029  1540 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:19.138533  7574 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:19.144273  7360 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:19.148979  7574 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.149478  7360 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.151639  7618 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:19.154786  7633 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:19.164533  7633 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:19.165827  7633 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.150905  7360 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.167440  7544 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.170655  7625 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.175496  7633 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:19.183807  7392 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.188951  7664 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.191774  7664 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.156206  7621 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:19.191185  7607 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.192813  1692 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:19.201699  7574 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:19.200695  7607 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.203656  7182 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.205977  1684 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:02:19.207180  1684 raft_consensus.cc:606] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:02:19.207587  1684 raft_consensus.cc:614] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:19.162230  7230 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:02:19.213392  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 64962.7us
I20260709 13:02:19.216429  1535 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:19.223220  1584 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:19.224164  1542 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:19.225279  2228 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:19.227044  2228 raft_consensus.cc:606] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:02:19.226797  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:19.227470  2228 raft_consensus.cc:614] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:19.228176  7392 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:19.226351  1692 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:19.234293  1684 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:49352
I20260709 13:02:19.235004  1684 raft_consensus.cc:606] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:19.235416  1684 raft_consensus.cc:614] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:02:19.236436  7642 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.829s	user 0.006s	sys 0.000s
I20260709 13:02:19.242038  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 2992) took 1729 ms. Trace:
I20260709 13:02:19.242609  7230 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:19.214573  7575 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 13:02:19.246053  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.247618  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.251101  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.251803  1690 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:19.251938   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:02:19.262038  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:19.242440  1683 rpcz_store.cc:276] 0709 13:02:17.513027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.513190 (+   163us) service_pool.cc:224] Handling call
0709 13:02:17.514046 (+   856us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:17.896505 (+382459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:17.896531 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:17.897085 (+   554us) log.cc:844] Serialized 52 byte log entry
0709 13:02:18.052671 (+155586us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:18.052710 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.984341 (+931631us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b540003fa54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:18.984385 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.223253 (+238868us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:19.241801 (+ 18548us) raft_consensus.cc:1685] finished
0709 13:02:19.241826 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.241971 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5104,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"spinlock_wait_cycles":911926912,"thread_start_us":155209,"threads_started":1,"wal-append.queue_time_us":431976}
I20260709 13:02:19.262045  7230 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.267259  7230 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.223932  7574 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:02:19.271524  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.273406  1683 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 12.
I20260709 13:02:19.272780  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:19.274462  1685 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.276530  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:19.280143  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:19.270898  7637 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:19.272694  7230 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:19.276155  7575 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:19.280400  7618 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.282533  7556 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:19.280827  7571 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.283524  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.281466  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:19.289600  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.293551  7571 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.279418  7544 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:19.295120  7544 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.276741  7392 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:02:19.291584  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:19.290274  7214 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:19.298547  7214 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:19.299480  7214 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:19.297901  7637 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.301846  7637 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.292841  7618 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.304080  7637 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.304637  7637 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:19.297344  7544 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.305099  7637 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.310444  7637 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.288746  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:19.312335  1687 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:19.293174  1584 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:19.313238  1685 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:19.317970  7567 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:19.318583  7567 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:19.319360  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 407185us
I20260709 13:02:19.319588  7567 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.320454  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:19.321247  7177 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:19.289721  7575 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:19.321705  7177 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.321975  7575 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:19.322154  7177 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:19.322556  7680 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.296s	user 0.001s	sys 0.000s
I20260709 13:02:19.325076  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:19.327499  1507 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:19.330229  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.330034  1582 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:19.333469  7618 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.349996  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.353657  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.329660  7230 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:19.329859  7360 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:19.337246  7392 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.361840  1692 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 13:02:19.366573  1689 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:02:19.363396  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:19.358734  1584 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:19.364910  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:19.370604  1683 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:19.371136  1684 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:19.372116  1687 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:19.352118  7564 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:19.341374  7658 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:19.352311  7657 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:19.331058  7649 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:02:19.380545  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.378451  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:19.341149  7664 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:02:19.378746  7564 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:19.379586  7326 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:19.355283  7178 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:02:19.408226  7618 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:19.408967  7618 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:19.409355  7618 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:19.410168  7692 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:02:19.378230  7232 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:02:19.414319  7232 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:19.414791  7232 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:19.415969  7232 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.388201  7658 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.405812  7564 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:19.385915  7657 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:19.414908  7661 log.cc:927] Time spent T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.695s	user 0.004s	sys 0.000s
I20260709 13:02:19.419795  1692 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: [17.9-19.10]
I20260709 13:02:19.421824   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:19.423251  7658 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:19.426556  7178 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:19.433369  7178 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:19.434314  7178 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.435700  7178 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:19.417364  7232 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.438752  7580 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:19.439400  7580 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:19.443913  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.445778  7623 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:19.446382  7623 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:19.449200  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.377974  7616 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:02:19.461962  1657 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:19.465696  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:19.466398  2237 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:19.471341  2231 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:19.429823  7510 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:19.472599  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:19.429399  7657 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:19.477277  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 98875.7us
I20260709 13:02:19.472945  1683 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:19.470011  7633 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:19.448655  7623 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:19.486095  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 127420us
I20260709 13:02:19.463452  7617 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:02:19.484637  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:19.491722  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:19.492700  2237 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 12.
I20260709 13:02:19.493263  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:19.493587  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.494393  2231 raft_consensus.cc:2368] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:19.494457  1582 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:02:19.496592  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.498291  2232 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 18: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:19.499507  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:19.486441  7580 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.509014  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:19.507131  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:19.500592  7214 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:19.514381  2237 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:19.517482  7544 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:19.518147  7544 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:19.519119  7544 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.521626  7571 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:19.523943  1584 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:19.527953  7571 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:19.529032  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:19.532757  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:19.533679  7624 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:19.534036  1611 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:19.534269  7624 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:19.535548  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:19.535329  7624 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.536036  1611 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:02:19.537901  1611 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 13:02:19.530295  6855 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:19.538682  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.542454  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:19.539892  1657 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:19.544188  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.546764  1657 leader_election.cc:400] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:19.547202  1657 leader_election.cc:403] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:02:19.545521  1537 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:19.548383  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.552287  7580 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:19.554581  7195 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:19.555030  7195 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:19.555801  7195 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.556134  7575 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:19.556993  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.516060  7510 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:19.559463  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:19.537782  7625 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:19.553344  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:19.538709  7214 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:19.577589  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:19.578811  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.586024  1616 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:19.580222  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.557052  7195 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:02:19.583602  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.582229  1612 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:19.540788  6855 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:19.600090  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.600976  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.606652  1617 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:02:19.625021  1691 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:02:19.625888  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.622020  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.511797  7641 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.576833  7178 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.560642  7637 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:19.629827  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:19.630251  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:19.602010  1536 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:19.630684  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.639070  1692 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:19.624449  1537 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:19.639918  1537 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:19.541674  7636 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:02:19.582587  7657 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.574553  7177 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.644218  1612 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:02:19.635892  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.646605  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.587996  7205 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:19.648481  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:19.643440  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:19.635504  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:19.649737  1614 raft_consensus.cc:2410] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:02:19.650683  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:19.652649  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:19.620883  7576 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:19.597371  7627 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:19.650291  7641 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:19.706478  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:19.605321  7600 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:19.633932  6976 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:19.582971  7510 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.675845  1540 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:19.651381  7657 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:19.681001  7667 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.771s	user 0.005s	sys 0.000s
I20260709 13:02:19.680583  7618 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:19.682554  7694 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 13:02:19.670634  7205 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.725615  7623 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:19.687063  7696 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:19.642282  7178 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.672122  6855 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.693763  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:19.699007  1617 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:19.703317  1618 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:19.637346  7637 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:19.613857  7564 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.702551  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:19.588482  7625 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.718413  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:19.719807  1692 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
W20260709 13:02:19.741262  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.718959  7439 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:19.760268  1610 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
W20260709 13:02:19.760694  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 155880us
I20260709 13:02:19.731097  7637 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:19.734746  7605 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:19.800535  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:19.833056  1610 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:19.804597  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:19.807732  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:19.808857  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:19.834368  1692 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:19.811676  1537 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.835717  1537 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:19.814276  1689 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:02:19.807073  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.826390  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.840119  1540 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:19.831138  1688 raft_consensus.cc:3055] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:19.841030  1688 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:19.849067  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 50038.5us
W20260709 13:02:19.851037  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 220854us
I20260709 13:02:19.851761  2201 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:19.852483  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:19.713445  7576 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.725142  7600 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:19.861294  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:19.863274  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:19.863947  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.717903  6976 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.727736  7627 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:19.868000  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 57184.9us
I20260709 13:02:19.799335  7614 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:19.870391  1609 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:19.877868  1609 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:19.798437  6855 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:19.734946  7623 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.125s
I20260709 13:02:19.874286  1688 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.884104  1612 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:19.886099  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3002) took 1672 ms. Trace:
W20260709 13:02:19.849802  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.882201  1610 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:02:19.884130  1688 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:19.886613  1686 rpcz_store.cc:276] 0709 13:02:18.214107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:18.214575 (+   468us) service_pool.cc:224] Handling call
0709 13:02:18.214967 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:18.325300 (+110333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:18.325331 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:18.330203 (+  4872us) log.cc:844] Serialized 26 byte log entry
0709 13:02:18.673593 (+343390us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:18.674039 (+   446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:18.674111 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:19.680389 (+1006278us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:19.876781 (+196392us) raft_consensus.cc:1685] finished
0709 13:02:19.876802 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:19.886014 (+  9212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94347,"raft.queue_time_us":110021,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":8952,"spinlock_wait_cycles":3734400,"thread_start_us":290623,"threads_started":1,"wal-append.queue_time_us":516104}
I20260709 13:02:19.878566  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.700929  7214 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:19.899309  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.900722  1686 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:19.890563  1540 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:19.866055  7605 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:19.859926  7513 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
W20260709 13:02:19.903771  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:19.904268  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 201893us
I20260709 13:02:19.905861  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:19.880153  7698 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:02:19.908751  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.927059  1542 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
W20260709 13:02:19.910768  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 58383.1us
I20260709 13:02:19.928164  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:19.929916  1584 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:19.930280  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:19.932792  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:19.937475  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.913827  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:19.914691  7600 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:19.915587  7178 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:19.920190  7633 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:19.759697  7439 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:19.973661  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 59925.7us
W20260709 13:02:19.977355  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:19.931104  7706 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:19.957607  1617 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 13:02:19.959687  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 107988us
I20260709 13:02:19.938540  7182 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:19.937551  7703 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:20.029206  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:20.037355  1542 raft_consensus.cc:2368] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
W20260709 13:02:20.030910  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 55362.3us
I20260709 13:02:20.038462  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:02:20.031061  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 56102.3us
I20260709 13:02:20.008277  7706 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.008399  7178 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:20.046114  1509 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:20.049494  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:19.941272  7671 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:02:20.050962  1534 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:19.997553  7600 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 13:02:20.024873  7182 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.028589  7205 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.024284  7214 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 13:02:20.050339  7706 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:19.859341  7643 consensus_queue.cc:1175] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "984c9089d9804652846f3caeb609d56e" responder_term: 18 status { last_received { term: 15 index: 9 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: [17.10-17.10]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:20.029197  7717 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:20.053203  7664 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:20.075663  7664 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:20.076139  7664 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:20.085256  7664 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.087771  7664 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:20.089884  7699 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 13:02:20.041320  7195 consensus_queue.cc:1059] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:02:20.056238  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:20.114396  1542 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:20.098191  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.105711  1534 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:20.064458  7564 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:20.113807  7195 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:20.138883  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:20.065316  7618 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:20.064196  7625 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:20.140303  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:20.141083  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.064648  7657 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:20.143005  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 119287us
I20260709 13:02:20.057686  7510 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:20.058485  7360 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:20.068601  7182 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:20.064397  7439 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.138226  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:20.141189  1542 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:20.121018  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:20.042346  7605 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.138797  7623 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:20.153539  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 107532us
I20260709 13:02:20.155588  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:20.138887  7195 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:20.157480  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:20.151038  7701 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:02:20.166163  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:20.167794  1534 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:02:20.176931  7392 consensus_meta.cc:287] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.803s	user 0.009s	sys 0.000s
I20260709 13:02:20.159492  2201 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:20.159806  1542 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:20.182968  7735 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.183748  7735 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.184592  1542 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:20.180596  7439 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:20.187914  1542 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:20.190212  7711 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
W20260709 13:02:20.161862  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 289908us
I20260709 13:02:20.191839  7657 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:20.193130  1539 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:02:20.197156  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:20.198971  1539 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 13:02:20.199357  1539 pending_rounds.cc:85] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 6
I20260709 13:02:20.199575  1539 pending_rounds.cc:107] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:02:20.199523  7741 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:20.200014  1539 raft_consensus.cc:2889] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:20.200119  7741 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:20.200549  7741 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:20.155597  7576 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.206278  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.207618  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.149403  6976 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:20.194856  7214 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.207794  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.209317  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.207398  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.210471  2234 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:20.211628  1688 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 13:02:20.212186  1688 pending_rounds.cc:85] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 8
I20260709 13:02:20.212466  1688 pending_rounds.cc:107] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:02:20.212780  1688 raft_consensus.cc:2889] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:20.213562  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2132) took 1510 ms. Trace:
I20260709 13:02:20.213091  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:20.214134  1688 rpcz_store.cc:276] 0709 13:02:18.703612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.022068 (+318456us) service_pool.cc:224] Handling call
0709 13:02:19.027816 (+  5748us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.831022 (+803206us) spinlock_profiling.cc:243] Waited 36.4 ms on lock 0x7b54001011a0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa63dc0 00007fe08faee1c0 00007fe08faef231 00007fe08faebd8f 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c
0709 13:02:20.213266 (+382244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.213510 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83770112}
I20260709 13:02:20.215209  1537 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:02:20.216280  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 58902.6us
I20260709 13:02:20.219035  1610 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 13:02:20.210673  2232 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:20.222010  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:20.192982  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:20.228840  1612 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:20.229149  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:20.208760  7392 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:20.206615  7627 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.242372  2232 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:20.242475  2231 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 13:02:20.223371  7610 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
W20260709 13:02:20.245401  1584 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.193:43459 timed out after 1.760s (SENT) [suppressed 17 similar messages]
W20260709 13:02:20.247190  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.263285  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:20.238427  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:20.268694  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.224283  7439 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:20.264845  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:20.279677  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.279984  2234 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:20.274652  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:20.277994  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:02:20.243510  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50608us
W20260709 13:02:20.226565  7665 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 13:02:20.240407  7575 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.288782  2231 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:02:20.268903  2233 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:20.279721  1611 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:20.281597  1539 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:20.291173  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:20.295179  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:20.288779  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.283844  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:20.295193  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.283839  2232 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:02:20.297919  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:20.220278  7214 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.298646  2232 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:20.250694  7739 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:20.300365  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:20.264293  7178 raft_consensus.cc:697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:20.310753  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.311884  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:20.314946  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.263546  7649 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:02:20.268086  7360 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
W20260709 13:02:20.320869  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.324769  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:20.330229  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:20.331772  7641 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.619s	user 0.012s	sys 0.000s
I20260709 13:02:20.308622  7734 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:20.307914  7392 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:20.373298  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:20.374408  2236 raft_consensus.cc:2410] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:02:20.321873  7564 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:02:20.335428  7557 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:20.382200  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.338737  7734 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.323302  7326 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:02:20.370867  7735 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:20.395853  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:20.397663  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 97526.4us
I20260709 13:02:20.335880  7742 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 13:02:20.380297  7392 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:20.387503  7557 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:20.412370  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:20.392501  7734 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:20.413444  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:20.417582  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:20.418454  1609 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:20.420632  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.393404  7574 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:20.370735  7744 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:02:20.423764  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.424134  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:20.424926  1612 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:20.431190  7742 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:20.433563  7574 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.434353  7214 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.435913  7574 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.436645  7741 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.438131  7580 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:20.438699  7580 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:20.439616  7580 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.441005  7580 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:20.392330  7182 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.442063  7754 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:20.442502  7754 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:20.443382  7754 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.444370  7182 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.444712  7754 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:20.395720  7178 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.451363  7392 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.415377  7557 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:20.453615  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:20.455222  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:20.455976  1540 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:20.456445  7353 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:20.420881  6855 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.002s
I20260709 13:02:20.457818  2232 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:20.451779  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:20.458757  1536 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:20.459061  2234 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
W20260709 13:02:20.459685  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 338775us
I20260709 13:02:20.460402  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:20.456990  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:20.465405  1543 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:20.465981  1540 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:20.404245  7513 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:20.475800  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.482728  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.484777  1658 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.483743  2236 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:20.486076  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.486738  2228 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:20.489568  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:20.485935  7637 consensus_meta.cc:287] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.623s	user 0.003s	sys 0.011s
I20260709 13:02:20.483671  7636 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:20.495956  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:20.497376  2230 raft_consensus.cc:3055] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:20.499111  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:20.497761  2203 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:20.484345  7633 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:20.501366  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.474655  7740 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:20.502033  2233 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:20.498572  2230 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:20.500550  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 217888us
I20260709 13:02:20.496567  7513 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:20.498592  7706 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.504446  7513 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.506595  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:20.493454  7623 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:20.510841  7645 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.511557  7177 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.511477  7645 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.515291  1615 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:20.517910  7544 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.519822  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:20.520426  2233 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:20.517802  2230 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.535511  1508 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.537094  1508 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.537662  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.538611  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:20.522380  1540 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:20.539295  1609 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:20.542629  2230 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:20.542688  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:20.517867  7636 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:20.519191  7544 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.510113  7623 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.547513  7623 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:20.548763  7353 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:20.509145  7633 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:20.549947  1583 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.551223  7353 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:20.551492  7633 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:20.555619  1540 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 13:02:20.556111  1540 pending_rounds.cc:85] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 8
I20260709 13:02:20.556401  1540 pending_rounds.cc:107] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 13:02:20.556663  1540 raft_consensus.cc:2889] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:20.557987  7600 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:20.558305  1615 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:20.558454  7600 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:20.558854  7600 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:20.560911  7700 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.629s	user 0.001s	sys 0.003s
I20260709 13:02:20.562093  7706 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.562433  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3137) took 1020 ms. Trace:
I20260709 13:02:20.562525  7645 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.538403  7605 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.563067  7614 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:20.563659  7741 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:20.563511  7605 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.564821  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.566174  7664 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:20.562834  1538 rpcz_store.cc:276] 0709 13:02:19.542228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.542365 (+   137us) service_pool.cc:224] Handling call
0709 13:02:19.542665 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:19.543008 (+   343us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:19.543049 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:19.543466 (+   417us) log.cc:844] Serialized 26 byte log entry
0709 13:02:19.727247 (+183781us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:19.727289 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.727359 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:20.560383 (+833024us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:20.562149 (+  1766us) raft_consensus.cc:1685] finished
0709 13:02:20.562171 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.562376 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4064,"raft.queue_time_us":845,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":6719,"spinlock_wait_cycles":6108928,"thread_start_us":183287,"threads_started":1,"wal-append.queue_time_us":337547}
I20260709 13:02:20.556038  7439 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:20.569834  1535 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:02:20.570384  7756 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:02:20.574810  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 77130.9us
I20260709 13:02:20.500182  7637 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:20.579553  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 94853.2us
I20260709 13:02:20.581545  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.581897  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.586234  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.587743  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:20.587877  1583 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.588467  1539 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:20.589998  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:20.590201  1615 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:20.592262  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:20.592840  1609 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:20.595652  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:20.596284  1683 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:20.584070  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:20.597618  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:20.598697  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:20.599355  1609 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:20.599423  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:20.600342  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:20.602381  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:20.603475  7178 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:20.603662  7761 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:20.603972  7178 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:20.605093  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.605261  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:20.606843  1687 raft_consensus.cc:2393] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:20.606846  1658 leader_election.cc:400] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:20.609098  1658 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.602371  7633 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:20.604140  7326 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:20.610201  7757 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:20.610478  7633 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:20.610725  7182 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:20.611001  7633 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:20.611217  7182 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.611760  7182 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:20.601241  7645 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:02:20.614266  7557 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.604240  7761 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:20.616714  7557 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.617331  7761 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:20.617746  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:20.617993  7617 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:20.619573  7734 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.620148  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:20.622288  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:20.623194  1690 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:20.624267  2233 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:20.619803  7633 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.649856  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:20.648608  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:20.662746  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.663775  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:20.664520  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:20.666668  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:20.667323  1538 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:02:20.651439  1689 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 17: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:20.647226  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:20.651471  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:20.595325  7716 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:20.667282  1691 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:20.648830  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:20.667894  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:20.604679  7738 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:20.604784  7636 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.668349  1683 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:20.656399  2228 raft_consensus.cc:2368] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:20.491349   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:20.679384  2203 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:20.672403  1539 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
W20260709 13:02:20.655205  7745 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.352s	user 0.002s	sys 0.000s
W20260709 13:02:20.682763  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.683976  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:20.674173  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 52782.3us
I20260709 13:02:20.604792  7618 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:20.685135  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:20.690832  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:20.669541  1542 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:20.686196  1691 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:02:20.692864  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:20.694294  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:20.693279  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:20.649237  7645 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:20.702229  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:20.657738  7605 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.706954  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.604857  7360 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 13:02:20.715623  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.646901  7178 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:20.649173  7728 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 13:02:20.683007  7755 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 13:02:20.697876  7744 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:20.730468  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:20.658109  7326 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.665632  7741 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 13:02:20.708132  7510 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:20.727691  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:20.727126  7177 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:20.740250  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:20.726493  7753 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:20.728865  7513 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:20.755934  1582 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:20.745781  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.760123  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:20.735227  7744 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.740700  7360 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:20.742436  7641 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:02:20.767809  7702 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.922s	user 0.005s	sys 0.007s
I20260709 13:02:20.768962  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 1055) took 1226 ms. Trace:
I20260709 13:02:20.795475  1611 rpcz_store.cc:276] 0709 13:02:19.542100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.542214 (+   114us) service_pool.cc:224] Handling call
0709 13:02:20.768739 (+1226525us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b540014afd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:20.768885 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1694033792}
I20260709 13:02:20.744717  7510 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.776543  7754 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.734308  7758 consensus_queue.cc:1059] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:02:20.809968  7754 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.812837  7754 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:20.780606  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:20.814065  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3138) took 1236 ms. Trace:
I20260709 13:02:20.707926  7636 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.743506  7614 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:20.742378  7326 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:20.792560  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 77059.9us
I20260709 13:02:20.754590  7177 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.754225  7617 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:20.831399  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 85692.8us
I20260709 13:02:20.798175  1582 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:20.805377  7734 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:20.806248  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 115539us
I20260709 13:02:20.795017  6855 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:20.841240  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:20.759111  7753 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.776396  7182 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.779866  7744 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:20.844956  1690 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:20.814651  1541 rpcz_store.cc:276] 0709 13:02:19.578147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.578286 (+   139us) service_pool.cc:224] Handling call
0709 13:02:19.603551 (+ 25265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:19.604040 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:19.604082 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:19.609246 (+  5164us) log.cc:844] Serialized 26 byte log entry
0709 13:02:19.753449 (+144203us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:19.753503 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:19.753587 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:20.767199 (+1013612us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:20.813659 (+ 46460us) raft_consensus.cc:1685] finished
0709 13:02:20.813691 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.813985 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17773,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3375,"spinlock_wait_cycles":24836992,"thread_start_us":138528,"threads_started":1,"wal-append.queue_time_us":236346}
W20260709 13:02:20.821923  7705 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 13:02:20.784718  7641 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:20.796236  7392 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:20.834322  2236 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:20.839159  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:20.816850  7510 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:20.801829  7757 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.831874  7177 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:20.832105  7617 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:20.841164  7614 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:02:20.887583  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:20.818882  7696 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:02:20.844272  6855 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:20.854175  7716 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:20.841060  7760 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:20.847443  7753 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:20.903795  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:20.865367  7392 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:20.831727  7728 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:02:20.871311  7637 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.962577  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:20.888480  7617 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:20.911609  7741 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.917193  7182 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:20.918177  7765 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:02:20.920707  1689 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:20.925733  7768 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:20.925930  7769 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:20.925793  7776 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:20.927780  7633 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.912534  7392 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:20.873193  7757 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.935639  7716 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:20.940408  7636 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.946971  7580 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:20.950215  7600 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.966799  7195 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:21.003774  7600 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:20.956562  1690 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:02:21.005460  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:21.007048  1690 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:20.945225  1609 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:02:20.968456  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:20.970559  1583 leader_election.cc:400] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:20.978667  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:21.015188  7701 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:20.984792  1616 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:02:21.017038  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:21.017701  7734 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.018070  7716 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.899335  6855 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:21.020565  7716 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:21.024050  7645 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:20.991722  7769 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.985822  7641 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.026700  7769 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:20.987910  7741 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.029752  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:21.030555  1689 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:21.031172  7741 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:21.031574  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:20.988193  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3026) took 1181 ms. Trace:
I20260709 13:02:21.017752  7580 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:21.033874  1692 rpcz_store.cc:276] 0709 13:02:19.806961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.862124 (+ 55163us) service_pool.cc:224] Handling call
0709 13:02:19.862400 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.862625 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:20.110724 (+248099us) log.cc:844] Serialized 14 byte log entry
0709 13:02:20.111586 (+   862us) log.cc:844] Serialized 14 byte log entry
0709 13:02:20.447134 (+335548us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b18003ec320. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c979c0a 00007fe0960b9219 00007fe0939c058f 00007fe08c9ff571 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd
0709 13:02:20.812722 (+365588us) spinlock_profiling.cc:243] Waited 1.71 ms on lock 0x7b18003ec320. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff522 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:02:20.864131 (+ 51409us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b18003ec320. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fc455 00007fe08fae4060 00007fe08faf6716 00007fe08fb07287 00007fe08fb07208
0709 13:02:20.987801 (+123670us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:20.987891 (+    90us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:20.987936 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.988027 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:20.988143 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27469,"prepare.queue_time_us":92621,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":4183,"raft.queue_time_us":12914,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":27706,"spinlock_wait_cycles":14869888,"thread_start_us":357905,"threads_started":2}
I20260709 13:02:21.018419  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:21.037761  2233 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:21.037590  7614 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.995748  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2179) took 1354 ms. Trace:
I20260709 13:02:21.038514  1618 rpcz_store.cc:276] 0709 13:02:19.641347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.669690 (+ 28343us) service_pool.cc:224] Handling call
0709 13:02:19.670092 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:19.730327 (+ 60235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:20.995393 (+1265066us) spinlock_profiling.cc:243] Waited 33 ms on lock 0x7b540008bbd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:20.995664 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75847424}
I20260709 13:02:20.985008  2230 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:21.045693  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:21.046598  2230 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:21.050438  1583 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:21.004406  7195 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:20.989332  7761 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:21.018353  7734 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.070155  7510 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.037055  7580 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:21.074463  7731 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 13:02:20.993525  7625 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:21.076948  7625 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:20.994735  7776 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.037004  7744 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.082597  7182 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:21.083150  7182 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.083567  7182 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:21.086342  7740 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:21.086779  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:21.086982  7740 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:21.087486  1618 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:21.087982  1543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 3121) took 1997 ms (client timeout 1760 ms). Trace:
W20260709 13:02:21.088403  1543 rpcz_store.cc:269] 0709 13:02:19.090320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.099050 (+  8730us) service_pool.cc:224] Handling call
0709 13:02:21.087756 (+1988706us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54000ae454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:21.087893 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2833731968}
I20260709 13:02:21.089063  7392 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:21.091279  7392 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:20.991456  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:21.092917  7731 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:21.094218  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 102867us
I20260709 13:02:21.048092  7326 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.095834  7740 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.028549  7178 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.097888  7701 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:21.098941  7731 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:21.099357  7731 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:21.100304  7731 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:20.984455  7623 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.102649  7214 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:21.103426  7214 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.118546  1683 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:02:21.120072  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.120443  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:21.121147  2230 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:21.123549  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:21.125682  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:21.126605  1609 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:21.129356  1618 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:21.104641  7625 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:21.097302  7326 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:21.083179  7744 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:21.139977  2230 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:21.144074  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:21.150653  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:21.151425  1689 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:21.158095  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:21.165019  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:21.149302  7574 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:21.147514  7658 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:21.129382  6855 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.132863  7177 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.171741  7744 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:21.190897  1687 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 1281) took 1607 ms (client timeout 1642 ms). Trace:
W20260709 13:02:21.191404  1687 rpcz_store.cc:269] 0709 13:02:19.583694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.583851 (+   157us) service_pool.cc:224] Handling call
0709 13:02:21.190625 (+1606774us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b540014a354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:21.190798 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2252901760}
I20260709 13:02:21.192879  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:21.175084  7771 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.193707  1688 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 12. Current term is 13. Ops: []
I20260709 13:02:21.193852  7771 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.196837  1687 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:21.204070  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:21.204924  1688 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:21.173288  7772 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:21.207122  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:21.212046  7753 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.219563  7769 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.214211  7214 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:21.214866  7740 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:21.215279  7610 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.219317  7771 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:21.212707  7177 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:21.240058  2228 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 17: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:21.222035  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:21.256311  1685 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:21.221361  1688 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:21.187100  7574 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:21.174050  7696 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:02:21.196341  7744 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.180375  7392 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 13:02:21.200589  7741 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:21.185509  7658 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.236465  7757 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:21.224964  7772 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:21.223353  7178 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:21.282153  7741 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.288862  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:21.289594  1688 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
W20260709 13:02:21.286720  7781 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
I20260709 13:02:21.293354  1687 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:21.280357  7744 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:21.278487  7544 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:21.228811  7769 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:21.295385  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:21.286020  7734 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:21.257378  7784 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:02:21.293536  1689 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:21.296494  1582 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:21.239355  7617 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.317467  7744 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.305001  1685 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:21.274041  7610 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:21.306442  7769 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:21.291744  7658 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:21.310745  7753 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:21.311703  7772 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.322470  7610 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:21.323182  2230 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:21.324900  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:21.314878  1691 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:21.328830  1691 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 13:02:21.329274  1691 pending_rounds.cc:85] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 10
I20260709 13:02:21.329583  1691 pending_rounds.cc:107] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 13:02:21.329870  1691 raft_consensus.cc:2889] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:21.330066  7744 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:21.330420  7623 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:21.331251  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.332613  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:21.332913  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:21.333619  1540 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
W20260709 13:02:21.334609  7786 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:02:21.335529  7627 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.336100  7741 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:21.336054  7627 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:21.338213  7627 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:21.338716  7627 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.339161  7627 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:21.341670  7643 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.342190  7643 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:21.344247  7643 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:21.338734  1684 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:02:21.318452  7617 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:21.341156  7627 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:21.361174  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:21.316378  6855 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:21.346257  1582 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:21.347817  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.365325  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:21.347710  1539 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:21.299785  1689 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:21.348613  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:21.366890  1543 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:21.349205  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:21.367714  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:21.368590  1539 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:21.368647  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:21.370324  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:21.371071  1542 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:21.300994  7544 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:21.359429  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:21.361747  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:21.378713  2228 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:21.349488  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:21.379801  1536 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:21.375231  1541 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:02:21.376019  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:21.381736  1539 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:21.385175  1542 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:21.388087  1534 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:02:21.391635  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:21.392807  2228 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:21.393837  2232 raft_consensus.cc:2368] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:21.377758  7544 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:21.407624  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.415223  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:21.420096  1543 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:21.421567  1539 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:21.422405  1541 raft_consensus.cc:2410] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:02:21.403635  7544 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:21.413228  1657 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:21.451025  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.452677  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:21.404564  7784 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
W20260709 13:02:21.498274  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 112015us
I20260709 13:02:21.402472  7734 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 13:02:21.413915  6855 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 13:02:21.495980  7658 consensus_queue.cc:1059] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:02:21.496260  7701 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:21.546402  1657 leader_election.cc:400] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:21.551184  1536 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:21.574168  1657 leader_election.cc:403] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 13:02:21.511210  7637 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:21.395470  7574 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:21.604079  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 393419us
W20260709 13:02:21.608573  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 105729us
I20260709 13:02:21.609035  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:21.614773  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:21.615170  1686 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
W20260709 13:02:21.617449  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.552107  7636 raft_consensus.cc:3055] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:21.621393  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:21.629136  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2182) took 1897 ms. Trace:
I20260709 13:02:21.557099  7731 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:21.630475  1686 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:21.630299  2201 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:21.630532  1617 rpcz_store.cc:276] 0709 13:02:19.732067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.732205 (+   138us) service_pool.cc:224] Handling call
0709 13:02:19.758491 (+ 26286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:20.011186 (+252695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:21.628782 (+1617596us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54003c87d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:21.629054 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456978304}
I20260709 13:02:21.617946  7623 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:21.638839  2201 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:21.639163  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:21.627863  7636 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:21.640806  1684 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:21.622556  7701 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:21.549692  7182 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.642649  2201 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:21.647330  2201 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:21.650673  2232 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:21.650907  2201 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:21.652304  7778 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:21.655787  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:21.660169  1616 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:21.640524  7772 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:21.650115  7178 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:21.652878  7778 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:21.658214  7600 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:21.667014  7178 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:21.682667  7178 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:21.695181  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.679443  7740 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:21.742323  7544 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.745371  7787 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:21.750128  7787 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:21.745716  7791 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 13:02:21.745769  7205 raft_consensus.cc:3055] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:21.757375  7205 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:21.758589  7205 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:21.760020  7205 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:21.753260  7787 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:21.764997  7544 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:21.770375  7757 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:21.771070  7757 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:21.772291  7757 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.712648  7772 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:21.799363  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:21.799773  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:21.822986  1538 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:21.823077  1536 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:21.679965  7701 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.680160  7636 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.829548  7636 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:21.813824  6855 consensus_queue.cc:1059] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:02:21.830649  6855 raft_consensus.cc:3055] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:02:21.831015  6855 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:21.831857  6855 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.832931  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 3045) took 1103 ms. Trace:
I20260709 13:02:21.812064  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:21.804040  7787 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:21.737592  7778 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.833330  1683 rpcz_store.cc:276] 0709 13:02:20.729996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.730131 (+   135us) service_pool.cc:224] Handling call
0709 13:02:21.832724 (+1102593us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54000aa5d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:21.832869 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850262912}
W20260709 13:02:21.847328  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.856853  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:21.843803  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:21.871301  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:21.871948  2236 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:21.819216  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:21.825414  7799 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:02:21.825314  7627 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:21.871552  1617 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:21.872182  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:21.824685  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:21.849591  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 503336us
I20260709 13:02:21.870591  1618 raft_consensus.cc:2368] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
W20260709 13:02:21.873274  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 54146.7us
I20260709 13:02:21.873359  1540 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:21.873277  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:21.818434  7625 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.668s	user 0.012s	sys 0.000s
I20260709 13:02:21.898465  1537 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:21.833233  6855 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:21.899461  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:21.836643  7776 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:21.908831  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:21.900913  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:21.909690  1613 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:21.849987  7392 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:21.917850  7618 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:21.920010  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:21.921684  7575 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:21.921625  7326 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:21.922475  7575 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:21.923557  7575 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.924072  7392 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:21.900401  1616 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:21.911461  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:21.811520  7788 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:21.838158  7182 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:21.838747  7616 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:21.888147  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:21.927718  1584 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:21.928855  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:21.875269  7610 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:21.875564  7544 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:21.939076  1617 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:02:21.829579  7637 consensus_queue.cc:1175] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cecad4f86a114a7696e398b3e8cc511e" responder_term: 18 status { last_received { term: 17 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:21.900513  7735 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:21.905438  7703 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
W20260709 13:02:21.982785  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 55187.5us
I20260709 13:02:21.983789  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:21.987248  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:21.988917  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:21.990649  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:21.991446  1616 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:21.876919  7787 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:21.985904  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:21.944802  7776 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:02:21.999609  1617 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:21.905741  7658 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 13:02:21.910027  7625 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:21.835284  7772 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.010028  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:22.027311  1613 raft_consensus.cc:2410] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:02:22.005852  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:22.011070  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:21.978855  7616 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:21.999655  7182 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:02:22.014488  7623 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:22.014981  7787 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:21.961452  7788 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:22.050832  7708 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:22.051568  7392 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:22.052191  7616 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:22.056931  7772 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:22.037294  7627 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.058250  7627 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.038560  7618 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:21.997107  7544 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.075371  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:22.076627  1617 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:22.081048  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:22.082185  1617 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:22.061180  7627 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:22.101559  1618 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:22.106043  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3154) took 1834 ms. Trace:
I20260709 13:02:22.036216  1616 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:22.002079  7641 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:22.038089  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:22.042105  7178 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.003463  7776 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:22.047755  1584 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:02:22.110533  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 62849.3us
W20260709 13:02:22.030627  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 372056us
I20260709 13:02:22.003158  7735 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:22.106650  1535 rpcz_store.cc:276] 0709 13:02:20.271791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:20.332641 (+ 60850us) service_pool.cc:224] Handling call
0709 13:02:20.371072 (+ 38431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:20.570309 (+199237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.105710 (+1535401us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b54003e2954. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:22.105945 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456177024}
I20260709 13:02:22.100977  1687 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:22.118896  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:22.121670  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:22.122537  1691 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:22.098187  7636 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:22.100126  7610 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:22.131840  7636 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:22.132535  7625 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:22.133675  7696 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:22.142153  7735 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.102807  7623 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:22.114859  7776 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:22.147709  7178 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:22.148320  7182 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:22.153844  7741 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:22.154917  7716 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:22.194417  7716 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:22.195405  7716 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.196923  7716 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:22.194410  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:22.159976  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.127758  2201 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:22.203652  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 75956.3us
I20260709 13:02:22.204541  2201 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:22.206851  7701 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:22.145550  6855 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:22.207224  7178 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:22.207499  6855 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:22.207957  7178 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:22.208607  7178 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:22.210204  6855 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:22.178537  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:22.192389  7232 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.193583  7182 raft_consensus.cc:697] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:22.188037  7741 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:22.206779  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:22.211851  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:22.213445  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:22.235687  2232 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 17: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:22.214362  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:22.236361  2231 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:22.172072  1618 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:22.236531  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.233736  7232 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.217149  6855 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:22.237696  6855 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.171787  7617 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:22.153890  7731 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 13:02:22.227872  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:22.245724  7618 raft_consensus.cc:2764] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 13:02:22.246165  2236 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 17: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:22.229871  2230 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:22.230584  1686 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:22.250787  7757 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:22.231561  2234 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:02:22.226823  7708 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:22.255355  1690 raft_consensus.cc:2393] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 13: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:22.240614  1691 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:22.235706  7557 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:22.242122  7788 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:22.241971  7617 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:22.257687  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:22.270591  1691 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:02:22.263167  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.175122  7610 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:22.218324  7701 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.182447  7214 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.074s
I20260709 13:02:22.275863  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.279000  6855 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:22.254441  7708 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:22.284557  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.275537  7701 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:22.287433  7232 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:22.288543  7708 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.289609  7636 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:22.291448  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.295159  7617 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:22.297890  7182 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.304266  7627 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.304636  7510 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:22.305987  7544 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:22.264616  7616 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:22.307214  7616 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:22.295218  1692 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.247s	user 0.028s	sys 0.028s
W20260709 13:02:22.310513  1692 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.273s	user 0.031s	sys 0.028s
I20260709 13:02:22.311457  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2160) took 1464 ms. Trace:
I20260709 13:02:22.312071  1692 rpcz_store.cc:276] 0709 13:02:20.847449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:21.036013 (+188564us) service_pool.cc:224] Handling call
0709 13:02:21.036343 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:21.036692 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:21.036728 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:21.037226 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 13:02:22.310935 (+1273709us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:22.311030 (+    95us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.311126 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:22.311190 (+    64us) raft_consensus.cc:1685] finished
0709 13:02:22.311207 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:22.311388 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27184,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"thread_start_us":1273370,"threads_started":1,"wal-append.queue_time_us":597167}
I20260709 13:02:22.185602  7623 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:22.318892  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 419483us
I20260709 13:02:22.320251  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:22.271461  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 366478us
I20260709 13:02:22.304903  7741 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.269593  7557 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.325733  1688 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:22.323540  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:22.321939  7636 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:22.332868  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:22.335819  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:22.333992  7636 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:22.336907  7636 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:22.339664  7792 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.030s	user 0.003s	sys 0.002s
I20260709 13:02:22.341333  7701 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.344046  7701 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:22.334087  7625 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:22.337646  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:22.315516  7627 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.351639  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:22.354766  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 394241us
I20260709 13:02:22.359191  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:22.360050  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.348709  7625 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:22.361393  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:22.362814  7761 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:22.364068  1537 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:22.364859  1535 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:02:22.363734  7636 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.368988  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:22.374048  1691 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:02:22.378059  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.361483  7625 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:22.335336  7741 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:22.383132  1692 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:22.341761  7784 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:22.403575  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:22.402981  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.420545  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:22.382025  7761 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:22.444689  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.381210  7616 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:22.450177  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:22.450947  1691 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:22.452137  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.455408  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.457540  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.446152  7772 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:22.462584  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:22.465361  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.465818  7815 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:22.466791  7757 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:22.466159  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.467724  1691 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:22.468760  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.474117  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.470180  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:22.471913  1535 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:22.469844  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.475492  7772 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:22.478655  7776 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:22.480444  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.483903  1538 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:02:22.484222  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:22.484422  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.486121  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:22.488044  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:22.489295  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:22.492204  1541 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:02:22.519002  7658 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:22.519603  7610 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:22.521052  7557 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:22.525504  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:22.526340  6855 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:22.526144  1541 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:22.532397  1691 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:22.492588  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:22.534124  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:22.447956  7761 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:22.565378  7761 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:22.535823  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47906.4us
W20260709 13:02:22.537639  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 45112.5us
I20260709 13:02:22.569180  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:22.569851  2231 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:22.538254  7717 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:22.539016  7178 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.540966  7784 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:22.543681  7623 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:22.524435  7617 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:22.546001  6855 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:22.588462  6855 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:22.524778  7616 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:22.558060  7812 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:02:22.559005  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:22.532811  7744 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:22.563088  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:22.564445  7636 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:22.571544  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:22.580571  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:22.597395  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 358221us
I20260709 13:02:22.560964  7776 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:22.591794  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:22.593449  6855 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:22.611471  1543 raft_consensus.cc:3055] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:22.611902  1543 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:22.612937  1536 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:02:22.612825  1543 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.585675  7784 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:22.615818  1543 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:22.617939  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:22.588155  7717 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:22.619926  1658 leader_election.cc:400] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:22.626493  6855 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:22.630316  1688 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:22.606009  7637 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:22.611286  7232 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:22.636849  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:22.646950  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.651975  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:22.653162  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:22.659544  2230 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:22.619411  7784 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:22.662314  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:22.663252  2236 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:22.664155  1535 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:22.664055  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:22.664880  2230 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:22.623626  7816 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:22.592986  7617 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:22.595644  7616 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:22.675730  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:22.677071  1616 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:22.668964  2228 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:22.669279  7557 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:22.675805  7326 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:22.671337  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:22.672719  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:22.645695  7772 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.606753  7744 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.677078  7716 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:22.684474  7575 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:22.684855  7716 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.686019  7744 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:22.681176  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:22.687062  2228 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:22.686988  7716 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:22.688040  7743 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:22.687817  7772 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.688567  7743 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.642623  7232 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:22.690241  7232 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:22.670042  7392 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.691800  1613 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 13:02:22.692082  7392 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.692535  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:22.693147  7178 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:22.693883  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:22.694296  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:22.693301  1615 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:22.696559  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:22.696810  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:22.698135  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:22.698931  2228 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:22.701519  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:22.702394  2228 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:22.704066  1611 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:22.704465  7772 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:22.714922  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:22.721241  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:22.738332  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:22.748108  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:22.750080  2236 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:22.752545  2230 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:22.714184  2234 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:22.761672  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:22.762740  1611 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 13: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:22.695956  7557 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:22.820014  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:22.824617  2201 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:22.700284  7326 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.840649  2201 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:22.840736  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:22.849395  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.836544  7831 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:22.837960  7829 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 13:02:22.825402  7735 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.015s	sys 0.025s
I20260709 13:02:22.857442  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:22.857899  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:22.858526  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:22.862409  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:22.863152  1610 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:22.865149  1508 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:22.870465  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:22.869639  1685 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:22.841611  7772 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:02:22.732501  7717 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:22.843185  7636 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:22.878330  1687 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:02:22.849826  7610 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:22.839584  7828 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:02:22.853662  7823 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:22.874498  7735 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.016s	sys 0.025s
I20260709 13:02:22.853852  7510 consensus_queue.cc:1059] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 13:02:22.910964  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:22.850163  7574 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:22.849838  7744 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:22.913688  1689 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:22.861127  7708 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 13:02:22.845999  7776 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:22.852097  7827 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:02:22.839205  7178 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 13:02:22.910390  7788 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:22.874027  7787 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 13:02:22.963634  2201 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:22.862089  7831 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:22.945829  7658 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:02:22.977536  7845 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:22.984896  1541 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:22.985562  7617 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:22.985944  7744 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:22.986966  7326 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:22.954391  7817 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:22.938906  7610 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:22.990355  7844 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:22.948228  7823 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.032193  7510 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:22.909457  7636 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.080845  7636 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:23.002606  7616 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:23.008927  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:23.082782  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:23.009873  7830 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
I20260709 13:02:23.009737  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:22.954173  7625 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.568s	user 0.010s	sys 0.001s
I20260709 13:02:23.010617  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:23.011193  7741 consensus_meta.cc:287] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.623s	user 0.011s	sys 0.000s
I20260709 13:02:23.011308  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:23.011644  7778 raft_consensus.cc:3055] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:23.014621  1582 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:23.014997  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 164929us
I20260709 13:02:23.101735  2232 raft_consensus.cc:2410] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 13:02:23.103022  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 88471.2us
W20260709 13:02:23.025542  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 62002.3us
I20260709 13:02:23.026060  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:22.976858  7826 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 13:02:23.019647  7178 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:23.019699  7627 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:22.968091  7761 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:23.002064  7833 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:02:23.086958  1613 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:23.021396  7776 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:23.116338  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:23.088905  7636 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:23.091748  7510 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:23.069072  7817 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.106204  7557 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:23.098748  7778 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:23.118301  7708 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:23.124219  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:23.108289  7178 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:23.144757  1688 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:23.212332  7178 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.214121  7178 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:23.118877  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 407975us
I20260709 13:02:23.098361  7625 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:23.211064  7868 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:23.114547  7326 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.051s
W20260709 13:02:23.220191  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 96050us
I20260709 13:02:23.129895  7853 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:23.227082  1688 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
W20260709 13:02:23.223063  7784 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.547s	user 0.016s	sys 0.000s
I20260709 13:02:23.130512  7232 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.139671  7636 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:23.261899  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 396805us
I20260709 13:02:23.237402  7326 consensus_queue.cc:1243] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:02:23.263070  1508 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:23.265236  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 132438us
I20260709 13:02:23.287948  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:23.288883  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:23.294337  1689 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:23.290007  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:23.251674  7867 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.304008  7867 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.306674  7867 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:23.307741  7867 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:23.308282  7867 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.312361  1583 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:23.314261  7850 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:02:23.251041  7784 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:23.314879  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.248772  7853 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:23.332890  7831 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:23.333149  7853 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:23.169673  7744 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.337170  7817 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:23.292596  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3072) took 1105 ms. Trace:
W20260709 13:02:23.339175  7860 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:02:23.340207  7823 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:23.341295  7856 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
W20260709 13:02:23.341940  7858 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 13:02:23.342140  7610 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:23.343386  2236 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:23.343767  7232 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:23.344751  7776 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:23.345865  7761 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:23.346589  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:23.347271  1692 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:23.264670  7778 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.321513  7625 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:23.287515  7866 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:23.325193  1691 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:23.328023  7772 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:02:23.287149  7510 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.294341  7637 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:23.355630  7637 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:23.356002  7716 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:23.329823  7784 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:23.337854  1683 rpcz_store.cc:276] 0709 13:02:22.187042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.187170 (+   128us) service_pool.cc:224] Handling call
0709 13:02:22.187517 (+   347us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:02:22.192573 (+  5056us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.192622 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:02:22.193405 (+   783us) log.cc:844] Serialized 78 byte log entry
0709 13:02:22.633319 (+439914us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:22.789090 (+155771us) log.cc:844] Serialized 14 byte log entry
0709 13:02:22.789373 (+   283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:22.789486 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.292325 (+502839us) raft_consensus.cc:1685] finished
0709 13:02:23.292357 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:23.292528 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":65329,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":2906,"raft.queue_time_us":47463,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":40057,"spinlock_wait_cycles":24183168,"thread_start_us":379557,"threads_started":1,"wal-append.queue_time_us":586992}
W20260709 13:02:23.362361  7857 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 13:02:23.363631  7734 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.363878  7743 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:23.364166  7734 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:23.364763  7851 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
W20260709 13:02:23.366122  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.367199  7734 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:23.349983  2201 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:23.368860  7772 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:23.369364  7772 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:23.369663  7822 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:23.370206  7822 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.370260  7772 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.370913  1612 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:23.371994  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:23.372558  7822 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:23.370222  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:23.374811  1613 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:23.376855  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:23.378123  1611 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:23.378598  7867 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:23.379149  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:23.380481  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:23.381229  1616 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:23.373061  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:23.382227  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:23.350932  1583 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:23.382961  7637 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:23.383754  7696 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:23.321890  2234 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:23.353700  7574 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:23.385807  7574 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:23.386013  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.386976  1613 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:23.386725  7574 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.389053  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:23.393188  7734 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:23.397676  7617 raft_consensus.cc:3055] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:23.398097  7617 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:23.398888  7617 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.400095  7617 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:23.401304  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:23.402588  7810 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:23.404162  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:23.404289  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:23.405870  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:23.407058  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.324622  7636 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:23.379264  1610 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:23.397114  2236 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:23.411157  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:23.425129  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:23.429212  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 166239us
I20260709 13:02:23.430302  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:23.430295  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:23.435353  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:23.436061  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:23.437554  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:23.437793  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.439335  1612 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:23.441291  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:23.441754  1688 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:23.444593  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:23.419206  7822 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:23.429555  7761 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:23.419486  7734 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:23.437175  7641 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:23.434724  7177 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.465862  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:23.469872  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:23.469569  7761 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:23.447278  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.463568  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:23.459816  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:23.447275  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:23.459755  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.470646  1618 raft_consensus.cc:2368] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19.
I20260709 13:02:23.472666  2236 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:23.435513  7815 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:23.447898  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:23.474167  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.447422  6855 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.476563  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:23.460062  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:23.479444  6855 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:23.419111  7753 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:02:23.480692  1683 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:23.445268  7637 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:23.493902  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:23.495262  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 64387.3us
I20260709 13:02:23.496469  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.471403  7788 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:23.498937  1692 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:23.496233  1507 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:23.500128  6855 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:23.500658  1684 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:23.501005  1683 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:23.473662  7761 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:23.478403  7636 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:23.502559  1657 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:23.503098  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.505079  1683 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 13:02:23.506425  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 76259.2us
I20260709 13:02:23.505174  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:23.507431  1508 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:23.508134  1657 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:23.509577  2230 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:23.511303  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:23.496897  7625 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:02:23.511102  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:23.519366  6855 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:23.519656  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.519842  6855 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.520244  6855 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:23.523207  1536 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 13:02:23.520831  7772 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:02:23.526928  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:23.518514  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:23.499219  7788 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.477969  7734 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.477818  7177 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.506042  7637 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:23.479883  7641 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.477836  7822 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.541343  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.552989  2236 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:23.508777  7627 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:23.545536  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:23.544080  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.548308  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:23.513446  7784 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:23.569262  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.516530  7867 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:23.539052  7625 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:23.516746  7776 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:23.523401  7845 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.524078  7787 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.530561  7717 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:23.513866  7862 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:23.556624  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2180) took 1096 ms. Trace:
W20260709 13:02:23.559635  7861 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.236s	user 0.003s	sys 0.000s
I20260709 13:02:23.514127  7616 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:23.595208  1691 rpcz_store.cc:276] 0709 13:02:22.460423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.541071 (+ 80648us) service_pool.cc:224] Handling call
0709 13:02:22.541391 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.352623 (+811232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.556359 (+203736us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b540028f5d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:23.556563 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1781574528}
I20260709 13:02:23.542618  7788 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:23.570066  1542 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:23.571193  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.602254  1543 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:23.603411  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.544116  7326 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:23.573854  2237 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:23.574673  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:23.550432  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.615391  1541 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:23.575451  1539 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:23.576747  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.578162  7703 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:02:23.550428  7864 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:23.569353  7784 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.601928  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:23.573727  7854 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:02:23.569535  7182 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:02:23.624272  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.552047  7214 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:02:23.611809  7778 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:23.622723  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.606760  7832 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:23.638486  6855 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.649377  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.643025  1542 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:23.595100  7862 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.686142  7862 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:23.645464  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.650444  1541 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:23.598589  7787 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.692242  7787 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:23.693472  7875 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:02:23.693889  7868 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:23.621608  7776 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:23.694432  7868 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:23.694795  7776 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:23.696712  7868 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:23.697731  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:23.698122  2237 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 13:02:23.698336  1540 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:23.698621  2237 pending_rounds.cc:85] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 12
I20260709 13:02:23.699044  2237 pending_rounds.cc:107] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 13:02:23.699328  2237 raft_consensus.cc:2889] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:23.699707  7832 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.646926  7864 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:23.574842  7574 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 13:02:23.601904  7717 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:23.574506  7627 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.658792  7641 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:23.672325  7177 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:23.672938  7625 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:23.676983  7867 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:23.682744  7822 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:23.685640  7734 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:23.708420  1540 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:23.709523  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:23.709800  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.711495  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.744190  1542 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:23.725376  7815 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.601114  7616 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:23.637027  7784 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:23.730059  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:23.730862  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 212417us
W20260709 13:02:23.737761  7874 log.cc:927] Time spent T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:02:23.696745  7776 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:23.720055  7787 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.765714  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.717767  6855 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:23.766551  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:23.598932  7845 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.705986  7696 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:23.746853  7618 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:23.779127  7845 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:23.780056  1536 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:23.781102  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.782168  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.756342  7616 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:23.754419  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.768493  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.785027  2237 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:23.737216  7772 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:02:23.791849  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:23.792667  2237 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:23.787472  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.788805  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.808055  2237 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:23.810629  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:23.748292  7867 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.796463  7623 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.823252  7623 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.701404   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:23.796195  7883 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:23.825898  7883 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:23.828017  7883 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:23.829244  7392 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:23.830255  2236 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:23.830528  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:23.831857  2237 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:02:23.845570  7864 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.848073  1583 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:23.849546  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:23.783223  7753 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:23.850431  7753 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:23.850862  7574 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:23.851542  7717 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.852353  7574 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:23.853627  7743 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:23.855573  7743 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:23.858697  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:23.859026  1689 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:23.850915  7753 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:23.860041  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:23.759795  7815 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.873947  1508 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:23.876488  6855 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:23.877206  6855 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:23.878263  6855 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.881996  1534 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:23.898650  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:23.911661  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 1179) took 1260 ms. Trace:
I20260709 13:02:23.771109  7787 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:23.912117  2232 rpcz_store.cc:276] 0709 13:02:22.651328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.651690 (+   362us) service_pool.cc:224] Handling call
0709 13:02:23.902799 (+1251109us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b540065bb54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:23.911603 (+  8804us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1030143744}
I20260709 13:02:23.873551  7623 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:23.915287  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 142106us
I20260709 13:02:23.791146  7618 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:23.934692  7855 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.612s	user 0.003s	sys 0.000s
I20260709 13:02:23.889304  7845 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 13:02:23.895053  7392 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 13:02:23.901211  7510 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 13:02:23.912727  7544 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:02:24.003240  2228 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1412) took 1213 ms. Trace:
I20260709 13:02:24.003778  2228 rpcz_store.cc:276] 0709 13:02:22.789827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:22.789999 (+   172us) service_pool.cc:224] Handling call
0709 13:02:22.790327 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:22.790704 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:22.790749 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:22.824219 (+ 33470us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.260646 (+436427us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:23.260690 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:23.260795 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:23.933935 (+673140us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:23.996036 (+ 62101us) raft_consensus.cc:1685] finished
0709 13:02:23.996069 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.003166 (+  7097us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9335,"raft.queue_time_us":86,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":14116,"spinlock_wait_cycles":62018304,"thread_start_us":401407,"threads_started":1,"wal-append.queue_time_us":478103}
I20260709 13:02:23.998739  7744 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:24.019348  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 178922us
W20260709 13:02:24.030296  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:24.030649  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:24.018321  7832 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:24.039868  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:24.041731  6855 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:24.062474  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 140076us
I20260709 13:02:24.067459  1508 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:24.068692  1509 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:24.069530  1684 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:02:24.073477  7891 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:02:24.075335  7618 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:24.076591  7787 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:24.089545  7845 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:24.067991  7734 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:02:24.091137  7890 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:02:24.019881  7627 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:24.096877  7867 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:24.099699  7717 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:24.100376  7717 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:24.102049  7889 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:02:24.103070  7717 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:24.106592  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:24.107899  1509 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:24.137400  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:24.084935  7788 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.093065  7862 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.149873  1509 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:24.089301  7761 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.000s
I20260709 13:02:24.187484  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:24.135989  7883 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:24.132409  7510 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.190729  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:24.184497  7845 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:24.085297  7753 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.130510  7864 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:24.256984  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:24.259087  2230 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
W20260709 13:02:24.255400  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 104048us
I20260709 13:02:24.256026  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:24.256081  1539 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: [17.16-17.16]
I20260709 13:02:24.286791  1541 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 13:02:24.258090  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 67458.3us
I20260709 13:02:24.136684  7627 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:24.146613  7618 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:24.254902  7232 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:24.298725  7182 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:24.299487  7182 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:24.260910  7510 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.302840  7510 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:24.279676  2228 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:24.303447  7753 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:24.252344  7717 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:24.260377  7883 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:24.313727  7906 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.314554  7906 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.317523  7901 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:02:24.325496  7717 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:24.326174  7717 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:24.328140  7870 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:24.328490  7717 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:24.328595  7870 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:24.329559  7870 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:24.331168  7870 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:24.321141  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:24.336354  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:24.339406  1538 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:24.342139  1538 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:02:24.344851  7788 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:24.347209  1535 pending_rounds.cc:85] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 9
I20260709 13:02:24.347671  1535 pending_rounds.cc:107] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:02:24.347985  1535 raft_consensus.cc:2889] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:24.300788  7182 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:24.278084  1657 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:24.357537  7866 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:24.358059  7866 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:24.358459  7866 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:24.359634  7905 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.360432  7905 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.295804  7618 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:24.365571  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 103346us
I20260709 13:02:24.367760  7618 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:24.369794  7862 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:24.372159  2228 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "474d3b57c33542aba459b97733122bf0"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40336
W20260709 13:02:24.372632  7784 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.608s	user 0.011s	sys 0.000s
I20260709 13:02:24.372824  2228 raft_consensus.cc:606] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:02:24.373204  2228 raft_consensus.cc:614] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:24.373054  7784 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.375584  7784 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:24.300845  7895 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.276s	user 0.000s	sys 0.000s
I20260709 13:02:24.254012  7761 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.384224  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:24.386754  2228 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:24.294364  7832 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.363350  7905 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:24.261511  7616 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:24.385776  7731 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:24.387924  7908 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.407796  7731 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:24.588081  7625 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.404844  7832 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:24.596118  7731 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:24.610348  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:24.611480  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 324373us
I20260709 13:02:24.612617  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:24.615147  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:24.646687  7618 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:24.604736  7625 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:24.675071  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 232752us
I20260709 13:02:24.683279  7753 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.686121  7616 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:24.546057  7908 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.602483  7866 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.693935  7753 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.704408  7761 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:24.707454  7832 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:24.730978  7908 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:24.739223  7866 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:24.746330  7625 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:24.674558  7905 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.747692  7905 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.750751  7905 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:24.839385  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3195) took 1108 ms. Trace:
I20260709 13:02:24.840513  1535 rpcz_store.cc:276] 0709 13:02:23.730992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.324998 (+594006us) service_pool.cc:224] Handling call
0709 13:02:24.325362 (+   364us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:24.349031 (+ 23669us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.349076 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:24.349794 (+   718us) log.cc:844] Serialized 52 byte log entry
0709 13:02:24.838104 (+488310us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:24.838281 (+   177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.838492 (+   211us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.838611 (+   119us) raft_consensus.cc:1685] finished
0709 13:02:24.838651 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.839000 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":323,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":104,"thread_start_us":487350,"threads_started":1,"wal-append.queue_time_us":331964}
W20260709 13:02:24.846451  1540 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.001s	sys 0.001s
W20260709 13:02:24.847033  1540 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.001s	sys 0.001s
I20260709 13:02:24.854451  1543 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:02:24.858201  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 242949us
I20260709 13:02:24.860147  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:24.863317  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:24.888931  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2633) took 1294 ms. Trace:
I20260709 13:02:24.889192  1536 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:24.890450  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3191) took 1177 ms. Trace:
I20260709 13:02:24.890674  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3186) took 1178 ms. Trace:
I20260709 13:02:24.890910  1541 rpcz_store.cc:276] 0709 13:02:23.712973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.137221 (+424248us) service_pool.cc:224] Handling call
0709 13:02:24.186095 (+ 48874us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.287243 (+101148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.889785 (+602542us) spinlock_profiling.cc:243] Waited 100 ms on lock 0x7b54000adf54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:24.890256 (+   471us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":231023488}
I20260709 13:02:24.888046  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:24.892553  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:24.894193  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:24.894745  1687 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:24.895392  1537 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:24.898198  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:24.900156  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:24.901530  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:24.901983  1687 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:24.902768  1537 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:24.906613  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:24.907059  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:24.908684  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:24.908815  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:24.910110  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:24.910490  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1551) took 1471 ms. Trace:
I20260709 13:02:24.910880  2231 rpcz_store.cc:276] 0709 13:02:23.439315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.439513 (+   198us) service_pool.cc:224] Handling call
0709 13:02:23.439889 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.570693 (+130804us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:23.570725 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:23.570767 (+    42us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:23.570798 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.909926 (+1339128us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b540065aed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:24.910026 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.910260 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300382592}
I20260709 13:02:24.912362  2231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:24.913021  2232 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:24.909047  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:24.914331  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:24.915105  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:24.891088  1536 rpcz_store.cc:276] 0709 13:02:23.712180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.783368 (+ 71188us) service_pool.cc:224] Handling call
0709 13:02:24.888827 (+1105459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.889703 (+   876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.890449 (+   746us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54001e4e50. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:24.890606 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2541311872}
I20260709 13:02:24.918185  7178 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:24.919317  7178 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.920811  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3190) took 1207 ms. Trace:
I20260709 13:02:24.921322  1539 rpcz_store.cc:276] 0709 13:02:23.712828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.842482 (+129654us) service_pool.cc:224] Handling call
0709 13:02:23.842896 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.286898 (+444002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.920062 (+633164us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b54001e5854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:24.920567 (+   505us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":949587712}
I20260709 13:02:24.916129  2236 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:24.906015  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:24.913298  2231 raft_consensus.cc:606] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:02:24.917128  6976 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:24.923342  7178 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:24.916139  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:24.935016  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 96801.3us
I20260709 13:02:24.937716  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1242) took 1204 ms. Trace:
I20260709 13:02:24.931823  2231 raft_consensus.cc:614] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:24.792987  7731 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.889398  1543 rpcz_store.cc:276] 0709 13:02:23.596743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.603088 (+  6345us) service_pool.cc:224] Handling call
0709 13:02:24.888562 (+1285474us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54001e4e54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:24.888815 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2872581504}
I20260709 13:02:24.924880  7326 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:24.940714  7574 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:24.960268  2230 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:24.961723  7627 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:24.962568  7731 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:24.962673  7627 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:24.963460  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 100394us
I20260709 13:02:24.963990  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:24.963817  7326 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.964660  1537 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:24.969883  7617 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:24.970695  7617 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.971725  2231 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:02:24.973369  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:24.976431  7557 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.004s	sys 0.006s
W20260709 13:02:24.990010  7557 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.004s	sys 0.006s
I20260709 13:02:24.974921  7392 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:24.991266  7743 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.992080  7743 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:24.963093  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:24.995198  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:24.996099  7743 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:24.997475  6976 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:24.997799  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:24.998198  6976 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:24.999231  2232 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:25.000815  6976 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:24.979244  1537 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:24.986289  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:24.939615  1540 rpcz_store.cc:276] 0709 13:02:23.733090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:24.326333 (+593243us) service_pool.cc:224] Handling call
0709 13:02:24.326593 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:24.326782 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:24.327041 (+   259us) log.cc:844] Serialized 14 byte log entry
0709 13:02:24.936997 (+609956us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:24.937117 (+   120us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:24.937266 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.937390 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:24.937632 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7959,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"thread_start_us":609286,"threads_started":2,"wal-append.queue_time_us":454972}
I20260709 13:02:24.969918  7510 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:24.986045  7784 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:25.014592  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 50223.1us
I20260709 13:02:25.006248  7617 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.021130  7703 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:25.021910  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.022236  7703 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:25.023156  1686 raft_consensus.cc:2410] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:02:25.033948  7930 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.035142  7930 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:25.036916  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.038573  7731 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:25.039492  7731 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.040596  2230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:25.041282  2230 raft_consensus.cc:606] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:02:25.041767  2230 raft_consensus.cc:614] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:25.041846  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.046248  2230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:25.047268  2230 raft_consensus.cc:606] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:24.985915  7866 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:25.019767  1690 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:25.059026  7914 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.059823  7914 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.064457  2230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:25.065279  2230 raft_consensus.cc:606] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:02:25.070513  2230 raft_consensus.cc:614] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:25.015852  7761 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.090306  7914 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:25.097191  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 58473.5us
I20260709 13:02:25.103897  2236 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:25.113121   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:25.113935  7870 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.114745  7870 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.117991  7867 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:25.117890  7828 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:25.118947  7867 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.119418  7627 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.120085  7627 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:25.143252  7625 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:25.144061  7625 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:25.148264  1435 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40172 (request call id 233) took 23468 ms (client timeout 15000 ms). Trace:
I20260709 13:02:25.150126  7827 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:25.150885  7827 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.152621  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.153126  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.153692  1614 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:25.155292  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.121003  7627 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.166399  2232 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:25.171362  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.090387  7866 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:25.197008  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 101255us
I20260709 13:02:25.093281  7914 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 13:02:25.210033  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.100931  7784 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.158113  7827 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.156247  7703 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:25.217176  1685 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:25.217756  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.218719  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.219070  6976 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:25.220038  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.220868  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.221846  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.222291  7832 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:25.222664  1583 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:02:25.223253  7618 consensus_meta.cc:287] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.554s	user 0.008s	sys 0.009s
I20260709 13:02:25.224499  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:25.224653  7916 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 13:02:25.211817  7870 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.233719  1690 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:25.237648  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.241772  7939 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:25.243016  7627 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.289683  1686 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:25.247210  7867 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.247684  1612 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:25.293128  1612 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:02:25.248929  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:25.299211  7933 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:25.299436  1537 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:25.252203  1687 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:25.274734  1616 raft_consensus.cc:2410] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 13:02:25.300019  7933 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.211828  7617 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:25.283568  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:25.301633  1536 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:25.301551  7617 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.302100  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:25.304035  7617 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.284660  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:25.305722  1611 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:25.285275  7618 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:25.287231  7939 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:25.288827  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.248097  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:25.307924  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 108450us
I20260709 13:02:25.315277  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.320739  1609 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:25.311787  7867 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.250892  7761 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.321184  7731 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:25.338542  7937 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:25.339186  7937 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.341742  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.344465  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:25.344689  1610 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
W20260709 13:02:25.349501  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 138682us
I20260709 13:02:25.353672  2201 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:25.355254  2201 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:25.355970  7510 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:25.356479  7510 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.356853  7510 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:25.357787  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3189) took 1645 ms. Trace:
I20260709 13:02:25.358183  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.359010  1684 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:25.361737  7939 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:25.361840  1685 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:25.363492  7326 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.345829  7827 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:25.366171  7827 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:25.366652  7827 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.343080  7867 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.353609  7934 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:25.344208  7901 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:25.369210  7867 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.358198  1534 rpcz_store.cc:276] 0709 13:02:23.712666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.811888 (+ 99222us) service_pool.cc:224] Handling call
0709 13:02:23.875204 (+ 63316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:23.882663 (+  7459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:25.357389 (+1474726us) spinlock_profiling.cc:243] Waited 63 ms on lock 0x7b54000ec4d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:25.357628 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144811264}
I20260709 13:02:25.375877  7617 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:25.345702  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.383586  7627 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.383760  7901 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.384536  7627 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:25.385228  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:25.385607  7617 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.387679  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:25.387981  7617 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.388634  7744 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:25.389062  7744 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.389461  7744 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:25.389678  7924 raft_consensus.cc:2764] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:02:25.390672  7924 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:25.391240  7924 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:25.392529  7924 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.393613  7854 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:25.393903  7761 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.394645  7948 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:25.396997  7896 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.317s	user 0.012s	sys 0.009s
W20260709 13:02:25.398830  7910 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.325s	user 0.010s	sys 0.012s
I20260709 13:02:25.400276  7933 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:25.400617  7627 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:25.400925  7945 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:25.401473  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.409889  1535 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:02:25.410856  1686 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:25.374138  7939 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:25.416268  1611 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:25.433885  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:25.447984  1507 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.664s (SENT) [suppressed 5 similar messages]
I20260709 13:02:25.439146  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.450484  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:25.451431  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.451800  7944 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:25.438105  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.452116  1583 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:25.455088  7944 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:25.439275  7939 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:25.460503  7939 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:25.461452  7906 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:25.461951  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:25.445919  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.464231  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3188) took 1751 ms. Trace:
I20260709 13:02:25.434612  7743 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:02:25.464911  2232 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:25.432385  7948 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:25.446837  2231 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:02:25.472314  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55118.5us
I20260709 13:02:25.474867  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:25.439100  7945 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:25.484112  7901 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:25.485513  7854 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.488188  7854 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:25.447055  7912 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.143s	user 0.011s	sys 0.009s
I20260709 13:02:25.490025  7815 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:25.490788  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1425) took 1582 ms. Trace:
I20260709 13:02:25.491536  7866 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.491624  7827 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.494240  7544 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:25.494825  7544 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:25.495326  7544 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:25.497550  7946 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.498164  7946 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.500005  7870 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:25.500481  7870 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:25.500846  7827 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.486009  7743 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:02:25.443436  7911 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.144s	user 0.007s	sys 0.068s
I20260709 13:02:25.426712  7937 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.534381  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:25.148844  1435 rpcz_store.cc:269] 0709 13:02:01.679194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:01.679441 (+   247us) service_pool.cc:224] Handling call
0709 13:02:03.193081 (+1513640us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:02:25.147965 (+21954884us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:14.956528 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:14.957566 (+  1038us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:14.957635 (+    69us) write_op.cc:435] Acquired schema lock
0709 13:02:14.957678 (+    43us) tablet.cc:662] Decoding operations
0709 13:02:15.096173 (+138495us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:15.096284 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:15.096316 (+    32us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:02:15.377035 (+280719us) tablet.cc:701] Row locks acquired
0709 13:02:15.377086 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:02:15.377294 (+   208us) write_op.cc:270] Start()
0709 13:02:15.377476 (+   182us) write_op.cc:276] Timestamp: P: 1783602135377213 usec, L: 0
0709 13:02:15.377519 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 13:02:15.432029 (+ 54510us) log.cc:844] Serialized 9788 byte log entry
0709 13:02:16.176198 (+744169us) op_driver.cc:464] REPLICATION: finished
0709 13:02:16.716997 (+540799us) write_op.cc:301] APPLY: starting
0709 13:02:16.717115 (+   118us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:16.750605 (+ 33490us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:16.751015 (+   410us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:17.283614 (+532599us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:17.413923 (+130309us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 13:02:17.413989 (+    66us) write_op.cc:312] APPLY: finished
0709 13:02:17.550246 (+136257us) log.cc:844] Serialized 220 byte log entry
0709 13:02:17.714941 (+164695us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:17.720117 (+  5176us) write_op.cc:454] Released schema lock
0709 13:02:17.730113 (+  9996us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:17.847676 (+117563us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:17.852638 (+  4962us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":296,"mutex_wait_us":2426616,"reported_tablets":39,"spinlock_wait_cycles":342841088,"child_traces":[["op",{"apply.queue_time_us":525325,"apply.run_cpu_time_us":67852,"apply.run_wall_time_us":1102122,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108777,"num_ops":25,"prepare.queue_time_us":361434,"prepare.run_cpu_time_us":71130,"prepare.run_wall_time_us":767797,"raft.queue_time_us":208464,"raft.run_cpu_time_us":30425,"raft.run_wall_time_us":464971,"replication_time_us":793297,"thread_start_us":1126531,"threads_started":5,"wal-append.queue_time_us":764333,"wal-append.run_cpu_time_us":48825,"wal-append.run_wall_time_us":2194413}]]}
I20260709 13:02:25.536207  1536 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:02:25.506371  7743 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:25.512187  7627 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.491166  2237 rpcz_store.cc:276] 0709 13:02:23.908071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.908256 (+   185us) service_pool.cc:224] Handling call
0709 13:02:23.913226 (+  4970us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.020503 (+107277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.020533 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.026652 (+  6119us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.057779 (+ 31127us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:24.057876 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.058010 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.446245 (+1388235us) spinlock_profiling.cc:243] Waited 39.6 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:25.490296 (+ 44051us) raft_consensus.cc:1685] finished
0709 13:02:25.490318 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.490608 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":418,"raft.queue_time_us":68838,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":10346,"spinlock_wait_cycles":91065600,"thread_start_us":135435,"threads_started":2,"wal-append.queue_time_us":232190}
I20260709 13:02:25.508505  7945 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:25.472074  1542 rpcz_store.cc:276] 0709 13:02:23.712501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.785173 (+ 72672us) service_pool.cc:224] Handling call
0709 13:02:23.785505 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:23.796772 (+ 11267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:23.796810 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:23.797401 (+   591us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.046193 (+248792us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:24.046576 (+   383us) log.cc:844] Serialized 14 byte log entry
0709 13:02:24.046796 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.046892 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.397814 (+1350922us) spinlock_profiling.cc:243] Waited 59.7 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:25.463739 (+ 65925us) raft_consensus.cc:1685] finished
0709 13:02:25.463867 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.464036 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10884,"prepare.queue_time_us":669,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":11640,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":29089,"spinlock_wait_cycles":137345920,"thread_start_us":142585,"threads_started":1,"wal-append.queue_time_us":200786}
I20260709 13:02:25.538895  7922 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:25.539255  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:25.541046  7892 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.461s	user 0.005s	sys 0.017s
I20260709 13:02:25.540125  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:25.542692  7888 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.468s	user 0.009s	sys 0.011s
W20260709 13:02:25.544135  7898 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.242s	user 0.023s	sys 0.051s
I20260709 13:02:25.508119  7392 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:25.553023  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.507746  7939 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:25.559692  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.561606  1611 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:25.562222  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:25.564314  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.565407  2230 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:02:25.565649  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 44579.7us
I20260709 13:02:25.567415  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:25.567713  2232 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:25.571349  1684 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:25.540180  7948 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:25.558516  7922 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.574209  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.534447  7510 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.564690  7392 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:25.507630  7870 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:25.580746  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:25.580910  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 133117us
W20260709 13:02:25.587214  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 51398.8us
I20260709 13:02:25.588573  1658 leader_election.cc:400] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19.
I20260709 13:02:25.588963  1658 leader_election.cc:403] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:02:25.579929  7922 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.593999  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.594867  1615 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:25.595813  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:25.598356  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:25.598094  7392 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.599931  7625 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:25.601837  7948 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19.
I20260709 13:02:25.580163  1534 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:25.603128  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:25.603690  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.607751  1611 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:25.609062  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:25.607169  1542 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:25.617830  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:25.620721  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:25.622454  7743 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.623170  6855 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:25.623608  6855 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:25.623942  7930 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.624033  7743 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:25.624221  7617 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:25.626590  7939 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:25.629087  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:25.630607  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:25.631356  1539 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:25.632570  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1191) took 2090 ms. Trace:
I20260709 13:02:25.633921  1685 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:02:25.633085  2234 rpcz_store.cc:276] 0709 13:02:23.541743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.541917 (+   174us) service_pool.cc:224] Handling call
0709 13:02:23.542344 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:23.549366 (+  7022us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:23.725546 (+176180us) log.cc:844] Serialized 14 byte log entry
0709 13:02:23.996196 (+270650us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:23.996753 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:02:23.996971 (+   218us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:23.997020 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.000123 (+  3103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:24.990449 (+990326us) spinlock_profiling.cc:243] Waited 3.98 ms on lock 0x7b48000cd758. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:02:25.540160 (+549711us) spinlock_profiling.cc:243] Waited 50.3 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:25.632117 (+ 91957us) raft_consensus.cc:1685] finished
0709 13:02:25.632148 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.632364 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75218,"prepare.queue_time_us":196087,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":3440,"raft.queue_time_us":125759,"raft.run_cpu_time_us":20087,"raft.run_wall_time_us":873619,"spinlock_wait_cycles":125794304,"thread_start_us":389079,"threads_started":2,"wal-append.queue_time_us":340996}
I20260709 13:02:25.636210  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.629120  7939 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:25.637956  7942 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:02:25.570693  7627 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:25.638525  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:25.639577  7907 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.339s	user 0.006s	sys 0.014s
I20260709 13:02:25.642186  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1325) took 1787 ms. Trace:
I20260709 13:02:25.642864  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:25.642565  1683 rpcz_store.cc:276] 0709 13:02:23.854846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.857309 (+  2463us) service_pool.cc:224] Handling call
0709 13:02:23.890203 (+ 32894us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:23.890560 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:23.890593 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:23.891051 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.149664 (+258613us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:24.149780 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.149906 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.638805 (+1488899us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:25.641707 (+  2902us) raft_consensus.cc:1685] finished
0709 13:02:25.641727 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.642019 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":875,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":15330,"spinlock_wait_cycles":99305216,"thread_start_us":201481,"threads_started":1,"wal-append.queue_time_us":385085}
I20260709 13:02:25.644557  7946 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.646504  7753 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:25.647702  7862 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:02:25.648808  7913 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.351s	user 0.014s	sys 0.006s
I20260709 13:02:25.649573  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:25.652643  7617 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:25.652855  7784 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.653105  7617 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:25.653549  7617 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:25.653929  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1426) took 1745 ms. Trace:
I20260709 13:02:25.654299  2233 rpcz_store.cc:276] 0709 13:02:23.908588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.908754 (+   166us) service_pool.cc:224] Handling call
0709 13:02:23.911913 (+  3159us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:24.061372 (+149459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:24.061454 (+    82us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:24.062524 (+  1070us) log.cc:844] Serialized 26 byte log entry
0709 13:02:24.201954 (+139430us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:24.202002 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:24.202078 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:25.647926 (+1445848us) spinlock_profiling.cc:243] Waited 45.6 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:25.653450 (+  5524us) raft_consensus.cc:1685] finished
0709 13:02:25.653472 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:25.653750 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":68650,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":5639,"spinlock_wait_cycles":104845184,"thread_start_us":287535,"threads_started":2,"wal-append.queue_time_us":233469}
I20260709 13:02:25.623961  6855 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:25.661298  2228 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:25.664362  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 76164.5us
I20260709 13:02:25.641186  7627 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:25.662078  7557 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.609620  7392 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.668622  1539 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:25.672367  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:25.574992  7744 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.675211  7744 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:25.676034  2236 raft_consensus.cc:2368] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19.
I20260709 13:02:25.678225  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:25.679597  1539 raft_consensus.cc:2410] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 13:02:25.680747  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 71515.9us
I20260709 13:02:25.678457  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.685683  1657 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:25.688270  7510 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:25.689123  7784 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:25.690675  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:25.689697  7784 raft_consensus.cc:697] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:25.694808  7784 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.696893  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:25.697804  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.697804  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:25.699514  1539 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 13.
I20260709 13:02:25.688570  7617 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.707434  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.708869  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.710178  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.712517  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.713901  7939 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:25.714668  7939 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:25.715771  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:25.717682  7557 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.718739  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.719960  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.720114  7182 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:25.721210  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.713317  2235 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:25.717096  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:25.728487  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:25.729034  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.729808  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.730497  2236 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:25.732875  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:25.732904  7510 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:25.733784  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 84385.1us
I20260709 13:02:25.734411  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:25.734817  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:25.735221  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:25.736045  1543 pending_rounds.cc:85] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 11
I20260709 13:02:25.698230  7870 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:25.736290  7894 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:25.736950  7894 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:25.738210  7894 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.743139  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.733809  7510 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.743912  1618 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:25.744248  7510 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:25.746043  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:25.746930  1610 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:02:25.747124  7772 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:25.747581  7772 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.747961  7772 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.751348  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.752107  2231 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:25.755092  1540 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 13:02:25.759626  7392 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:25.766459  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.768131  1684 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:25.769088  1615 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:25.764590  7772 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.759517  7544 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:25.769536  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:25.771306  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.782855  1534 raft_consensus.cc:2393] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 18: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:25.770036  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.781350  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:25.797904  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.798573  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.736395  1509 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:25.799288  6855 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:25.801818  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 65497.7us
I20260709 13:02:25.717195  7939 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:25.736369  1543 pending_rounds.cc:107] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:02:25.804741  1543 raft_consensus.cc:2889] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:25.804988  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:25.779960  7744 raft_consensus.cc:993] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: : Instructing follower 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 13:02:25.803184  1509 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:25.805863  1615 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:25.810842  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.811334  1692 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:25.811757  1688 raft_consensus.cc:2410] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:02:25.812775  7627 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:25.813230  7627 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:25.813606  7627 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:25.815286  6855 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.817781  1687 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:02:25.818082  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.823776  1609 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:25.731580  7557 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.835911  7772 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:25.839926  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:25.840039  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.843981  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.844851  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.845353  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:25.845798  1690 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 21: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:25.846145  1687 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:25.847082  1689 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:02:25.847518  1685 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 13:02:25.847700  7744 raft_consensus.cc:1081] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Signalling peer 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 13:02:25.848444  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:25.848627  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.848977  7744 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:25.849799  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:25.850662  1542 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:25.851336  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.852285  1691 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 17: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:25.853883  7743 consensus_queue.cc:1059] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:25.854764  7178 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:25.854827  1614 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:25.855252  7571 raft_consensus.cc:3055] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:25.855749  7575 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:25.855787  7571 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:25.854066  7772 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:25.856408  7575 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:25.857751  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:25.863814  1691 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:25.856336  7178 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:25.885151  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:25.866824   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:25.885942  1618 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.886759  1613 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:45356
I20260709 13:02:25.887465  1613 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:25.887849  1613 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:25.889910  7962 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:02:25.891532  1657 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:25.892016  7617 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.892506  1657 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:25.896255  1535 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:25.896353  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:25.897158  1540 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.877250  7658 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:02:25.877568  7232 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:25.878575  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:25.902691  1538 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:02:25.879305  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:25.835911  7922 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.881992  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.857151  7571 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.885056  6855 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:25.912264  7575 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:25.868945  7744 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:25.919602  7571 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:25.923089  1611 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:25.923014  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:25.906111  7232 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:25.939903  1685 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:02:25.947250  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 115298us
I20260709 13:02:25.952679  1534 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:25.952750  6855 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.957288  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.958370  1683 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:25.959419  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.960173  1685 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:25.906472  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:25.898284  7946 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:25.906201  7939 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:25.970431  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:25.944756  7232 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.973820  1683 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 18
W20260709 13:02:25.966482  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 60163.5us
I20260709 13:02:25.972421  1618 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:25.917338  7753 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
W20260709 13:02:25.987679  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:25.989326  6855 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:25.989889  1538 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:25.984401  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:25.976455  7939 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:26.010867  7939 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:26.013996  7948 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:25.973656  7744 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:25.970700  7946 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:26.003522  7943 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.595s	user 0.000s	sys 0.007s
I20260709 13:02:26.004724  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:26.007400  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:26.008390  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:26.024873  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:26.009255  1583 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:26.020972  7744 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:26.026724  7744 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:25.995285  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.027863  7510 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:25.976754  7926 consensus_queue.cc:1059] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:02:26.023262  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.014637  7948 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.035518  1536 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:02:26.032812  1540 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:26.023761  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:26.018116  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:26.038759  7618 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:26.039161  7618 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.039438  1534 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: [17.16-17.16]
W20260709 13:02:26.044030  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 51116.5us
I20260709 13:02:26.040906  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:26.050854  7917 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:26.051517  7917 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:26.052584  7917 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.055572  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.056001  7772 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:26.033814  7946 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:26.132030  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:26.022768  7922 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:26.108330  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.105549  7627 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:26.122601  1617 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:26.039548  7926 raft_consensus.cc:3055] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:02:26.125909  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 85108.5us
I20260709 13:02:26.126494  1688 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
W20260709 13:02:26.146549  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.148564  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:26.149226  1618 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:26.039525  7618 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:26.121107  2233 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:26.060026  1613 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.127763  2203 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:26.132608  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:26.134227  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:26.157552  1685 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:26.140547  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.161640  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:26.139578  1683 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:26.176409  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:26.141142  1686 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:26.066052  7948 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:26.180871  1613 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:26.182197  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:26.175110  7974 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:02:26.213502  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:26.213750  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:26.214120  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:26.182400  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:26.214910  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:26.218555  1539 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:26.130221  7772 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:26.131189  7969 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:26.223009  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:26.227167  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.230167  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.233963  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:26.239140  1684 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:26.240460  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:26.183825  7901 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:26.142131  7926 raft_consensus.cc:740] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:26.249588  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:26.251946  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:26.254457  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:26.255115  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 122596us
I20260709 13:02:26.255796  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:26.253049  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.231380  7510 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:26.265925  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:26.266371  6855 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:26.232471  7743 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:26.269327  6855 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.269770  6855 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:26.276304  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:26.283512  1615 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:26.273538  7922 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:02:26.296862  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.306075  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:26.271795  7862 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:26.278072  7658 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:26.316316  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:26.319367  1539 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:26.319914  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.316902  1508 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:26.323652  7922 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:26.325489  7905 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:26.325649  7972 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:02:26.325884  7916 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:26.325976  7905 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.327030  7905 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:26.329555  7616 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:26.330134  7616 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:26.330475  1584 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:26.331171  7924 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:26.331331  7939 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:26.332197  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 77826.3us
I20260709 13:02:26.332517  1684 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:26.332319  7828 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:26.333248  7828 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:26.333364  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:26.334570  7617 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:26.335234  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:26.319360  7557 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:26.336115  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.336865  1684 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 21: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:26.333689  7828 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:26.337652  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.335359  1687 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:26.340806  1690 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:26.340952  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.331645  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:26.277285  7743 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.345932  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:26.346808  1684 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:26.346517  6855 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:26.348836  1691 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:26.358021  7617 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:26.316433  7658 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:26.362443  1685 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:26.362754  1584 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:26.365885  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:26.367895  6855 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:26.371225  7901 raft_consensus.cc:697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:26.374168  1690 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:26.368571  7926 consensus_queue.cc:260] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:26.375874  7832 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:26.365684  7658 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:26.369138  6976 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:26.382896  7924 consensus_queue.cc:1059] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:26.352394  7916 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:26.369326  7939 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:26.386520  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50371us
W20260709 13:02:26.389304  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 123463us
I20260709 13:02:26.390734  2201 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:26.395315  7544 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:26.395445  7894 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:26.396281  7894 raft_consensus.cc:697] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:26.398058  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.397554  7894 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.405488  7753 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:26.409811  1539 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:02:26.413569  2201 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:26.415441  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:26.420320  7743 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:26.423209  7510 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:26.428233  7977 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:02:26.428933  7933 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:26.430426  7969 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:26.430673  7975 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 13:02:26.431818  7627 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:26.373005  7901 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.321739  7772 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.390466  7981 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:26.441044  7969 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:26.440824  7933 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:26.394299  7232 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.449703  7981 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:26.450344  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 84562.8us
I20260709 13:02:26.498593  1534 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:26.525002  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:26.450906  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 115805us
I20260709 13:02:26.528051  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:26.529433  7948 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:26.530944  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:26.531612  7744 consensus_meta.cc:287] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.000s
I20260709 13:02:26.531852  7948 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:02:26.532073  7744 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.533500  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:26.534360  7948 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:26.535143  7926 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:26.535698  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:26.535570  7969 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:26.499238  7933 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:26.452585  7922 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.453665  7985 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:26.513500  7981 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:26.514732  7232 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:26.501634  7618 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:26.560607  7618 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:26.520006  7616 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:26.522571  7979 log.cc:927] Time spent T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.201s	user 0.000s	sys 0.004s
I20260709 13:02:26.561743  7969 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:26.504321  7978 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.134s	user 0.004s	sys 0.000s
I20260709 13:02:26.571940  7828 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:26.525628  7905 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.549285  7985 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.575112  7905 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:26.512677  7658 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.575582  7985 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:26.549471  7948 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:26.576557  7948 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:26.577214  7658 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:26.577528  7948 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:26.578097  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 162758us
I20260709 13:02:26.578914  2201 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:26.580799  7985 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:26.581339  7985 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:26.582068  7772 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:26.582247  7985 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:26.450579  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 80377.2us
W20260709 13:02:26.443578  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 118674us
I20260709 13:02:26.557862  7214 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:26.558346  7744 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:26.589289  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:26.592482  1437 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55374 (request call id 322) took 80851 ms (client timeout 15000 ms). Trace:
I20260709 13:02:26.593505  7753 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:26.602970  1692 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:26.533123  7946 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:26.608821  1437 rpcz_store.cc:269] 0709 13:01:05.740985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:05.743581 (+  2596us) service_pool.cc:224] Handling call
0709 13:02:26.592132 (+80848551us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3838429,"reported_tablets":47}
I20260709 13:02:26.628767  7182 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:26.583891  1434 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:26.632970  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:26.632035  7182 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:26.634931  1508 leader_election.cc:400] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19.
I20260709 13:02:26.636411  1508 leader_election.cc:403] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:26.638283  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:26.641105  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:26.644119  1434 catalog_manager.cc:5697] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 8 to 16. New cstate: current_term: 16 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:26.644941  1688 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:26.653530  7986 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:26.654112  1583 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:26.655045  7946 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:26.655752  1583 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:26.656765  7990 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:02:26.657459  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 121846us
I20260709 13:02:26.658420  7392 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:26.658798  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.672055  1688 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:26.662357  7618 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:26.628945  7617 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.665158  2237 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:26.665719  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:26.654191  7986 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:26.667179  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:26.679761  1692 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:26.680061  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:26.642683  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:26.636389  7828 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:26.683689  7182 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.626269  7969 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 13:02:26.687206  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:26.672823  7987 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19.
I20260709 13:02:26.690832  1684 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:26.631474  7753 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.673592  7946 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:26.683311  7571 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:26.681767  7815 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:26.699574  1584 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:26.701352  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:26.705243  1615 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:26.707149  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:26.707964  1616 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:26.710062  7933 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.712248  7933 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:26.672567  7627 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:02:26.713172  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:26.713825  7627 raft_consensus.cc:3055] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:26.714223  7627 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:26.714439  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:26.715161  7627 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.716439  7627 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:26.693845  7617 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:26.709266  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:26.718997  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:26.719650  7617 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:26.720306  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:26.720759  7214 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:26.721194  7930 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:26.721787  7930 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:26.722596  7993 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:26.723315  7988 raft_consensus.cc:3055] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:26.723448  7939 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:26.723222  7993 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.723798  7988 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:26.722761  7930 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:26.709859  7986 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:26.734364  1692 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:26.731307  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:26.737372  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 94758.5us
I20260709 13:02:26.739130  1614 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:26.736788  1615 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:26.741045  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:26.743306  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:26.743930  2231 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:26.744269  2231 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:26.747100  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:26.751811  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:26.752929  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:26.753075  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:26.753245  7993 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:26.753773  1612 raft_consensus.cc:2368] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 21. Current term is 22.
I20260709 13:02:26.743335  7981 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:26.755740  2228 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:02:26.756120  7981 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:26.757124  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:26.757899  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:26.758932  1613 raft_consensus.cc:2410] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:02:26.759367  1508 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:26.761117  7969 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:26.762925  1508 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:26.757766  7772 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:26.763996  7883 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:26.764369  7772 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:26.764987  7883 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:26.765787  7883 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:26.765373  7772 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.766165  7883 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:26.766340  7753 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:26.770411  1686 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:26.772562  1611 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:02:26.769855  1509 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:26.779093  7815 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.780539  7617 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:26.782706  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:26.784788  2229 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:26.725396  1434 catalog_manager.cc:5697] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 12 to 14, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:26.786428  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:26.705078  7990 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:26.729013  7571 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:26.731726  7510 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:26.797448  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.797739  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:26.798540  1689 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:26.745072  2231 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.795574  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:26.800415  1541 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:26.801115  2231 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:26.803391  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:26.810868  1684 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:26.813897  2229 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:02:26.814911  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:26.816494  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:26.818607  7571 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.819818  7991 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:26.821741  7717 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:26.822515  7717 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:26.781138  7617 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:26.824323  7617 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:26.828841  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:26.739969  7946 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.831190  7946 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:26.832965  1434 catalog_manager.cc:5697] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 9 to 19, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 19 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:26.843093  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:26.843811  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 123583us
I20260709 13:02:26.844033  1540 raft_consensus.cc:2393] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 18: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:26.852922  1614 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:26.853892  7815 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:26.861722  1536 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:26.858238  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:26.813650  7784 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:26.854313  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:26.861269  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:26.882441  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:26.766991  7883 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.813210  7575 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:02:26.886704  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:26.864147  7906 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:26.864665  7544 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:26.894138  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:26.895187  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:26.881371  7326 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:26.849594  7905 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 13:02:26.906530  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:26.906742  1540 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:26.907972  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:26.908725  7939 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:26.908859  1658 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:26.891937  7717 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.917212  2235 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:26.862452  7926 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:02:26.890887  7832 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:26.886482  7980 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:26.933626   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:26.802757  7993 consensus_queue.cc:1175] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "984c9089d9804652846f3caeb609d56e" responder_term: 18 status { last_received { term: 17 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:26.942196  7948 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:26.872081  7981 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:02:26.918087  7988 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.900516  7544 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:26.964941  1540 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:26.919292  7906 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:26.972468  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:26.927625  7326 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:26.959486  7980 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:26.967459  7883 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:26.969146  7544 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:26.972122  7946 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:26.974790  7988 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:27.021855  7906 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.975170  1434 catalog_manager.cc:5697] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 4 to 20, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 20 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:27.037845  7326 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:27.014343  7983 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.042394  7983 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:26.975133  7832 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:27.046856  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.047479  7832 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:27.050993  2231 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:27.045100  7983 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:27.070729  1689 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:27.078411  1692 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:27.014739  7939 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:26.999069  7980 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:27.085136  1691 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:02:27.085166  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 151754us
I20260709 13:02:27.090838  2233 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:27.093488  7178 consensus_queue.cc:1059] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:27.095395  7178 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:27.095790  7178 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:27.096671  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:27.097056  7983 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:27.097584  1540 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:27.089043  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:27.096601  7178 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:27.103657  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 227345us
I20260709 13:02:27.129917  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:27.144537  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 82740.1us
W20260709 13:02:27.149012  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.150301  7232 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:02:27.151823  7232 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:27.152182  7232 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:27.029333  1434 catalog_manager.cc:5697] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 9 to 16, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 16 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:27.164763  1434 catalog_manager.cc:5697] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 14 to 15, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 15 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:27.167142  1584 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:27.166797  1434 catalog_manager.cc:5697] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 8 to 14, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:27.174048  7178 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:27.168612  1434 catalog_manager.cc:5697] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 10 to 17, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 17 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:27.175530  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:27.176281  1538 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:27.177150  1434 catalog_manager.cc:5697] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 11 to 17, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 17 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:02:27.181288  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.175607  7992 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:27.184259  7992 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:27.185366  7992 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:27.186841  7992 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:27.188489  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.204731  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 97669.7us
W20260709 13:02:27.201507  7985 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.004s	sys 0.030s
W20260709 13:02:27.205896  7985 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.004s	sys 0.030s
I20260709 13:02:27.192240  1434 catalog_manager.cc:5697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 13 to 16, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 16 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:27.210086  7870 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:02:27.211028  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 63227.7us
I20260709 13:02:27.225543  1692 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:27.226027  1509 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:27.227170  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 52490us
I20260709 13:02:27.101764  7772 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.103s
I20260709 13:02:27.240935  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:27.248543  1509 leader_election.cc:400] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 21. Current term is 22.
I20260709 13:02:27.249051  1509 leader_election.cc:403] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:02:27.213938  7392 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:27.153064  7232 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:27.259714  1689 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:27.238845  7717 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:02:27.286698  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 377865us
I20260709 13:02:27.252516  7926 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:27.310606  7926 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:27.313042  7926 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:27.313705  1434 catalog_manager.cc:5697] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:27.315994  1434 catalog_manager.cc:5697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 8 to 15. New cstate: current_term: 15 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:27.292667  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:27.294152  7784 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:27.296304  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:27.290654  7627 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:27.304642  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:27.305773  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:27.306546  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:27.307240  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:27.307986  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:27.309139  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:27.316761  7980 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:27.320583  7232 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:27.256764  7617 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:02:27.256949  7616 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:27.351641  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.353124  1615 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:27.355257  1612 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:27.357945  1538 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:27.357275  1537 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:27.358927  1543 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:02:27.369160  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:27.372638  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.376652  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:27.379124  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:27.379997  1539 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
W20260709 13:02:27.381162  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.381119  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:27.360102  7571 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.393503  1611 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 13:02:27.393944  1611 pending_rounds.cc:85] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 12
I20260709 13:02:27.394163  1611 pending_rounds.cc:107] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:02:27.394408  1611 raft_consensus.cc:2889] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:27.389570  1658 connection.cc:256] RPC call timeout handler was delayed by -2.200190436000014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
I20260709 13:02:27.400358  2237 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:27.358934  7544 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:27.408506  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:27.406121  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:27.408632  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.410398  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 154206us
I20260709 13:02:27.411422  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.417165  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.414857  7178 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:27.416191  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.418681  7178 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:27.412200  8018 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:27.418747  1616 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:27.414857  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:27.419214  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.418918  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:27.384150  7627 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:27.426447  7178 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:27.430552  1612 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:27.431622  2203 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:27.433074  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:27.441148  1427 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab5b971190d342d2b2568c30f3c0c81f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cecad4f86a114a7696e398b3e8cc511e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:27.443048  7980 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:27.437024  7883 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 13:02:27.444332  7883 raft_consensus.cc:3055] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:27.444691  7883 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:27.401801  7571 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:27.447314  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:27.451726  1539 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:02:27.453538  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.454464  1617 raft_consensus.cc:2410] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:02:27.456423  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.457876  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.459242  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.459614  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.420692  7510 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:02:27.416501  7992 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:02:27.435632  7990 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:27.461957  7990 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:27.462337  7990 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:27.459756  7544 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:27.457664  7917 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:27.456000  1536 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:27.465366  1610 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:27.465620  7182 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:27.467240  7182 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:27.469462  1612 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
W20260709 13:02:27.474081  8016 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:02:27.455148  7557 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.477082  7557 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:27.479416  7557 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:27.437820  7870 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:02:27.482244  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.484227  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.485556  1539 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:27.485853  2236 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:27.485707  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.489600  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.489734  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:27.492921  1609 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:27.501277  8025 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:27.504412  2234 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:27.504647  7182 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:27.506063  7182 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:27.506363  7326 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:27.509626  7969 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:27.511860  7627 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:27.505311  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.445602  7883 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:27.518072  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:27.496201  7617 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 13:02:27.522415  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:27.526448  1539 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:27.527204  1617 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:27.528391  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:27.536550  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.538946  1609 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
W20260709 13:02:27.539953  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 144012us
I20260709 13:02:27.498273  7717 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:27.540107  1612 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 13:02:27.540529  1658 connection.cc:256] RPC call timeout handler was delayed by -1.9724904359999869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
I20260709 13:02:27.543088  2203 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:27.544847  1686 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:27.549763  7178 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:27.548151  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:27.446581  1434 catalog_manager.cc:5697] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 8 to 15, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 15 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:27.554769  7178 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:02:27.557564  1658 connection.cc:256] RPC call timeout handler was delayed by -1.6916904359999876s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
W20260709 13:02:27.558934  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2659904359999814s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2702s for a response
I20260709 13:02:27.560920  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:27.563284  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:27.568475  1692 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:27.567214  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:27.570569  1617 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:27.575143  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:27.578320  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:27.581228  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.581552  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.553738  7571 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:27.587574  8021 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:27.587987  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:27.588838  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:27.589473  7983 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:27.567850  7906 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:27.594527  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:27.600411  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:02:27.601590  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.562942  7924 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:27.547220  7926 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 13:02:27.607388  1686 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:27.607270  7753 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:27.609643  7510 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:27.610200  7510 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:27.611091  7510 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:27.614615  7894 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:27.615018  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:27.615511  7991 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:27.615576  7894 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:27.617172  7939 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:27.617619  7939 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:27.618122  7939 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:27.562151  7717 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:27.619150  1536 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:27.617731  7980 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 13:02:27.620644  7717 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:27.585217  8017 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:02:27.622079  8017 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:27.622437  8017 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:27.609261  1540 raft_consensus.cc:2410] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 19 because replica has last-logged OpId of term: 18 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 15.
I20260709 13:02:27.623234  8017 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:27.624450  8017 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:27.562255  8025 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:02:27.640345  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:27.625216  7815 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:27.583103  7544 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:02:27.582839  7743 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:02:27.641664  1617 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:27.643996  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:27.654599  1536 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:27.655694  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:27.689291  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:27.663007  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:27.655560  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:27.757819  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 241807us
I20260709 13:02:27.765491  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:27.556479  1434 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 15 to 16, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 16 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:27.657179  7815 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:27.884996  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:27.887919  1536 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:27.890100  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 304761us
I20260709 13:02:27.893726  1617 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:27.894451  1618 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:02:27.912992  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:27.895128  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 357025us
I20260709 13:02:27.765182  7571 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.896595  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:27.771011  7980 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:27.750535  7990 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:27.898626  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:27.899596  8031 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.237s	user 0.000s	sys 0.000s
I20260709 13:02:27.907207  1609 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:27.884011  7926 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:27.935076  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 176476us
I20260709 13:02:27.763633  7939 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:27.893213  7815 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:27.990398  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 80225.2us
I20260709 13:02:27.993988  1507 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:27.886843  7894 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 13:02:27.999161  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:28.001734  1684 raft_consensus.cc:2368] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 21. Current term is 22.
I20260709 13:02:28.003705  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:28.010413  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:28.011627  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:28.015949  1507 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:28.021950  1536 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:28.000468  7627 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:28.039254  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:27.980105  7571 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.006934  8037 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.056406  1684 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:28.001044  7980 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:28.067734  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 51847.3us
W20260709 13:02:28.073268  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 156686us
W20260709 13:02:28.074342  8038 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:02:28.076752  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:28.027961  7926 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.043s
W20260709 13:02:28.063809  8030 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.407s	user 0.001s	sys 0.000s
I20260709 13:02:28.078950  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:28.065925  1536 pending_rounds.cc:85] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 11
I20260709 13:02:28.082474  1536 pending_rounds.cc:107] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 13:02:28.082787  1536 raft_consensus.cc:2889] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:28.088551  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:28.089874  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.091894  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:28.118800  7616 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.752s	user 0.008s	sys 0.004s
I20260709 13:02:28.119346  7616 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 21. Current term is 22.
I20260709 13:02:28.064240  8041 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 13:02:28.121497  1684 raft_consensus.cc:3055] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:28.122023  1684 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:28.130023  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:28.060757  8037 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:28.171072  1611 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.002s	sys 0.017s
I20260709 13:02:28.122924  1684 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:28.191572  1611 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.002s	sys 0.018s
I20260709 13:02:28.191849  1684 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:28.193807  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.132095  8044 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:28.070837  1434 catalog_manager.cc:5697] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 11 to 14, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:28.202107  7939 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:28.217800  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.218297  1534 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:02:28.215847  7990 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:28.225557  7232 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.010s	sys 0.005s
W20260709 13:02:28.227990  7232 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.010s	sys 0.006s
I20260709 13:02:28.230185  1684 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:02:28.231524  1684 pending_rounds.cc:85] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 8
I20260709 13:02:28.231762  1684 pending_rounds.cc:107] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:02:28.231999  1684 raft_consensus.cc:2889] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:28.228340  8045 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:02:28.235517  1539 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
W20260709 13:02:28.238298  8053 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:02:28.240921  7815 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.247354  7815 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:28.228302  8039 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
W20260709 13:02:28.250376  1507 leader_election.cc:336] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.572s (SENT)
I20260709 13:02:28.252224  1434 catalog_manager.cc:5697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.227195  7980 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:28.267133  8037 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:28.272341  7571 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:28.272862  7214 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:28.273305  7214 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:28.274731  8017 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.650s	user 0.011s	sys 0.003s
I20260709 13:02:28.274314  7214 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.275506  7214 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:28.258566  7815 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.296052  7815 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.299250  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:28.296423  8037 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.312249  8037 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.314769  8056 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.314920  8037 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:28.315582  1534 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 13:02:28.315430  8056 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.315999  1534 pending_rounds.cc:85] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 15
I20260709 13:02:28.315862  8065 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:28.316418  1534 pending_rounds.cc:107] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 13:02:28.269795  1434 catalog_manager.cc:5697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.316524  8065 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:28.311117  7969 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.016s	sys 0.017s
I20260709 13:02:28.332584  1539 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:28.333484  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 75125.4us
I20260709 13:02:28.324707  8056 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:28.314805  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 1277) took 1468 ms. Trace:
I20260709 13:02:28.326929  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:28.338407  1543 rpcz_store.cc:276] 0709 13:02:26.846529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.846670 (+   141us) service_pool.cc:224] Handling call
0709 13:02:28.314582 (+1467912us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b540012cad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:28.314741 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1107096448}
I20260709 13:02:28.328070  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:28.304736  7883 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:28.316746  1534 raft_consensus.cc:2889] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:28.319322  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:28.330691  1610 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:28.332644  1541 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:02:28.321846  1434 catalog_manager.cc:5697] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 15 to 17, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 17 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:28.336894  1684 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 13:02:28.339203  2230 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:28.342100  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:28.342792  2229 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:28.345041  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:28.345083  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:28.342746  7926 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:28.336915  7969 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.016s	sys 0.017s
I20260709 13:02:28.336750  8065 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:28.355199  7784 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:28.348850  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:28.349216  7906 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.357316  7906 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.357378  7926 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:28.361644  7815 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:28.365991  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:28.366127  1434 catalog_manager.cc:5697] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.352787  7894 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:02:28.353910  2228 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:28.331670  8037 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:28.374078  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:28.375095  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:28.379555  7784 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:28.381170  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:28.381042  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:28.387308  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:28.389806  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:28.368484  1434 catalog_manager.cc:5697] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:28.392047  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:28.394820  1541 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:28.409309  1691 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:02:28.411173  1684 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:28.415217  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:28.424743  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.383852  8037 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.431448  8037 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:28.444634  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:28.449657  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:28.450851  1615 raft_consensus.cc:2410] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 19 because replica has last-logged OpId of term: 17 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 15.
I20260709 13:02:28.397369  7784 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:28.396869  7939 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:28.453536  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:28.464135  1609 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:28.455047  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:28.455848  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:28.459017  7232 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.462073  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:28.465597  7232 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.467911  7232 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:28.472714  7182 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:28.475704  1582 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:28.393833  7992 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 13:02:28.434125  8021 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:28.496896  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 52368.6us
I20260709 13:02:28.433357  7984 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:28.452469  8009 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:28.502941  7182 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:28.459035  7753 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:28.459019  7906 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:28.477617  7939 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:28.462966  7917 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:28.627784  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 122366us
W20260709 13:02:28.587495  1507 leader_election.cc:336] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 2.154s (SENT)
I20260709 13:02:28.641187  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:28.642421  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:28.644369  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:28.645452  1615 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:28.645550  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 58093.2us
I20260709 13:02:28.647675  7232 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:28.653990  1507 leader_election.cc:336] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.597s (SENT)
I20260709 13:02:28.505998  7753 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:28.652376  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 248576us
I20260709 13:02:28.657786  1509 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:28.635005  7917 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:28.627091  7939 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:28.670909  7232 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.669462  7753 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:28.698948  1615 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:28.627141  8009 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.674435  7987 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:28.649348  7784 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:02:28.509825  7984 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.712507  7232 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:28.681236  8056 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:28.730219  7232 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.730584  8027 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.102s
I20260709 13:02:28.730857  7232 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.666685  8065 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:28.733142  8062 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:28.734179  7232 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:28.734031  8062 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:28.736936  8062 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:28.716535  1615 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:28.714411  8060 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:28.715216  7987 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:28.738130  7784 raft_consensus.cc:3055] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:28.741312  7744 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:28.766645  8065 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:28.755775  7987 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:28.755530  8060 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:28.833293  7744 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.836483  8060 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:28.808446  7917 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.826275  7575 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.826861  8009 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:28.827626  8077 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:28.859313  8077 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:28.860368  8077 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.829510  7984 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:28.744726  8056 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.756923  7784 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:28.843240  1507 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.953s (SENT)
I20260709 13:02:28.837052  7744 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:28.857532  7575 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:28.864467  8077 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:28.876699  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 218782us
W20260709 13:02:28.890447  1507 leader_election.cc:336] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.732s (SENT)
I20260709 13:02:28.879793  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:28.917395  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:28.874435  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 486449us
I20260709 13:02:28.950675  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:28.951745  1613 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:28.952750  1618 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:02:28.877487  7883 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.530s	user 0.010s	sys 0.003s
W20260709 13:02:28.969952  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 90855.6us
I20260709 13:02:28.949338  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:28.979022  1658 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:28.981204  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:28.953253  8060 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:28.983111  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:28.950011  7984 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.980744  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:28.987643  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:28.968000  8062 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:28.957322  8075 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:02:28.976555  7969 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.010800  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:28.991932  7980 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:29.016927  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:29.019670  8052 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.827s	user 0.005s	sys 0.000s
I20260709 13:02:29.020814  8056 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:29.020989  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1462) took 1504 ms. Trace:
I20260709 13:02:29.021560  2234 rpcz_store.cc:276] 0709 13:02:27.516148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.516292 (+   144us) service_pool.cc:224] Handling call
0709 13:02:27.539069 (+ 22777us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:27.539552 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.539616 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:27.561103 (+ 21487us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.009667 (+448564us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:28.009745 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.010171 (+   426us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:28.872969 (+862798us) spinlock_profiling.cc:243] Waited 3.47 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:29.020579 (+147610us) raft_consensus.cc:1685] finished
0709 13:02:29.020611 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.020908 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":645,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":134,"spinlock_wait_cycles":7972736,"thread_start_us":405467,"threads_started":1,"wal-append.queue_time_us":503169}
I20260709 13:02:28.983605  7883 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:28.998965  7815 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:28.990723  7984 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.012130  8084 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:29.016985  7980 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:29.080659  7980 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:29.035235  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:29.082461  1538 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:29.087927  8009 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:29.008502  8074 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 13:02:29.088544  8009 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:29.054383  7658 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.023s	sys 0.021s
I20260709 13:02:29.090371  8074 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:29.090728  8009 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:29.000305  8062 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:28.989970  7784 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:28.995505  8065 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:02:29.081341  8088 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:28.992370  7939 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.035287  8090 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:29.010557  7969 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.060284  7917 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:29.091120  8074 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:29.031153  7883 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:29.098570  1538 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
W20260709 13:02:29.096630  1657 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.040s	sys 0.057s
W20260709 13:02:29.110857  1657 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.041s	sys 0.058s
W20260709 13:02:29.111387  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 609986us
I20260709 13:02:29.040059  7753 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.122800  1683 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:29.036156  7815 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.125321  1535 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
W20260709 13:02:29.143893  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 109498us
I20260709 13:02:29.112788  7575 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:29.096921  8083 consensus_queue.cc:1059] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:02:29.121376  8075 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:29.152079  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 142755us
I20260709 13:02:29.145738  7784 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:29.110133  8090 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.157253  2203 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:29.145038  8074 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:29.161684  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 173116us
W20260709 13:02:29.165230  8086 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 13:02:29.165792  8089 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 13:02:29.166601  8090 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:29.161314  8091 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:29.165578  8084 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:29.231887  8084 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:29.187757  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.189644  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:29.190799  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.233789  1684 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:29.195698  8093 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:29.196593  1686 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:29.198621  7753 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:29.199836  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.201195  8062 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:29.202605  7987 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.157289  8075 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:29.204133  7984 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:29.205092  7883 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:29.175028  8094 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:29.218528  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1299) took 1353 ms. Trace:
I20260709 13:02:29.242435  1534 rpcz_store.cc:276] 0709 13:02:27.864982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.226670 (+361688us) service_pool.cc:224] Handling call
0709 13:02:28.315231 (+ 88561us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.340747 (+ 25516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.218303 (+877556us) spinlock_profiling.cc:243] Waited 85.3 ms on lock 0x7b540012cad0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:29.218461 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196237696}
I20260709 13:02:29.220566  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:29.221488  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:29.243705  1538 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:29.243778  1537 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:29.226256  7939 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:29.226444  7926 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:29.230054  7815 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:29.231950  8091 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:29.173019  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:29.238596  1690 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:29.239585  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:29.170511  8065 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:29.255959  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.246915  8075 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:29.277215  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:29.274012  1543 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:29.277923  2230 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:29.254119  7575 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:29.279021  8065 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.281069  7575 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:29.281975  7987 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:29.247629  6976 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:29.283129  7575 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:29.283707  7984 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:29.279964  8093 raft_consensus.cc:697] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:29.284255  7984 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.282789  6976 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.285035  8093 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.286407  7984 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:29.287002  6976 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:29.287977  8076 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.607s	user 0.000s	sys 0.004s
W20260709 13:02:29.288636  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:29.293033  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 120046us
I20260709 13:02:29.293563  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.293946  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:29.282819  8065 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:29.298763  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:29.305704  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:29.307090  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:29.307288  8065 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:29.308027  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:29.312127  1427 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab5b971190d342d2b2568c30f3c0c81f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cecad4f86a114a7696e398b3e8cc511e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:29.312227  8084 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:29.312943  1538 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:29.298425  7939 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:29.322197  2230 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:29.299871  7753 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:29.280066  8097 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:29.325327  7753 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.325376  8097 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.327867  8097 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:29.328055  2228 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:29.329085  7753 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:29.305766  8098 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:29.256186  8094 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:29.373836  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 132811us
I20260709 13:02:29.375599  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:29.311200  8074 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.369343  8065 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:29.397643  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:29.402266  8098 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:29.403024  8098 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:29.404258  8098 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.378221  8090 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:29.396812  8077 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.002s
I20260709 13:02:29.374393  7980 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:29.436511  2234 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:29.441079  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 135468us
I20260709 13:02:29.373950  8091 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:29.443204  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 121381us
I20260709 13:02:29.444355  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:29.423369  8090 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.452200  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:29.435911  6976 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:29.456148  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:29.469177  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:29.452301  8107 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:29.477530  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:29.478196  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1358) took 2798 ms. Trace:
I20260709 13:02:29.475878  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1310) took 1078 ms. Trace:
I20260709 13:02:29.478650  1685 rpcz_store.cc:276] 0709 13:02:26.671807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:26.671979 (+   172us) service_pool.cc:224] Handling call
0709 13:02:26.682931 (+ 10952us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:26.835686 (+152755us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:27.070018 (+234332us) log.cc:844] Serialized 14 byte log entry
0709 13:02:27.361612 (+291594us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:27.361669 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:27.361719 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:28.397090 (+1035371us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b18001271e0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9cd2a6 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419
0709 13:02:29.470296 (+1073206us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b540038aed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:29.470352 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.470496 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":200114,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":96,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":8607,"spinlock_wait_cycles":356740096,"thread_start_us":463049,"threads_started":2,"wal-append.queue_time_us":428865,"wal-append.run_cpu_time_us":1194,"wal-append.run_wall_time_us":1112240}
I20260709 13:02:29.483330  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:29.479708  1541 rpcz_store.cc:276] 0709 13:02:28.397776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.399828 (+  2052us) service_pool.cc:224] Handling call
0709 13:02:28.400304 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.400547 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:28.439935 (+ 39388us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.475333 (+1035398us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.475543 (+   210us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.475595 (+    52us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:29.475644 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.475711 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.475820 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1034522,"prepare.queue_time_us":40290,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":4688,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":91}
I20260709 13:02:29.483266  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:29.484752  2232 raft_consensus.cc:2368] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:29.476840  7939 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:29.487234  8090 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:29.487478  7980 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:29.483549  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1306) took 1095 ms. Trace:
I20260709 13:02:29.488787  2236 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:29.495604  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:29.457758  6976 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.500622  6976 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:29.501551  6976 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:29.502645  6976 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.507639  6976 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:29.477514  8065 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:29.483409  8009 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.544898  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:29.545889  8036 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.556s	user 0.004s	sys 0.001s
I20260709 13:02:29.545930  2230 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:29.547331  8094 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:29.547622  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1377) took 2038 ms. Trace:
I20260709 13:02:29.548158  1689 rpcz_store.cc:276] 0709 13:02:27.509147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.509277 (+   130us) service_pool.cc:224] Handling call
0709 13:02:27.511106 (+  1829us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:27.511484 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:27.511511 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:27.512083 (+   572us) log.cc:844] Serialized 52 byte log entry
0709 13:02:27.651180 (+139097us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:27.651220 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:27.651287 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.509691 (+1858404us) spinlock_profiling.cc:243] Waited 5.56 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:29.547212 (+ 37521us) raft_consensus.cc:1685] finished
0709 13:02:29.547242 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.547548 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2477,"raft.queue_time_us":1110,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5181,"spinlock_wait_cycles":12784896,"thread_start_us":138735,"threads_started":1,"wal-append.queue_time_us":477564}
W20260709 13:02:29.548174  8058 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:29.551766  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:29.552920  8094 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:29.553606  8094 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:29.553884  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2290) took 1114 ms. Trace:
I20260709 13:02:29.554137  8094 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:29.554406  1617 rpcz_store.cc:276] 0709 13:02:28.439487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.439645 (+   158us) service_pool.cc:224] Handling call
0709 13:02:28.440107 (+   462us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:28.440625 (+   518us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:28.493937 (+ 53312us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.319516 (+825579us) spinlock_profiling.cc:243] Waited 2.32 ms on lock 0x7b180043d680. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a 00007fe08faf6a2a
0709 13:02:29.438148 (+118632us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:29.438794 (+   646us) log.cc:844] Serialized 52 byte log entry
0709 13:02:29.438940 (+   146us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:29.439139 (+   199us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.442161 (+  3022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.442239 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.553565 (+111326us) raft_consensus.cc:1685] finished
0709 13:02:29.553594 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.553784 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":59477,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":4498,"raft.queue_time_us":4821,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":14148,"spinlock_wait_cycles":5330816,"thread_start_us":117799,"threads_started":1,"wal-append.queue_time_us":970845}
W20260709 13:02:29.558529  1610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2677) took 1665 ms (client timeout 1657 ms). Trace:
W20260709 13:02:29.559115  1610 rpcz_store.cc:269] 0709 13:02:27.893449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:27.893563 (+   114us) service_pool.cc:224] Handling call
0709 13:02:29.558251 (+1664688us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b54002bdc54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:29.558422 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679894784}
W20260709 13:02:29.559794  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.560302  7926 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:29.561086  7926 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.563823  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1236) took 1212 ms. Trace:
I20260709 13:02:29.564339  2233 rpcz_store.cc:276] 0709 13:02:28.351359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.351483 (+   124us) service_pool.cc:224] Handling call
0709 13:02:28.351876 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:28.371424 (+ 19548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:28.371457 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:28.371941 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 13:02:28.478788 (+106847us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:28.478878 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.563308 (+1084430us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b5400472a54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:29.563373 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.563447 (+    74us) raft_consensus.cc:1685] finished
0709 13:02:29.563469 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.563748 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26959,"raft.queue_time_us":403,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":9889,"spinlock_wait_cycles":44016000,"thread_start_us":74573,"threads_started":1,"wal-append.queue_time_us":89449,"wal-append.run_cpu_time_us":1635,"wal-append.run_wall_time_us":1001647}
W20260709 13:02:29.567768  8111 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:02:29.492738  8098 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:02:29.575831  8105 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:02:29.578518  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.587486  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:29.596623  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 119223us
I20260709 13:02:29.588425  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:29.593822  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:29.595731  2201 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:29.597374  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:29.597527  1538 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:29.511551  6976 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:29.602088  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.604337  7784 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:29.609099  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:29.613610  1686 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:29.623765  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:29.618889  8090 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:29.599860  7901 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:02:29.618010  7784 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:29.637516  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 147123us
I20260709 13:02:29.621577  8098 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:29.640131  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2292) took 1191 ms. Trace:
I20260709 13:02:29.600975  8009 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:29.618541  7926 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:29.643524  1507 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:29.634071  8083 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:29.644292  1616 rpcz_store.cc:276] 0709 13:02:28.448427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.448574 (+   147us) service_pool.cc:224] Handling call
0709 13:02:29.228594 (+780020us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b5800147398. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7a22 00007fe0958e0c09 00007fe094a3aee0 00007fe094b040ab 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:02:29.228868 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.229173 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.229212 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.229840 (+   628us) log.cc:844] Serialized 26 byte log entry
0709 13:02:29.372545 (+142705us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:29.372599 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.377008 (+  4409us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:29.575200 (+198192us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:29.639888 (+ 64688us) raft_consensus.cc:1685] finished
0709 13:02:29.639917 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.640078 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26100736,"thread_start_us":142319,"threads_started":1,"wal-append.queue_time_us":213137}
I20260709 13:02:29.654656  1583 leader_election.cc:400] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:29.642148  7784 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.656244  1583 leader_election.cc:403] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:29.660440  6976 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:02:29.673147  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:29.656250  8083 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:29.674348  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:29.676402  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:29.677132  1614 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:29.677608  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:29.679159  1616 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:29.677692  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:29.680330  1617 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: []
W20260709 13:02:29.681092  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.682221  1611 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:29.676919  7987 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:29.690474  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:29.691352  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.690908  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:29.696080  1540 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 13:02:29.700307  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:29.707293  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:29.708488  1611 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:29.672621  8122 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 13:02:29.691800  7926 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:29.693681  8098 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.677594  7883 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
W20260709 13:02:29.723188  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 63062.8us
I20260709 13:02:29.730592  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:29.726822  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:29.733649  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:29.734467  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:29.732648  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:29.733523  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:29.735493  2233 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:29.724663  8126 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.738607  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:29.726815  1538 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:29.738834  1583 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:29.740490  8083 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:29.760852  8009 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:29.761541  8009 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:29.761898  8009 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:29.762707  8009 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.763927  8009 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:29.753692  1507 leader_election.cc:336] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.662s (SENT)
I20260709 13:02:29.772264  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:29.755033  8118 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 13:02:29.775144   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:29.796007  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.796793  1692 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:29.810343  7544 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.811168  7544 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:29.743449  8126 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:29.820125  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 63971.9us
I20260709 13:02:29.732126  8098 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.832409  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.827661  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:29.836591  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.835301  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:29.841408  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.845113  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:29.846047  7658 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:29.846455  7658 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.846796  7658 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:29.846856  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:29.848117  7985 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:29.849695  8025 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:29.850088  8025 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:29.850459  8025 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:29.844483  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:29.854343  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.854766  1507 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:29.854914  1685 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:29.855891  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.856454  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:29.856704  8132 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:29.859465  1685 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:29.813781  8122 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:29.862155  1507 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:29.864935  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:29.866323  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:29.867718  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:29.874540  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:29.876595  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:29.840674  7883 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:29.859216  8084 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:29.886534  6976 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:29.893105  1534 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:29.904033  1536 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:29.907964  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:29.905000  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:29.861178  8090 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:29.905139  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:29.863867  8062 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:29.879329  7743 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.863574  8122 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:29.843779  8088 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.062s
I20260709 13:02:29.870501  7744 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:29.888051  7985 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:29.904798  7987 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:29.885733  7883 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:29.845045  8094 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.943198  8094 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:29.926527  8128 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:02:29.944850  8065 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:29.945381  8065 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:29.945796  8027 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:29.946355  8027 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.947468  8100 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:29.948207  8100 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.949019  8027 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:29.885380  8064 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:29.950170  8064 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:29.951831  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:29.952322  8064 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:29.952567  2232 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:29.955030  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3187) took 1547 ms. Trace:
I20260709 13:02:29.955475  1683 rpcz_store.cc:276] 0709 13:02:28.407595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.410487 (+  2892us) service_pool.cc:224] Handling call
0709 13:02:28.440932 (+ 30445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:29.144619 (+703687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.954747 (+810128us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b54000749d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:29.954952 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1566590720}
I20260709 13:02:29.956347  8108 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:29.956859  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.957000  8108 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.860507  7658 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:29.937744  8083 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:29.999174  8124 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.270s	user 0.001s	sys 0.002s
I20260709 13:02:29.999646  8108 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.000547  7784 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:29.489017  1542 rpcz_store.cc:276] 0709 13:02:28.388097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.388294 (+   197us) service_pool.cc:224] Handling call
0709 13:02:28.415676 (+ 27382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:28.415964 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:28.439505 (+ 23541us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.041204 (+601699us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b1800472540. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff522 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:02:29.474961 (+433757us) spinlock_profiling.cc:243] Waited 6.72 ms on lock 0x7b1800472540. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c979c0a 00007fe0960b9219 00007fe08ca01a40 00007fe08c9fc4e4 00007fe08c9fc455 00007fe0934e43c7 00007fe08faf69a7 00007fe08fb060bf 00007fe08fb06038
0709 13:02:29.483018 (+  8057us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.483220 (+   202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:29.483276 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:29.483323 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.483386 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:29.483498 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1013993,"prepare.queue_time_us":1077340,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":80,"raft.queue_time_us":5497,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":10001,"spinlock_wait_cycles":19797632}
I20260709 13:02:30.001637  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:30.004665  8098 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:29.919023  7815 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:29.959115  1583 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:29.916308  8062 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:29.932335  7987 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:29.934689  8134 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:29.930171  7744 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:30.006547  8120 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.358s	user 0.000s	sys 0.002s
I20260709 13:02:30.009089  8100 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:30.012338  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:29.921801  7743 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.045269  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:30.022867  8009 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:30.099236  1685 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 13.
I20260709 13:02:29.924078  8122 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:29.953089  8064 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:30.015069  1616 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:30.017019  1692 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:30.103786  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:30.104691  1692 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:30.105907  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:30.026868  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:30.027784  2234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:30.110569  2234 raft_consensus.cc:606] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:02:30.110944  2234 raft_consensus.cc:614] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:30.001179  8126 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.035069  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:30.112888  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:30.113687  1692 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:29.968217  7883 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:29.967343  7544 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:30.039011  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:30.039630  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 80608.7us
I20260709 13:02:30.008980  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:30.024739  8091 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:30.039342  7987 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:30.036574  8075 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:30.115953  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:30.036669  8062 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:30.038770  7815 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.040650  8123 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:30.118641  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 1317) took 1140 ms. Trace:
I20260709 13:02:30.137768  7815 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:30.138509  8123 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:30.139182  8123 raft_consensus.cc:697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:30.140312  8123 consensus_queue.cc:237] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:30.024611  7980 consensus_queue.cc:1059] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.077s
I20260709 13:02:30.112334  8021 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:30.142644  8021 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:30.046541  8108 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:30.145637  8108 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:30.146165  8108 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:30.036810  7784 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.026036  8083 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:30.114477  8009 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:30.149607  8021 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:30.150969  8083 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:30.151441  8009 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:30.115221  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:30.129681  7924 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:30.128118  1687 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:30.149196  7784 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.101902  8098 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:30.033990  8025 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.138175  1535 rpcz_store.cc:276] 0709 13:02:28.978253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:28.979692 (+  1439us) service_pool.cc:224] Handling call
0709 13:02:30.118482 (+1138790us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54003453d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:30.118587 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":314764544}
W20260709 13:02:30.163007  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.115723  8064 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:30.192049  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 157037us
I20260709 13:02:30.098932  8134 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.216536  2230 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:30.214846  7544 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:30.173187  8009 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:30.259393  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 227729us
I20260709 13:02:30.262804  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:30.263159  1611 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 16. (term mismatch)
I20260709 13:02:30.263259  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:30.282356  1617 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:30.113335  7985 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.263757  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:30.126091  8091 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.172340  8075 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:30.281279  1611 pending_rounds.cc:85] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 15
I20260709 13:02:30.312767  1611 pending_rounds.cc:107] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 18.16
I20260709 13:02:30.313103  1611 raft_consensus.cc:2889] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:30.223484  8148 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:30.200754  8094 consensus_queue.cc:1059] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:02:30.310639  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:30.194152  8065 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.192324  7924 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:30.366905  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:30.317279  1686 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:30.318133  2234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "474d3b57c33542aba459b97733122bf0"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:30.374675  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:30.387775  2234 raft_consensus.cc:606] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:02:30.388345  2234 raft_consensus.cc:614] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:30.367790  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3202) took 1076 ms. Trace:
I20260709 13:02:30.367686  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:30.390821  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:30.365795  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:30.392803  1538 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 13.
I20260709 13:02:30.318488  8075 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:30.319324  8147 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:30.306850  8126 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.389894  1690 rpcz_store.cc:276] 0709 13:02:29.291332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.302730 (+ 11398us) service_pool.cc:224] Handling call
0709 13:02:29.303058 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:29.305168 (+  2110us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:29.433588 (+128420us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.541750 (+108162us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.541961 (+   211us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:29.542601 (+   640us) log.cc:844] Serialized 26 byte log entry
0709 13:02:29.542825 (+   224us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:29.542875 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.542971 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:30.005924 (+462953us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:30.367564 (+361640us) raft_consensus.cc:1685] finished
0709 13:02:30.367592 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.367732 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22023,"prepare.queue_time_us":292592,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":258,"raft.queue_time_us":52423,"raft.run_cpu_time_us":461,"raft.run_wall_time_us":35674,"spinlock_wait_cycles":8193280,"thread_start_us":185902,"threads_started":2,"wal-append.queue_time_us":179520}
I20260709 13:02:30.429387  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:30.429831  2231 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:02:30.430112  2228 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:30.279551  7544 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.324106  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:30.325063  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:30.209143  8098 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:30.436523  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.351716  7987 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:30.352401  7744 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:30.367187  8021 raft_consensus.cc:3055] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:30.355268  8091 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.407275  7870 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:30.365139  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:30.365352  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:30.394498  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:30.486229  1537 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:30.395521  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:30.487376  2230 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:30.397970  1685 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:30.367311  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:30.284655  7575 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:30.386802  1534 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:30.262481  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:30.287524  7933 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.138s
I20260709 13:02:30.414446  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:30.349232  2237 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
W20260709 13:02:30.379192  8139 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.519s	user 0.001s	sys 0.000s
I20260709 13:02:30.447376  1686 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:30.467317  2233 raft_consensus.cc:2410] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
W20260709 13:02:30.499150  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 133853us
I20260709 13:02:30.472781  1687 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:30.353001  7658 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:30.474232  2229 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:30.475299  1540 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:30.511454  8150 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:30.366500  8025 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:30.348855  7969 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.512609  7987 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:30.512737  7969 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:30.513522  7987 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:30.384014  8133 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.513908  7987 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:30.363477  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:30.514303  7987 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:30.514585  1535 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:30.347106  7901 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.130s
I20260709 13:02:30.281702  8009 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.367205  8062 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.465299  8083 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:30.477792  8075 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:30.496439  1616 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:30.506883  8094 consensus_queue.cc:1243] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:02:30.507346  7980 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:02:30.507648  8064 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.501183  2232 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
W20260709 13:02:30.508625  8149 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 13:02:30.509621  8155 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:02:30.365800  7906 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:30.494820  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:30.557314  1536 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:30.465176  8126 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.478047  8157 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:30.538830  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 278134us
I20260709 13:02:30.478180  8021 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:30.503726  1507 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:30.572016  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68382.8us
I20260709 13:02:30.348454  8148 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.495918  8151 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.543610  7784 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:02:30.555708  1657 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:30.521011  8150 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:30.566414  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 181867us
I20260709 13:02:30.567164  1508 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:30.584532  1508 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:30.567786  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 73082.9us
W20260709 13:02:30.546976  8111 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 13:02:30.554247  8084 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.644s	user 0.007s	sys 0.003s
I20260709 13:02:30.532949  2236 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:30.575310  1537 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:30.502493  7870 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:30.513327  7969 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:30.562384  7906 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:30.534929  7575 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:30.596577  1691 raft_consensus.cc:2410] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:02:30.590606  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:30.590061  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:30.547400  8108 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.580247  8132 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:30.577468  7658 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:30.514320  8133 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:30.585934  8150 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:30.587823  8152 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:30.610854  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1396) took 1031 ms. Trace:
I20260709 13:02:30.556241  8075 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:30.557848  8083 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.658355  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:30.656462  1689 rpcz_store.cc:276] 0709 13:02:29.579809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.580181 (+   372us) service_pool.cc:224] Handling call
0709 13:02:29.580648 (+   467us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:29.581110 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:29.581148 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:29.582457 (+  1309us) log.cc:844] Serialized 52 byte log entry
0709 13:02:29.860067 (+277610us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:02:29.908412 (+ 48345us) log.cc:844] Serialized 14 byte log entry
0709 13:02:29.911871 (+  3459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:29.911945 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:30.374640 (+462695us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b1800084940. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:02:30.378365 (+  3725us) spinlock_profiling.cc:243] Waited 4.98 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:30.610561 (+232196us) raft_consensus.cc:1685] finished
0709 13:02:30.610592 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:30.610779 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3192,"prepare.queue_time_us":45829,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":7975,"raft.run_cpu_time_us":247,"raft.run_wall_time_us":2833,"spinlock_wait_cycles":15422080,"thread_start_us":276990,"threads_started":1,"wal-append.queue_time_us":262839}
W20260709 13:02:30.696565  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 103636us
I20260709 13:02:30.612056  7658 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:30.701198  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 99508.3us
W20260709 13:02:30.699404  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 114972us
W20260709 13:02:30.705053  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 121571us
I20260709 13:02:30.710286  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2234) took 1010 ms. Trace:
I20260709 13:02:30.613180  8091 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:30.596275  8064 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:02:30.711344  6976 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 13:02:30.710574  1583 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:30.719787  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:30.712534  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:30.718773  1691 rpcz_store.cc:276] 0709 13:02:29.690114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:29.690245 (+   131us) service_pool.cc:224] Handling call
0709 13:02:30.700398 (+1010153us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b540003dc54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:30.700567 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2069584640}
I20260709 13:02:30.723801  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:30.636040  8152 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:30.609982  1427 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab5b971190d342d2b2568c30f3c0c81f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cecad4f86a114a7696e398b3e8cc511e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:30.577687  8157 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.732402  1535 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
W20260709 13:02:30.731076  1509 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.632s (SENT) [suppressed 24 similar messages]
I20260709 13:02:30.737840  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:30.732033  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:30.703536  7993 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:30.584345  8151 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:30.704147  8132 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:30.751801  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:30.753175  2232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:30.754958  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:30.779736  2232 raft_consensus.cc:606] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:30.758036  7743 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:30.739723  1427 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 132109us
I20260709 13:02:30.722330  7658 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:30.763993  7772 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:30.765154  8009 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.766394  7870 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:30.767752  8075 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:30.768986  8064 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:30.737692  7510 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 13:02:30.703918  8161 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:30.770843  7969 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:30.875290  8151 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:30.875063  7987 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:30.876197  7772 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:30.877220  7772 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.773074  8065 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.879283  7987 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:30.787695  7993 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:30.610020  8122 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:30.733384  8104 log.cc:927] Time spent T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:02:30.780260  2232 raft_consensus.cc:614] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:30.734874  8152 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:30.732615  7784 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:30.733383  8134 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:30.888559  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 170405us
I20260709 13:02:30.629607  8021 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.750849  8091 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 20 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:30.864282  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 112902us
I20260709 13:02:30.893848  1582 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:30.787480  7924 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:30.870321  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 72564.1us
W20260709 13:02:30.898401  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 146659us
I20260709 13:02:30.773608  7906 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:30.882772  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 178453us
I20260709 13:02:30.804389  8145 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:30.797820  8098 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.697253  8168 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.114s
I20260709 13:02:30.718264  7980 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:30.870967  8148 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.897382  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:30.880561  8075 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:30.882449  8062 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.908114  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:30.894066  8065 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:30.913170  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:30.913370  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:30.928254  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:30.916570  1534 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:30.920547  1543 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:30.923206  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:30.925278  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:30.931190  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:30.931991  1535 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:30.935127  1583 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:30.930697  1540 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
W20260709 13:02:30.951735  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:30.909901  7575 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:30.998669  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:30.903854  8145 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:30.904366  8126 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.961814  8077 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:30.934132  7883 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.811s	user 0.006s	sys 0.005s
I20260709 13:02:30.938517  7906 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:30.906849  8175 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:30.981161  8108 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.981612  8183 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:30.982813  2234 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:30.998226  8065 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 18 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:30.984545  8091 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:30.933274  8131 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:30.986652  8083 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.987932  8122 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:30.922103  7510 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 13:02:30.937772  7753 consensus_queue.cc:1059] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:02:30.978435  7993 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:30.997526  8160 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
W20260709 13:02:31.006397  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 83291.8us
W20260709 13:02:31.006099  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 286381us
I20260709 13:02:31.007709  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:31.010049  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:31.011049  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:31.018508  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 83484us
I20260709 13:02:31.010135  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:31.021260  2234 raft_consensus.cc:606] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:02:30.922709  7980 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.001515  8145 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:31.027575  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:31.030232  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:31.030234  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:30.998008  8168 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 13:02:31.008774  7575 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:31.020236  8174 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:31.031454  8133 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:31.042548  1543 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:31.025962  8183 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:31.048664  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:31.050406  1508 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:31.080602  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:31.081064  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:31.082856  1508 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:31.019904  8175 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.051580  2234 raft_consensus.cc:614] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:31.040851  8075 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.008301  7883 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.058589  1658 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:31.088205  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:31.089596  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:31.079180  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 304369us
I20260709 13:02:31.084869  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.068112  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:31.068429  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:31.090477  1539 raft_consensus.cc:2410] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:31.059139  1687 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:31.091621  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:31.043696  7753 raft_consensus.cc:3055] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:31.094416  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.094416  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.049836  1691 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:31.098572  1611 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:31.099076  1534 raft_consensus.cc:2368] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:31.049085  8103 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.102638  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.107465  1617 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:31.104542  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:31.108997  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.106551  1542 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:02:31.053331  8134 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.104988  1543 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:31.066802  8157 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:31.117048  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.087436  8133 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:31.048954  8176 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:31.137809  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.086699  8062 raft_consensus.cc:2764] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:02:31.055939  8108 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:31.086551  8175 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:31.087680  8131 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:31.112203  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.131471  1618 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:31.066054  8174 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.095453  7926 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:31.060109  8168 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:02:31.129106  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.142587  1541 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:31.043011  8178 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:31.142904  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.142637  1542 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:31.066394  7980 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:31.097661  7753 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:31.145202  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.147858  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.180938  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.103161  8083 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:31.103856  7784 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:31.161150  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:31.132864  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.206347  1618 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:31.163875  1614 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:31.165071  1538 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:31.159562  1534 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:31.144268  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:31.134411  8133 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.093772  8075 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.076814  7658 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:31.227097  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 82936.5us
I20260709 13:02:31.159191  8131 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:31.099608  8098 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:31.182585  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.232693  1535 raft_consensus.cc:2368] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19.
I20260709 13:02:31.184618  7753 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.189538  7544 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:31.189822  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.192844  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2237) took 1500 ms. Trace:
I20260709 13:02:31.197410  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.238605  1609 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:31.185267  8025 raft_consensus.cc:3055] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:31.053109  8150 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.258200  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.258802  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.203717  8146 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:31.216135  1688 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
W20260709 13:02:31.221171  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.143540  8183 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.179383  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:31.272482  1617 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:31.237644  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.238291  1683 rpcz_store.cc:276] 0709 13:02:29.691930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.121312 (+429382us) service_pool.cc:224] Handling call
0709 13:02:30.121673 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:30.152788 (+ 31115us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:30.212705 (+ 59917us) log.cc:844] Serialized 14 byte log entry
0709 13:02:30.779373 (+566668us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:30.780064 (+   691us) log.cc:844] Serialized 26 byte log entry
0709 13:02:30.780210 (+   146us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:30.791166 (+ 10956us) log.cc:844] Serialized 14 byte log entry
0709 13:02:31.101754 (+310588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.192395 (+ 90641us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b5400375354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:31.192452 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:31.192588 (+   136us) raft_consensus.cc:1685] finished
0709 13:02:31.192605 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:31.192778 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":64475,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":4725,"raft.queue_time_us":197347,"raft.run_cpu_time_us":421,"raft.run_wall_time_us":61567,"spinlock_wait_cycles":72519424,"thread_start_us":807505,"threads_started":3,"wal-append.queue_time_us":479965}
I20260709 13:02:31.149242  8157 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
I20260709 13:02:31.060534  8065 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.133574  8134 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.175997  8168 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:31.174750  8178 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:31.246552  8185 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:02:31.247231  8188 consensus_queue.cc:1059] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:02:31.247421  8152 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.248065  8075 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:31.249008  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.278929  7753 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:31.239239  8098 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:31.249847  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:31.250976  8190 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:02:31.252444  8180 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:31.149638  8176 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.215581  8133 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:31.256139  1618 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:31.263736  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.361384  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:31.269692  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.119462  8103 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.211416  7784 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:31.158670  8108 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.279511  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.241346  8131 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:31.236572  7980 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:31.290203  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.284557  8168 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:31.245066  8025 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:31.246248  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 333172us
I20260709 13:02:31.302623  1617 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:31.332880  1534 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:31.249557  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.379670  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:31.346817  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.349527  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.380836  1613 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:31.335214  8183 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:31.356627  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.386039  1542 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:31.278573  8146 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.343633  8094 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:02:31.367587  1535 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:31.367852  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.288000  8178 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:31.368165  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.376935  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.393769  1536 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:31.396284  8146 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:31.393769  1610 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:31.390267  1618 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:31.349557  8098 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:31.417125  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.417896  1539 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:31.418216  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.419395  2201 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:31.420389  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:31.370239  8123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.022s	sys 0.033s
I20260709 13:02:31.431041  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.449226  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.450381  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:31.451265  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 168335us
I20260709 13:02:31.452419  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:31.452874  1536 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:31.442126  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:31.352011  8075 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:02:31.456318  1507 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:31.369410  7784 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:31.371417  7980 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:31.475378  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:31.457794  8084 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:31.419669  8021 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:31.476871  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:31.477375  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:31.477804  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:31.432166  1657 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:31.457187  1539 raft_consensus.cc:2368] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19.
I20260709 13:02:31.479128  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:31.488631  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:31.488744  1584 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:31.473640  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:31.479944  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:31.413286  7883 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:31.482991  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:31.445693  8123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.025s	sys 0.036s
I20260709 13:02:31.430617  7985 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:31.491221  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:31.491878  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.492580  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:31.504796  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.495834  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:31.529068  2232 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:31.496727  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:31.498107  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.502264  8083 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:31.502507  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.503109  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:31.532789  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:31.458570  8162 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:31.533824  2235 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:31.502926  8098 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.504261  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:31.481671  7980 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:31.506362  8094 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:31.508544  7926 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:31.555074  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:31.504793  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:31.538695  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.484045  8146 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:31.546227  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:31.504293  7544 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:02:31.530810  2233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:31.568687  2233 raft_consensus.cc:606] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:31.569069  2233 raft_consensus.cc:614] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:31.535398  8027 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:31.558303  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:31.580650  1583 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:31.566784  6976 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:31.582655  6976 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.583194  6976 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:31.557322  2234 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:31.558070  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.587095  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:31.569850  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 96293.2us
I20260709 13:02:31.494524  8177 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:02:31.491699  8173 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:02:31.588442  1616 raft_consensus.cc:2376] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17. Re-sending same reply.
I20260709 13:02:31.600950  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.592671  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:31.601387  2232 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:31.602041  2230 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:31.595674  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:31.549109  8174 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:31.611184   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:31.510394  8145 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.555855  7926 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:31.613387  2236 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:31.552194  7883 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:31.628723  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 196626us
I20260709 13:02:31.616179   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:31.616879  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:31.587041  8027 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.623104  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:31.547669  8083 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.629976  1657 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:31.600013  8166 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:31.541240  7985 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.525199  8025 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:31.646760  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.645789  6976 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.659053  1612 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:31.548672  8178 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.671130  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:31.671943  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 101015us
I20260709 13:02:31.643375  7926 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:31.646194  8027 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:31.736186  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 155657us
I20260709 13:02:31.739163  1616 raft_consensus.cc:2368] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16.
I20260709 13:02:31.655572  8134 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:31.601475  8126 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
W20260709 13:02:31.748343  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 75622.5us
I20260709 13:02:31.779036  1584 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:31.779057  1613 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 21. Current term is 22. Ops: []
W20260709 13:02:31.750398  8195 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:02:31.750983  8080 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 13:02:31.751624  8108 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:31.762115  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 91480.4us
I20260709 13:02:31.774184  6976 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:31.775669  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 201244us
I20260709 13:02:31.667047  8202 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:31.707535  8152 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:31.781183  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:31.804774  1535 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:31.792506  8083 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:31.798717  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:31.666431  7883 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.801734  8176 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:31.803056  1584 leader_election.cc:400] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19.
I20260709 13:02:31.820655  1584 leader_election.cc:403] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:02:31.759119  8205 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:31.833328  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 44536.5us
I20260709 13:02:31.829717  8207 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:31.831125  8084 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:31.830770  7926 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.831533  7883 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:31.834614  1584 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:31.835265  6976 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:31.841512  1538 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:31.846283  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:31.821476  8134 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:31.880121  8083 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:31.892648  8203 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.236s	user 0.000s	sys 0.000s
I20260709 13:02:31.892163  8083 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:31.893532  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:31.895183  8083 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:31.903344  7926 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:31.904879  7985 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:31.908211  8083 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:31.908951  8083 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.911329  8083 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:31.913697  8083 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:31.914469  8083 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.917378  8083 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:31.920055  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:31.920609  8145 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:31.922024  1684 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:02:31.920764  1685 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:31.933866  2230 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:31.937968  7784 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.938434  7744 raft_consensus.cc:3055] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:31.940128  7744 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:31.940547  7784 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:31.941084  7744 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.862550  8173 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:31.865577  8152 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:31.948423  1685 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
W20260709 13:02:31.948699  7753 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.604s	user 0.006s	sys 0.003s
I20260709 13:02:31.948936  1692 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 21: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:31.949733  8103 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:31.950755  2231 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:31.949910  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:31.881129  8134 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:31.954039  8134 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:31.852351  8205 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:31.959177  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:31.961977  8025 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:31.968719  7744 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:31.855398  8207 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.979386  8100 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:31.980300  8100 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.984948  8100 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:31.972738  8152 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.995932  8100 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:31.996693  8100 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:31.997053  8103 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:31.997010  8152 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:31.997867  8103 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.961151  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:32.000264  8103 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:31.943249  8027 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:31.858526  8084 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:32.005120  8084 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:32.004235  8027 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:32.007469  8100 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:32.010293  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:31.879400  6976 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19.
I20260709 13:02:32.436913  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2260) took 1416 ms. Trace:
W20260709 13:02:32.053370  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.457043  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.049041  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:32.144843  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:32.020722  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:32.432191  8133 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.056s	user 0.010s	sys 0.002s
I20260709 13:02:32.458714  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:32.170557  8084 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.459519  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 3272) took 1210 ms. Trace:
I20260709 13:02:32.459385  8133 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:32.121393  1613 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
W20260709 13:02:32.407789  8201 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.904s	user 0.004s	sys 0.000s
I20260709 13:02:32.460004  1540 rpcz_store.cc:276] 0709 13:02:31.271097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.020318 (+749221us) service_pool.cc:224] Handling call
0709 13:02:32.459453 (+439135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.460680  1615 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:31.927166  8098 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:32.461241  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.462005  1613 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
W20260709 13:02:32.433166  2201 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.004s	sys 0.062s
I20260709 13:02:32.462455  8133 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:32.462921  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:32.410063  8175 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.231s	user 0.011s	sys 0.003s
I20260709 13:02:32.463749  8175 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:32.244728  8187 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.004s	sys 0.027s
W20260709 13:02:32.033106  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.465267  1689 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:32.464202  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.465248  8084 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:32.462960  2201 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.004s	sys 0.062s
I20260709 13:02:32.466953  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:32.467823  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 635420us
I20260709 13:02:32.467931  8084 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:32.464856  8187 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.959s	user 0.007s	sys 0.040s
W20260709 13:02:32.038513  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 187764us
I20260709 13:02:32.457726  1542 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
W20260709 13:02:32.687079  8025 consensus_meta.cc:287] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.725s	user 0.010s	sys 0.000s
I20260709 13:02:32.469187  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.464560  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:32.687722  2229 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:32.456354  1688 rpcz_store.cc:276] 0709 13:02:31.007547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.007735 (+   188us) service_pool.cc:224] Handling call
0709 13:02:32.423215 (+1415480us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54000f6754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:32.423419 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381667712}
W20260709 13:02:32.474655  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.476424  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:32.479144  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:32.481233  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:32.690183  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:32.690650  1538 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:32.690929  2236 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:31.961094  8205 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:31.971879  8207 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:32.490497  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:32.052161  7980 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:32.500209  8176 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.003s	sys 0.013s
I20260709 13:02:32.140551  7658 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:32.327359  8178 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:32.500948  8188 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.119s	user 0.004s	sys 0.010s
I20260709 13:02:32.122032  8146 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:32.172079  8186 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:32.686271  8187 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.354434  8215 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:02:32.257984  8170 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:02:32.677680  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:32.677012  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:32.498646  8131 consensus_meta.cc:287] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.085s	user 0.007s	sys 0.004s
I20260709 13:02:32.677870  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1406) took 2739 ms. Trace:
I20260709 13:02:32.677189  1683 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:32.683761  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 196883us
I20260709 13:02:32.682487  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:32.722456  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:32.682374  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 2328) took 1082 ms. Trace:
I20260709 13:02:32.692914  6976 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:32.758172  6976 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:32.759214  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:32.693137  2236 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:02:32.694800  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:32.695358  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:32.696434  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:32.700124  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:32.700454  1535 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:02:32.761250  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:32.761664  2231 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:32.702171  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:32.689541  8083 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:32.673890  8075 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.822s	user 0.030s	sys 0.041s
I20260709 13:02:32.704226   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:02:32.704172  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.708680  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 314564us
I20260709 13:02:32.712301  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.767465  1613 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
W20260709 13:02:32.682327  1509 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.632s (SENT)
W20260709 13:02:32.352784  8210 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.333s	user 0.000s	sys 0.001s
I20260709 13:02:32.718218  1538 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
W20260709 13:02:32.708039  8176 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.005s	sys 0.018s
I20260709 13:02:32.702963  8080 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.720695  1687 raft_consensus.cc:2393] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:32.710397  8207 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:32.785945  8207 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:32.788352  8207 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:32.715945  8188 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.355s	user 0.005s	sys 0.013s
I20260709 13:02:32.726868  1617 raft_consensus.cc:2376] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17. Re-sending same reply.
I20260709 13:02:32.703630  8025 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:02:32.753719  8131 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:32.806636  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.807435  1617 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:32.750926  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:32.755892  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.701169  1534 raft_consensus.cc:2393] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:32.808390  1610 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:32.809357  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 1160) took 1020 ms. Trace:
I20260709 13:02:32.809787  1610 rpcz_store.cc:276] 0709 13:02:31.789650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.727448 (+937798us) service_pool.cc:224] Handling call
0709 13:02:32.809301 (+ 81853us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.810894  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:32.726279  1684 rpcz_store.cc:276] 0709 13:02:29.938185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.426435 (+488250us) service_pool.cc:224] Handling call
0709 13:02:30.426748 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.922472 (+1495724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.594316 (+671844us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b540014a354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:32.677805 (+ 83489us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3437915904}
W20260709 13:02:32.768873  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 86588.9us
I20260709 13:02:32.763412  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:32.812233  1688 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:32.770936  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:32.812985  1686 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:32.813946  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:32.756088  1427 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab5b971190d342d2b2568c30f3c0c81f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cecad4f86a114a7696e398b3e8cc511e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:02:32.771823  8075 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.926s	user 0.030s	sys 0.041s
I20260709 13:02:32.724505  8157 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.775506  6976 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:32.776791  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:32.726665  8170 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:32.779331  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.724694  8094 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:32.755191  8146 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:32.828043  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.783048  8080 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:32.808410  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.772831  7743 raft_consensus.cc:3055] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:32.830760  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:32.806725  8222 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:32.818203  1611 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:32.726305  8186 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:32.814561  1686 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:32.836822  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:32.820839  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:32.820297  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 69526.3us
I20260709 13:02:32.822877  8178 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:32.791225  8192 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:32.851120  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:32.829025  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.778115  8083 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:32.831403  1617 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 22: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:32.817114  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:32.816257  8212 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:32.835877  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1418) took 1388 ms. Trace:
W20260709 13:02:32.822770  1427 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 66732.3us
I20260709 13:02:32.855161  1691 rpcz_store.cc:276] 0709 13:02:31.447110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.447626 (+   516us) service_pool.cc:224] Handling call
0709 13:02:31.447967 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:31.448307 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.448338 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:31.458093 (+  9755us) log.cc:844] Serialized 26 byte log entry
0709 13:02:31.626056 (+167963us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:31.626113 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.635525 (+  9412us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.351993 (+716468us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:32.830754 (+478761us) raft_consensus.cc:1685] finished
0709 13:02:32.830778 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.835816 (+  5038us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1307,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":2747,"spinlock_wait_cycles":68632960,"thread_start_us":74889,"threads_started":1,"wal-append.queue_time_us":334721}
I20260709 13:02:32.856999  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:32.857199  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:32.864681  1610 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:32.866508  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:32.832157  8226 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:32.869288  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.870257  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2708) took 1393 ms. Trace:
I20260709 13:02:32.870677  1617 rpcz_store.cc:276] 0709 13:02:31.476994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.869042 (+1392048us) service_pool.cc:224] Handling call
0709 13:02:32.870179 (+  1137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:32.832549  7753 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:32.901536  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:32.846860  8207 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:32.839504  6976 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:32.904817  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2698) took 1786 ms. Trace:
I20260709 13:02:32.902765  1686 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
W20260709 13:02:32.873893  8168 consensus_meta.cc:287] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.452s	user 0.023s	sys 0.007s
I20260709 13:02:32.844385  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:32.845774  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:32.836539  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:32.847983  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:32.853035  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.832437  7743 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:32.876091  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.914232  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 77775.4us
I20260709 13:02:32.872290  2201 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:32.826944  8157 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:32.852185  8178 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:32.848174  8222 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:32.904898  6976 consensus_queue.cc:1243] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:02:32.925524  6976 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:32.891513  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.907853  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:32.832945  8094 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:32.924541  8178 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:32.905092  8228 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:32.908099  8226 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:32.920846  8084 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:32.928607  7980 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:32.873764  8212 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:32.918586  8168 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:32.920459  7901 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:32.919406  8131 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:32.912523  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 358910us
I20260709 13:02:32.915236  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:32.916030  2201 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:32.905483  1612 rpcz_store.cc:276] 0709 13:02:31.118495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.122375 (+  3880us) service_pool.cc:224] Handling call
0709 13:02:31.151694 (+ 29319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.673738 (+522044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.904477 (+1230739us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54003c96d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:32.904734 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1166095232}
I20260709 13:02:32.915242  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:32.867399  8175 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:32.753827  8187 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:32.913574  7744 consensus_meta.cc:287] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.944s	user 0.018s	sys 0.006s
I20260709 13:02:32.909126  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.904563  8207 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:32.926218  6976 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:32.850607  8192 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:32.865140  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:32.920663  8025 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:02:32.908620  7753 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:32.981683  1617 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:32.930371  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:32.976119  1613 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:32.941941  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1498) took 1375 ms. Trace:
W20260709 13:02:32.948706  8199 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.394s	user 0.000s	sys 0.003s
W20260709 13:02:32.930106  8194 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.283s	user 0.001s	sys 0.012s
W20260709 13:02:32.948751  8196 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.454s	user 0.000s	sys 0.012s
I20260709 13:02:32.891112  8170 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:32.763633  1616 rpcz_store.cc:276] 0709 13:02:31.600405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.600661 (+   256us) service_pool.cc:224] Handling call
0709 13:02:32.593702 (+993041us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54003c8cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:32.682242 (+ 88540us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315641600}
I20260709 13:02:33.062069  2235 rpcz_store.cc:276] 0709 13:02:31.566344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.566571 (+   227us) service_pool.cc:224] Handling call
0709 13:02:31.593259 (+ 26688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:31.748693 (+155434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.748724 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:31.748768 (+    44us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:31.748796 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:32.941644 (+1192848us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b540023d7d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:32.941702 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:32.941867 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307779968}
I20260709 13:02:33.076152  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.840477  1583 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:33.077114  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 236778us
I20260709 13:02:32.984243  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:32.950711  1614 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:32.963711  1684 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:33.079363  1688 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:32.960229  2236 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
W20260709 13:02:32.946000  8198 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.377s	user 0.002s	sys 0.009s
I20260709 13:02:32.948673  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2266) took 1474 ms. Trace:
I20260709 13:02:32.934176  8133 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:32.935362  8094 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:32.985028  8223 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:32.978900  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 63716.3us
I20260709 13:02:32.955953  8186 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.106210  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:32.929486  1610 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 14.
I20260709 13:02:32.984338  8188 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:02:32.950449  1615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 2308) took 1782 ms (client timeout 1537 ms). Trace:
I20260709 13:02:33.082140  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.107611  1688 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
W20260709 13:02:32.949074  8204 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.146s	user 0.000s	sys 0.003s
I20260709 13:02:33.083379  7575 raft_consensus.cc:2764] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:02:32.986841  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:32.938199  7926 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.158s
I20260709 13:02:33.090174  2234 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:33.086035  1687 rpcz_store.cc:276] 0709 13:02:31.474645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.475117 (+   472us) service_pool.cc:224] Handling call
0709 13:02:32.948415 (+1473298us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b540014a854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:32.948583 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2858316032}
I20260709 13:02:33.000200  8207 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:32.951226  8228 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:32.930576  8218 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 13:02:32.958541  1690 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 14: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:33.096814  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:32.960057  8212 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:33.099190  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 212970us
I20260709 13:02:32.956759  8084 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:32.968472  7901 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:32.981863  8180 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:32.959620  8168 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:33.112517  8150 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:33.112084  8094 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:33.107574  1613 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:33.144788  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:33.093999  8146 raft_consensus.cc:697] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:33.108301  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.067539  1686 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:33.109884  6976 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.103061  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:32.954548  7743 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:32.967228  8134 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.012s	user 0.014s	sys 0.000s
I20260709 13:02:33.074702  7980 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:33.100461  8133 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:33.110246  1615 rpcz_store.cc:269] 0709 13:02:31.167669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.180698 (+ 13029us) service_pool.cc:224] Handling call
0709 13:02:32.950207 (+1769509us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54003499d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:32.950366 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2941202816}
I20260709 13:02:33.155171  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.155820  1615 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:33.156653  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.157073  1615 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:33.114956  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:33.088523  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:33.081183  8232 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:33.098374  1582 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:33.116271  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:32.958781  7883 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:33.162943  8232 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:33.163414  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 65135.1us
I20260709 13:02:33.163473  7883 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.119726  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:33.121269  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 290690us
I20260709 13:02:33.165728  7883 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.166647  8091 raft_consensus.cc:3055] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:33.167042  8091 raft_consensus.cc:740] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:33.167286  8157 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:33.101181  8223 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:33.110630  8170 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:02:33.165588  1582 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:33.168057  8091 consensus_queue.cc:260] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:33.168680  7575 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:33.129849  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:33.131592  1657 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:33.133697  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 374538us
I20260709 13:02:33.134110  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:33.112082  8080 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:33.137851  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:33.184515  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:33.138185  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:32.976467  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 60581.2us
I20260709 13:02:33.143340  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:33.146008  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:33.191824  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:33.192541  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.192713  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:33.120834  8188 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:33.134948  8183 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:33.110954  1616 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:33.153297  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2706) took 1921 ms. Trace:
I20260709 13:02:33.158034  1541 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:33.158313  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 180702us
I20260709 13:02:33.136973  8084 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:33.159405  8083 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:33.136483  7901 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:33.160634  7753 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:33.161572  8231 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 13:02:33.149652  8175 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:33.167088  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:32.956337  8226 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:33.167652  7883 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:33.236351  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:33.169075  7575 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:33.169697  1582 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:33.191267  1508 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:33.203137  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.194392  1618 rpcz_store.cc:276] 0709 13:02:31.231952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.404323 (+172371us) service_pool.cc:224] Handling call
0709 13:02:31.409869 (+  5546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:31.410269 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.410299 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:31.410802 (+   503us) log.cc:844] Serialized 26 byte log entry
0709 13:02:31.617601 (+206799us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:31.617646 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.617719 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.948523 (+1330804us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:33.147447 (+198924us) raft_consensus.cc:1685] finished
0709 13:02:33.147467 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.153231 (+  5764us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1106,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":115,"spinlock_wait_cycles":44316416,"thread_start_us":206446,"threads_started":1,"wal-append.queue_time_us":346752}
I20260709 13:02:33.204981  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.160513  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1353) took 2525 ms. Trace:
I20260709 13:02:33.243122  7575 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:33.245824  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:33.246438  8226 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:33.246799  1536 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:33.247501  8226 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:33.230180  8090 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:33.242277  1537 rpcz_store.cc:276] 0709 13:02:30.635017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:30.635903 (+   886us) service_pool.cc:224] Handling call
0709 13:02:30.636370 (+   467us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:31.010924 (+374554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:31.010970 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:31.027550 (+ 16580us) log.cc:844] Serialized 52 byte log entry
0709 13:02:31.237645 (+210095us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:31.251604 (+ 13959us) log.cc:844] Serialized 14 byte log entry
0709 13:02:31.252117 (+   513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:31.252830 (+   713us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:32.789727 (+1536897us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:33.160216 (+370489us) raft_consensus.cc:1685] finished
0709 13:02:33.160315 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.160439 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13479,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":36,"raft.queue_time_us":98468,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":39845,"spinlock_wait_cycles":383163776,"thread_start_us":525226,"threads_started":2,"wal-append.queue_time_us":312690}
I20260709 13:02:33.225109  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:33.249997  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 58825.5us
I20260709 13:02:33.250139  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:33.250555  2232 raft_consensus.cc:2376] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20. Re-sending same reply.
I20260709 13:02:33.250800  1508 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:33.233870  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:33.251313  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:33.251605  8237 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:33.252533  8237 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:33.237480  1610 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:33.237051  2235 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:33.238601  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:33.253518  8187 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.253681  1538 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:33.241217  1616 raft_consensus.cc:2376] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19. Re-sending same reply.
I20260709 13:02:33.236301  7993 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:33.254446  8187 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:33.254853  7993 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.255303  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:33.256675  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.256940  7993 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:33.248466  7883 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.143469  8180 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.251643  1534 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:33.259768  8180 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.169267  8091 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:33.263114  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.263824  1617 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:33.264145  1508 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:33.264750  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.207268  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.230515  8025 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:33.248944  8090 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:33.231983  8183 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.253022  8133 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.175264  7980 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.154484  8134 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:33.367444  8025 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:33.371484  8134 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:33.371017  8025 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.362066  8090 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:33.373365  8083 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:33.373533  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 239525us
I20260709 13:02:33.374392  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:33.375542  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:33.376401  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:33.171583  8228 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:33.266180  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2305) took 2153 ms. Trace:
I20260709 13:02:33.378988  7980 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.267367  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2703) took 2101 ms. Trace:
W20260709 13:02:33.265899  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 96262us
I20260709 13:02:33.379712  1614 rpcz_store.cc:276] 0709 13:02:31.165938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.219720 (+ 53782us) service_pool.cc:224] Handling call
0709 13:02:32.950526 (+1730806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.082739 (+132213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.267162 (+184423us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54003499d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:33.267308 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3922462080}
I20260709 13:02:33.380151  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:33.266319  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:33.379076  1611 rpcz_store.cc:276] 0709 13:02:31.112447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.112889 (+   442us) service_pool.cc:224] Handling call
0709 13:02:31.119479 (+  6590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:32.835774 (+1716295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.265921 (+430147us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400102554. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:33.266098 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3906214144}
I20260709 13:02:33.382352  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:33.382777  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.383551  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:33.384012  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:33.269608  8222 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:33.165373  8232 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.384603  1611 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:33.384857  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.299412  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:33.202896  8223 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:33.236871  1507 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:33.205513  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 62252.8us
I20260709 13:02:33.390978  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:33.391414  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:33.279956  1615 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
W20260709 13:02:33.328744  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 64666.5us
I20260709 13:02:33.133389  8192 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:33.353608  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:33.364324  8235 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.131s	user 0.000s	sys 0.005s
W20260709 13:02:33.354849  1690 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2270) took 1874 ms (client timeout 2078 ms). Trace:
W20260709 13:02:33.406021  1690 rpcz_store.cc:269] 0709 13:02:31.480313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.480435 (+   122us) service_pool.cc:224] Handling call
0709 13:02:33.270573 (+1790138us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b540041e1d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:33.354785 (+ 84212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3277822336}
I20260709 13:02:33.407720  1690 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
W20260709 13:02:33.355403  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.270339  8188 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:33.171905  8233 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:33.214334  7744 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:33.275033  8146 consensus_queue.cc:237] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:33.265377  1617 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:33.432335  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:33.294907  7926 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:33.151154  8094 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.357564  8249 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:33.359165  1534 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:33.359756  7575 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:33.203435  8238 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:33.363992  2229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 1492) took 1878 ms (client timeout 2088 ms). Trace:
W20260709 13:02:33.364037  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 232521us
I20260709 13:02:33.364059  2234 raft_consensus.cc:2410] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:02:33.366499  8098 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:33.376677  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:33.462654  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 86039us
I20260709 13:02:33.421957  8150 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:33.378247  8176 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:33.236020  8175 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.381035  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.470646  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:33.471055  1612 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:33.471627  1614 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:33.471971  1657 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:33.472308  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1161) took 1647 ms. Trace:
I20260709 13:02:33.472788  1614 rpcz_store.cc:276] 0709 13:02:31.828740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.471188 (+1642448us) service_pool.cc:224] Handling call
0709 13:02:33.471469 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.472070 (+   601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.472255 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.395416  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:33.473706  1684 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:33.439823  7926 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:33.397800  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:33.399645  1538 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 14: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:33.257332  1616 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:33.479418  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.402297  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:33.479990  1688 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:33.480587  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 78397.3us
I20260709 13:02:33.481266  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:33.402287  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1158) took 1697 ms. Trace:
I20260709 13:02:33.482942  1615 rpcz_store.cc:276] 0709 13:02:31.704877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.401596 (+1696719us) service_pool.cc:224] Handling call
0709 13:02:33.401874 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.402023 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.402044 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.402081 (+    37us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:33.402109 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.402153 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.402235 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.483590  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:02:33.411674  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 244646us
I20260709 13:02:33.484108  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.484488  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:33.413744  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:33.414070  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:33.414822  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:33.378351  8084 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:02:33.416787  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 179990us
I20260709 13:02:33.418059  1610 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:02:33.490190  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1159) took 1715 ms. Trace:
I20260709 13:02:33.490537  1610 rpcz_store.cc:276] 0709 13:02:31.775911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.409744 (+1633833us) service_pool.cc:224] Handling call
0709 13:02:33.410222 (+   478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.489971 (+ 79749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.490143 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.491578  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.491715  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:33.427301  1689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2261) took 2397 ms (client timeout 1965 ms). Trace:
I20260709 13:02:33.428236  1685 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:33.432680  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 1169) took 1429 ms. Trace:
I20260709 13:02:33.356391  1541 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:33.492434  1610 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:33.436444  1543 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:33.441006  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:33.381067  8183 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.397085  8133 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:33.456496  1611 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
W20260709 13:02:33.457255  2229 rpcz_store.cc:269] 0709 13:02:31.476798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.477441 (+   643us) service_pool.cc:224] Handling call
0709 13:02:33.269734 (+1792293us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b5400484ed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:33.355298 (+ 85564us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2754727296}
I20260709 13:02:33.463497  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:33.377774  8202 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:33.431325  8232 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:33.449047  8192 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:33.475029  1684 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:33.396577  1542 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:33.476408  1657 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:33.503918  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:33.504945  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:33.476210  2201 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:33.476643  1614 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:02:33.446520  8240 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:33.426687  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:33.492693  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.493600  6976 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.492695  1617 rpcz_store.cc:276] 0709 13:02:32.003191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.264513 (+1261322us) service_pool.cc:224] Handling call
0709 13:02:33.432613 (+168100us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:33.492425  1689 rpcz_store.cc:269] 0709 13:02:31.008159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:31.008285 (+   126us) service_pool.cc:224] Handling call
0709 13:02:33.405051 (+2396766us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54002ce054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:33.405170 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3233929728}
I20260709 13:02:33.493382  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.496794  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:33.496083  1615 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:33.497077  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:33.498862  1611 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:02:33.499655  2231 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:33.466790  8075 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:33.477849  8065 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:02:33.557103  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.507622  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2709) took 1623 ms. Trace:
I20260709 13:02:33.507002  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:33.493356  7883 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.500898  8150 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:33.505910  8228 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:33.498243  8133 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:33.509482  8083 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:33.488559  7980 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:33.510128  8157 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:33.510456  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:33.510881  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:33.559562  1614 rpcz_store.cc:276] 0709 13:02:31.884431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.476131 (+1591700us) service_pool.cc:224] Handling call
0709 13:02:33.476428 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.507355 (+ 30927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.507570 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.568472  2236 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:33.558704  1692 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 22. Current term is 23. Ops: []
W20260709 13:02:33.560928  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 62406.7us
I20260709 13:02:33.569756  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:33.563874  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2319) took 2211 ms. Trace:
W20260709 13:02:33.563867  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 53832.5us
I20260709 13:02:33.439977  8233 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:33.477630  8098 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.090s
W20260709 13:02:33.571904  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64990.3us
I20260709 13:02:33.582520  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:33.500792  8080 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:33.565831  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2711) took 1674 ms. Trace:
I20260709 13:02:33.570065  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1166) took 1567 ms. Trace:
I20260709 13:02:33.573834  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:33.586954  1611 rpcz_store.cc:276] 0709 13:02:31.891752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.494805 (+1603053us) service_pool.cc:224] Handling call
0709 13:02:33.498639 (+  3834us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:33.565156 (+ 66517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.565706 (+   550us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.586989  1616 rpcz_store.cc:276] 0709 13:02:32.002336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.569296 (+1566960us) service_pool.cc:224] Handling call
0709 13:02:33.569597 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.569762 (+   165us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.569787 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.569832 (+    45us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:33.569862 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.569912 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.570012 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.571517  8083 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:33.589277  1616 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:33.505980  8187 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:33.501181  8202 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:33.576792  1612 rpcz_store.cc:276] 0709 13:02:31.352519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.385975 (+2033456us) service_pool.cc:224] Handling call
0709 13:02:33.386248 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.475100 (+ 88852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.563642 (+ 88542us) spinlock_profiling.cc:243] Waited 84.6 ms on lock 0x7b540021ff54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:33.563819 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194623744}
W20260709 13:02:33.577948  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 77719.2us
I20260709 13:02:33.560683  6976 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.498769  7993 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:33.578749  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68721.9us
I20260709 13:02:33.574309  8176 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:33.579875  2237 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:33.564972  8212 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:33.506155  8232 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:33.591229  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:33.575333  1617 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:33.594046  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1175) took 1588 ms. Trace:
I20260709 13:02:33.566650  7744 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.564975  8065 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:33.571450  8133 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:33.651770  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1171) took 1647 ms. Trace:
I20260709 13:02:33.651552  1616 rpcz_store.cc:276] 0709 13:02:32.010854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.588669 (+1577815us) service_pool.cc:224] Handling call
0709 13:02:33.589074 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.593753 (+  4679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.593988 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.652168  1617 rpcz_store.cc:276] 0709 13:02:32.003934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.568879 (+1564945us) service_pool.cc:224] Handling call
0709 13:02:33.569215 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.651502 (+ 82287us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.651715 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.653664  1617 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:33.567502  8150 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:33.600577  1509 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:33.572791  1615 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:33.602473  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1177) took 1480 ms. Trace:
I20260709 13:02:33.655293  1612 rpcz_store.cc:276] 0709 13:02:32.122779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.598635 (+1475856us) service_pool.cc:224] Handling call
0709 13:02:33.601797 (+  3162us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.602027 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.602064 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.602119 (+    55us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:33.602172 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.602243 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.602398 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.656208  1616 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:02:33.593767  8083 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:33.657543  1616 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:33.657833  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1170) took 1654 ms. Trace:
I20260709 13:02:33.658191  1615 rpcz_store.cc:276] 0709 13:02:32.003665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.570777 (+1567112us) service_pool.cc:224] Handling call
0709 13:02:33.571040 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.657602 (+ 86562us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.657781 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.573638  8249 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 13:02:33.636677  6976 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.642541  7926 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.589756  8183 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:33.595309  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2712) took 1553 ms. Trace:
I20260709 13:02:33.605747  8202 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:33.659696  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 170369us
I20260709 13:02:33.655913  8065 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:33.578358  8157 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.670756  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:33.671890  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:33.603857  8131 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 13:02:33.673887  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:33.674623  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:33.597807  7980 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:33.676079  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 84983.6us
W20260709 13:02:33.779404  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 175153us
I20260709 13:02:33.676889  8168 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:33.646765  8207 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
I20260709 13:02:33.615840  1692 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:33.678715  1617 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:33.789825  8227 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:33.679373  8170 consensus_meta.cc:287] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.004s
I20260709 13:02:33.790774  8217 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.283s
I20260709 13:02:33.790478  8227 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.791778  8217 raft_consensus.cc:3055] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:33.792178  8217 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:33.682008  1611 rpcz_store.cc:276] 0709 13:02:32.041541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.590877 (+1549336us) service_pool.cc:224] Handling call
0709 13:02:33.594828 (+  3951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.595006 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:33.595032 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:33.595072 (+    40us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:33.595105 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:33.595158 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:33.595254 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:33.793980  8227 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.684273  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:33.795984  1542 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:33.649679  8080 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:33.706158  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 105680us
I20260709 13:02:33.657434  8134 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.122s
I20260709 13:02:33.647209  8212 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:33.669486  8178 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.712s	user 0.002s	sys 0.013s
I20260709 13:02:33.672610  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:33.659859  8232 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:33.659528  8150 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:33.805599  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:33.805742  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:33.709880  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 139035us
I20260709 13:02:33.806463  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:33.806638  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:33.809283  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.810343  1615 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 22: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:33.806396  1617 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:02:33.733722  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 144819us
I20260709 13:02:33.603507  7993 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:33.783823  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:33.783775  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:33.682579  8228 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.786207  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:33.823668  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:33.782383  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:33.824992  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:33.796835  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 90372.9us
I20260709 13:02:33.826066  1583 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:02:33.799310  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.799634  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:33.710150  8062 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:33.683702  8090 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.662873  8238 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.123s
I20260709 13:02:33.669070  7744 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.800776  8246 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:33.823988  1539 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:33.827553  1541 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:33.804431  8243 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.783224  7980 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:33.818781  1618 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:33.838514  8175 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.834599  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:33.836396  1509 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:33.828912  8254 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.806358  8212 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:33.823083  8178 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.820328  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:33.838939  8168 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:33.793068  8217 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.780643  7901 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.833158  1538 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 13. Current term is 14. Ops: []
I20260709 13:02:33.825943  8180 consensus_queue.cc:1059] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.083s
I20260709 13:02:33.967226  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:33.897723  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 79767.9us
I20260709 13:02:33.898702  6976 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:33.905893  7980 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:33.968310  6976 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:33.931277  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 96774.1us
I20260709 13:02:33.882747  8098 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:33.936905  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 100611us
I20260709 13:02:33.970729  1509 leader_election.cc:400] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16.
I20260709 13:02:33.971115  1509 leader_election.cc:403] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:33.940141  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 114161us
I20260709 13:02:33.883111  8084 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:33.819953  7575 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:33.907718  8131 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:02:33.941318  8175 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:33.946496  8090 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:33.946511  8178 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.907898  8246 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.981694  8175 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:33.986317  8175 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:33.986809  7743 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:33.909129  8244 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.381s	user 0.001s	sys 0.000s
I20260709 13:02:33.910919  8243 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.989599  8243 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:33.990046  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.990278  1535 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:33.991487  8255 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
W20260709 13:02:33.992911  8257 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
W20260709 13:02:33.993659  8258 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.109s	user 0.003s	sys 0.000s
I20260709 13:02:33.994181  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:33.950100  8233 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:33.941336  8254 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:34.015846  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:33.823182  7993 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:33.979977  8009 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:33.884974  8261 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:33.885408  8062 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:33.979944  8094 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:33.840616  8237 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:33.966135  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 163354us
I20260709 13:02:33.972043  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:33.985008  8217 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:33.840392  8249 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:33.787662  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.056051  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:33.800185  8207 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:33.976161  1538 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:33.938999  8226 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:33.980837  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:33.882998  8228 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:33.960655  8168 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:33.994381  8251 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.338s	user 0.001s	sys 0.000s
I20260709 13:02:33.949980  1609 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:33.984522  6976 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.016733  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1173) took 2012 ms. Trace:
I20260709 13:02:33.902176  1541 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:33.981830  8187 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:33.943007  8264 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.105s	user 0.004s	sys 0.000s
I20260709 13:02:34.045368  8237 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:34.046335  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 74375.8us
I20260709 13:02:34.095804  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:33.984143  8134 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:02:34.052143  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 212082us
I20260709 13:02:34.057364  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:34.044798  8273 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:02:34.050138  8270 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:02:33.981910  8075 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:02:34.073812  1539 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:34.075325  1614 rpcz_store.cc:276] 0709 13:02:32.004555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.572166 (+1567611us) service_pool.cc:224] Handling call
0709 13:02:33.572431 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:33.572636 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:33.707588 (+134952us) log.cc:844] Serialized 14 byte log entry
0709 13:02:33.974959 (+267371us) log.cc:844] Serialized 14 byte log entry
0709 13:02:34.016221 (+ 41262us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:34.016301 (+    80us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:34.016374 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.016482 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:34.016659 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":122068,"prepare.queue_time_us":322318,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":159189,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":5867,"spinlock_wait_cycles":104841344,"thread_start_us":298453,"threads_started":2,"wal-append.queue_time_us":242820}
I20260709 13:02:34.084995  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:34.046433  1536 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:34.096534  1691 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:34.054133  8178 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:34.055330  8009 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:33.983919  8227 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 13:02:34.075366  8168 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.098804  8237 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:34.116438  8267 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 13:02:34.121258  8269 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:02:34.121562  8062 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.122583  8094 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:34.126946  8274 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:34.129947  7744 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:34.098402  8134 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:34.100137  8273 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:34.006654  8080 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.138504  1542 raft_consensus.cc:2410] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 13:02:34.139451  8274 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.136981  8094 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:34.095387  8268 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 13:02:34.048700  8261 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.121580  8186 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:02:34.167997  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 189561us
W20260709 13:02:34.170590  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 85677us
I20260709 13:02:34.219509  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:34.101960  8178 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:34.193492  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 132266us
W20260709 13:02:34.196238  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 215436us
I20260709 13:02:34.223495  1509 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:34.228348  1691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 3234) took 1243 ms (client timeout 1583 ms). Trace:
W20260709 13:02:34.228829  1691 rpcz_store.cc:269] 0709 13:02:32.985189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.986520 (+  1331us) service_pool.cc:224] Handling call
0709 13:02:34.228132 (+1241612us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b540003dc54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:34.228284 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2256449920}
I20260709 13:02:34.229642  8275 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:34.231109  1609 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 1165) took 2229 ms (client timeout 1914 ms). Trace:
W20260709 13:02:34.232838  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 134150us
I20260709 13:02:34.233050  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:34.193864  8271 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:34.231555  1609 rpcz_store.cc:269] 0709 13:02:32.001985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:32.789968 (+787983us) service_pool.cc:224] Handling call
0709 13:02:34.230879 (+1440911us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54000bf9d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:34.231032 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2557248768}
I20260709 13:02:34.235244  1509 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:34.197149  8091 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.936s	user 0.016s	sys 0.002s
W20260709 13:02:34.236039  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 179042us
I20260709 13:02:34.193116  8238 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.242658  1683 raft_consensus.cc:2410] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 15 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 13:02:34.242568  8238 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.240615  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:34.217574  8249 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:34.192824  8274 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:34.230216  8275 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.246776  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:34.251189  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:34.306697  1542 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:34.235491  8180 raft_consensus.cc:2764] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:02:34.258111  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:34.239441  8245 consensus_queue.cc:1059] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 13:02:34.218719  8186 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:34.224756  7901 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:34.251601  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:34.230809  8178 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.223549  7883 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:34.308101  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:34.328594  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 106844us
I20260709 13:02:34.294684  8271 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:34.329376  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:34.329629  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:34.330276  1610 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:34.251551  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:34.311094  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:34.312014  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:34.312930  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 246633us
I20260709 13:02:34.316428  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:34.332525  1540 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:34.305683  8084 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:02:34.307587  8226 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:34.327346  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:34.332825  1535 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:34.302337  8249 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:34.294229  8098 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 13:02:34.347986  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:34.322033  8207 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:34.323583  1543 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:34.234380  8150 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.338681  7743 consensus_queue.cc:1059] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:02:34.244309  8176 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.084s
W20260709 13:02:34.347106  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 112731us
I20260709 13:02:34.428706  7993 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:34.409569  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 76891.9us
W20260709 13:02:34.469384  8223 consensus_meta.cc:287] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 1.042s	user 0.006s	sys 0.005s
I20260709 13:02:34.322508  8094 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.443316  8246 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.299757  8134 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.347756  8254 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.245725  8009 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:34.412192  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.405133  8238 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.296276  8273 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:34.406855  8261 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:34.408452  8275 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:34.412333  8278 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:02:34.386113  8287 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.345117  8249 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:34.328382  7883 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:34.434098  7980 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.345913  8084 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:34.438653  8285 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:34.435613  1687 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
W20260709 13:02:34.442523  8192 consensus_meta.cc:287] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.930s	user 0.011s	sys 0.000s
I20260709 13:02:34.570606  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:34.467221  8133 consensus_meta.cc:287] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.790s	user 0.009s	sys 0.002s
W20260709 13:02:34.320513  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 82854.7us
W20260709 13:02:34.419059  8065 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.637s	user 0.004s	sys 0.005s
I20260709 13:02:34.543162  8183 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
W20260709 13:02:34.429540  8202 consensus_meta.cc:287] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.649s	user 0.008s	sys 0.002s
W20260709 13:02:34.482540  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.470904   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:34.424280  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 3235) took 1363 ms. Trace:
I20260709 13:02:34.611776  1683 rpcz_store.cc:276] 0709 13:02:33.060475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.129587 (+ 69112us) service_pool.cc:224] Handling call
0709 13:02:34.423946 (+1294359us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540003eb54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:34.424199 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2466831616}
I20260709 13:02:34.533098  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:34.483209  8232 consensus_meta.cc:287] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.640s	user 0.012s	sys 0.000s
I20260709 13:02:34.537194  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:34.537951  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:34.619184  2232 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:34.540001  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:34.470021  8236 raft_consensus.cc:3055] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:34.543834  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:34.510757  8175 consensus_meta.cc:287] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.000s
I20260709 13:02:34.550592  2233 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:34.554723  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:34.421190  8212 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:34.555714  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 238115us
W20260709 13:02:34.559099  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 455439us
I20260709 13:02:34.544411  8265 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:34.564095  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:34.564766  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:34.572278  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 242962us
I20260709 13:02:34.552520  8094 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:34.543327  8245 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:02:34.583930  1613 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
W20260709 13:02:34.423626  8083 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.740s	user 0.008s	sys 0.000s
W20260709 13:02:34.559767  1427 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab5b971190d342d2b2568c30f3c0c81f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cecad4f86a114a7696e398b3e8cc511e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:34.590936  8133 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.595237  8186 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.597708  8228 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.563867  8134 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:34.598325  7993 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:34.649834  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.654132  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.655076  1614 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:34.625984  8236 raft_consensus.cc:740] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:02:34.600468  8240 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.028s	user 0.010s	sys 0.002s
I20260709 13:02:34.567266  8287 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.578075  8273 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:34.544894  8223 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:34.614410  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 74903us
I20260709 13:02:34.614485  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:34.626717  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:34.622035  1688 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
W20260709 13:02:34.626717  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.627656  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:34.636897  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:34.665904  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 80998.7us
I20260709 13:02:34.637652  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:34.666397  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:34.634550  1684 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:34.667515  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:34.666410  1685 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
W20260709 13:02:34.623265  8187 consensus_meta.cc:287] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.528s	user 0.011s	sys 0.000s
I20260709 13:02:34.647081  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:34.599897  8098 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:34.656450  1614 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:34.650378  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:34.657294  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.648801  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:34.659461  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.672892  1618 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:34.660168  8289 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:34.660745  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.674793  1609 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:34.660616  8268 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:02:34.660988  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:34.653160  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.676729  1617 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:34.677120  1584 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:34.652227  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.633455  8265 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:34.667533  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:34.650058  8080 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:34.681272  1583 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:34.681598  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:34.624161  8232 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.666791  1686 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:34.667824  1657 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:34.671224  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:34.672988  1688 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:34.673832  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:34.683944  1692 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:34.683820  1618 raft_consensus.cc:2376] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19. Re-sending same reply.
I20260709 13:02:34.611374  8065 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.663091  8186 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:34.678711  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:34.679535  2203 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:34.683109  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:34.635648  8175 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.685040  1612 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:34.691701  1584 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:34.677937  1615 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:02:34.647667  1427 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 90349.9us
I20260709 13:02:34.624918  8202 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.668538  8240 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:34.645211  8083 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.696074  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.697245  1686 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:34.693835  1658 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:34.695984  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:34.658708  8274 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.717823  1534 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:34.699875  1617 raft_consensus.cc:3055] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:34.719388  1617 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:34.673727  8228 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:34.674295  8187 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16.
I20260709 13:02:34.704622  8150 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:34.704941  8133 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:34.709030  1685 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:34.709723  8261 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:34.710935  8065 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.711414  8223 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:34.711324  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:34.712030  7980 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:34.712168  8293 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:34.712984  8237 consensus_meta.cc:287] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.575s	user 0.010s	sys 0.000s
I20260709 13:02:34.798069  8186 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:34.797859  8237 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:34.798597  8295 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 13:02:34.685396  8285 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:34.680392  8084 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.681996  7993 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:34.803638  7993 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.804133  7993 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:34.805182  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1186) took 1991 ms. Trace:
W20260709 13:02:34.760756  8258 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:02:34.805646  1612 rpcz_store.cc:276] 0709 13:02:32.814215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.656317 (+842102us) service_pool.cc:224] Handling call
0709 13:02:34.425978 (+769661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.690934 (+264956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.804112 (+113178us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54003219d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:34.804953 (+   841us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b54003219d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:34.805116 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2358287872}
I20260709 13:02:34.674046  8098 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:34.713961  8175 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:34.680722  7575 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:34.806900  8098 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:34.714732  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.807212  8274 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.689637  8289 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.715289  8232 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:34.694918  7743 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:34.685101  8254 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:34.730916  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:34.720715  8288 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:34.735898  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:34.809674  8288 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:34.699576  8168 consensus_meta.cc:287] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.000s
I20260709 13:02:34.810374  8168 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.810755  8288 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.812700  8168 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.737787  8083 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:34.722675  8207 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:34.813796  1688 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:34.769312  8178 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.814992  8157 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.815929  8178 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.816159  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3291) took 1142 ms. Trace:
I20260709 13:02:34.736887  8301 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:02:34.738513  1691 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:34.739368  8236 consensus_queue.cc:260] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.741593  8287 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.663657  7883 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:34.742913  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.743271  8265 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.742132  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:34.834867  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 136753us
I20260709 13:02:34.745805  8202 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.746213  8279 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:34.724709  8290 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:34.746959  8228 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:34.748275  8249 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.759028  1690 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:34.720184  1617 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.696779  7753 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:34.700151  8275 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:34.778373  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 67109.2us
I20260709 13:02:34.833766  8254 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:34.845079  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:34.790663  1685 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:34.694314  8080 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:34.793025  8291 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
W20260709 13:02:34.794138  8294 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:02:34.807294  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3287) took 1135 ms. Trace:
I20260709 13:02:34.847599  1534 rpcz_store.cc:276] 0709 13:02:33.672126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.672255 (+   129us) service_pool.cc:224] Handling call
0709 13:02:33.672562 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.718271 (+1045709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.807065 (+ 88794us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000ada54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:34.807234 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2403427072}
I20260709 13:02:34.807655  1609 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:34.738381  8212 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:34.816592  1543 rpcz_store.cc:276] 0709 13:02:33.673399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.837726 (+164327us) service_pool.cc:224] Handling call
0709 13:02:33.838091 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.486557 (+648466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:34.815897 (+329340us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b54000eddd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:34.816094 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1116031744}
W20260709 13:02:34.820120  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 84274.2us
I20260709 13:02:34.850191  1584 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:34.850720  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:34.851972  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:34.852347  8232 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:34.852834  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:34.853057  8232 raft_consensus.cc:2764] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:02:34.853852  8232 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:34.854337  8232 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:34.854820  8232 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:34.841897  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.813817  8207 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:34.856067  8207 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.848969  8080 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:34.856951  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:34.853096  1685 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:34.860124  8296 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:02:34.862421  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:34.863369  1617 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:34.860870  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:34.864464  1616 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:34.869704  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:34.869915  1508 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:34.874202  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:34.874672  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:34.874941  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:34.876520  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:34.877717  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:34.864113  8265 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:34.903431  8009 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:34.844324  7753 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.858448  8245 consensus_queue.cc:1059] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:02:34.862917  8275 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.905293  1584 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:34.906082  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:34.866174  7743 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:02:34.843823  8237 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.907008  2235 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:34.909144  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:34.909446  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:34.909754  1683 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:34.867918  8202 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:34.911059  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:34.916276  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:34.916697  2236 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:34.920720  2229 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:34.922188  1507 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:34.923447  8084 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:34.927851  8304 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:34.927930  8084 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.928681  8009 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:34.930477  8146 raft_consensus.cc:3055] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:34.930742  8084 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.930922  8146 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:34.931734  8146 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.927254  1685 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:34.931989  8207 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.933619  1685 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:34.933382  8228 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.934939  1685 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:34.935678  8304 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:34.911023  7753 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:34.936698  2229 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:34.938225  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.939196  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:34.908684  8296 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:34.942603  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:34.912493  8275 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:34.916074  8202 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:34.944448  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:34.945109  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:34.943444  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:34.916383  8249 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:34.945828  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.934484  8207 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.949077  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:34.949888  1543 raft_consensus.cc:2410] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:02:34.949268  1537 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
W20260709 13:02:34.947943  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 251117us
I20260709 13:02:34.939759  8301 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:34.951202  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:34.951051  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.942451  8180 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:34.946359  8297 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:34.958251  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:34.958961  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:34.947347  8289 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.960212  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:34.945194  8226 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:34.957942  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:34.951216  8202 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.962298  2229 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:34.963001  8180 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.944622  7575 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:02:34.973670  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.967902  1583 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:34.971212  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:34.971378  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.971751  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.975502  1617 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:34.970474  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:34.945099  8287 consensus_queue.cc:1059] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:02:34.976599  1618 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:02:34.974539  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.979461  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:34.976770  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:34.977785  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.954698  8245 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:02:34.980926  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:34.989130  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:34.981464  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:35.007582  7743 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:35.007640  8287 raft_consensus.cc:3055] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:35.008430  8287 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:35.009117  8232 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.009318  8287 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:35.010780  8287 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:35.010931  8098 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.011539  8232 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:34.962630  8301 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.017536  8286 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:35.018532  1692 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:34.953735  8243 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:02:34.982895  1583 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:34.984519  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:35.021615  1611 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:34.979426  8150 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:35.023604  8308 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:02:34.986617  8237 raft_consensus.cc:3055] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:34.973479  8223 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:02:35.025596  8237 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:35.025785  8180 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:35.026528  8280 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:34.989714  1618 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:35.020736  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:34.973901  8297 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:34.944412  8304 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:34.982252  2203 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:35.028463  1612 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
W20260709 13:02:35.027755  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 44927us
I20260709 13:02:34.962208  8212 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 13:02:35.033653  1611 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:35.034106  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:35.038951  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:35.047703  1612 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:34.966789  8249 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:35.040338  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 58127.5us
I20260709 13:02:35.043020  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:35.051199  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:35.057542  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:35.058802  1617 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:35.059283  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:35.059691  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:35.060863  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:35.061592  8288 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:35.062318  8288 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:35.063232  8207 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:35.063607  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:35.064349  8207 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:35.064870  8207 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:35.064764  8273 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:35.065733  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.066073  8273 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:35.066529  8273 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:35.065970  8207 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:35.067304  8273 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.068879  8273 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:35.069970  8297 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:35.026618  8237 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:35.072003  8237 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:35.054225  1618 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:35.060807  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:35.053316  1584 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:35.013289  8098 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:35.080363  1583 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:35.082050  1617 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:02:34.972503  8226 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.078440  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:35.076387  1611 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:35.079384  8192 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:35.083953  7743 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:35.084596  8192 consensus_queue.cc:1059] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:35.085242  7743 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:35.085668  7743 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:35.086524  7743 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.087797  8226 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:35.082141  2203 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:35.090759  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:35.093518  1583 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:35.085686  8192 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:35.094693  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:35.095786  1583 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:35.046422  8214 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:35.047580  8170 raft_consensus.cc:2764] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 13:02:35.098205  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.101858  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.099725  8192 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:35.099050  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.103514  2234 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:35.103372  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.104316  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:35.107450  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:35.103814  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:35.102929  8025 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:35.109766  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:35.089246  8131 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:35.080521  8301 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:35.087886  7743 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:35.092533  8285 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:35.093657  8226 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.115574  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.105682  8282 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:35.109205  8288 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:35.102769  8274 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:35.094347  8310 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:35.105981  8098 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.122781  2234 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:35.077943  8249 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.112201  8084 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:35.114785  8131 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.133626  8131 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:35.126396  8282 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.130759  8241 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:02:35.131392  8098 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.133093  8084 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 18 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:35.114490  8094 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:02:35.129493  8192 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.146173  8192 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:35.125938  8226 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.148667  8094 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:35.150306  8094 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:35.152806  8226 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:35.166644  8146 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:35.168669  8288 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:02:35.153360  8252 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 13:02:35.154541  8275 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:35.155753  8202 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:35.171355  8249 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:35.155776  8282 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:35.177857  8275 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:35.177933  8286 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:35.178826  8286 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:02:35.180171  8281 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.986s	user 0.005s	sys 0.000s
I20260709 13:02:35.179749  8286 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:35.181079  8286 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:02:35.225526  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.227862  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.231113  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.231817  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.231324  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:35.233356  1542 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:35.235661  1610 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:35.301170  1610 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:35.302981  1610 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:35.237603  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:35.238821  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:35.240818  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:35.241390  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:35.243379  1613 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2717) took 2460 ms (client timeout 2070 ms). Trace:
I20260709 13:02:35.244846  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:35.246546  1618 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 13:02:35.246568  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 135144us
I20260709 13:02:35.249951  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:35.255523  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:35.258564  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:35.260278  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:35.262318  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.264279  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3217) took 2431 ms. Trace:
I20260709 13:02:35.265151  1614 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
W20260709 13:02:35.278797  2228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1618) took 1781 ms (client timeout 2177 ms). Trace:
I20260709 13:02:35.279558  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:35.280485  2231 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:35.280699  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1444) took 1853 ms. Trace:
I20260709 13:02:35.281248  2232 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:35.281884  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:35.282866  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:35.284003  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.291522  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:35.299836  6976 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:35.300323  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:35.234088  1609 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:35.319293  1541 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:35.326926  8226 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:35.328132  8226 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.328583  8243 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:35.329808  8243 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.330060  7980 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:02:35.332490  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 88086.5us
I20260709 13:02:35.334061  8065 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:35.334633  8065 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:35.338920  8240 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:35.340042  8240 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.340873  8243 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:35.334005  1509 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:35.312731  2228 rpcz_store.cc:269] 0709 13:02:33.496748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.496863 (+   115us) service_pool.cc:224] Handling call
0709 13:02:35.278184 (+1781321us) spinlock_profiling.cc:243] Waited 996 ms on lock 0x7b5400741154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:35.278502 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2290935680}
I20260709 13:02:35.151119  8094 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.354645  2228 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:35.356235  8094 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:35.358822  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.358742  8303 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:35.172024  8202 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:35.367055  8238 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.366030  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:35.369066  7993 consensus_meta.cc:287] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.564s	user 0.013s	sys 0.000s
I20260709 13:02:35.426234  7993 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.427980  2236 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:02:35.371269  8240 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:35.366495  8180 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:35.429173  8180 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.430286  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.430588  7993 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:35.432178  8180 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:35.384032  8062 raft_consensus.cc:3055] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:35.385703  8084 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.387595  8226 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:35.447042  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.431352  7753 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.390306  2233 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:35.460754  8214 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:02:35.461764  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.462327  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.464226  2233 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:35.391224  1612 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:35.396660  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.397526  1538 raft_consensus.cc:2393] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:02:35.398260  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64577.3us
I20260709 13:02:35.398859  8243 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:35.465723  8062 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:35.466331  8243 raft_consensus.cc:3055] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:35.393993  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:35.466553  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:35.336110  8065 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.467510  1538 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:35.405821  8131 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.467682  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:35.469375  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:35.469921  8131 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:35.410002  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.407109  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:35.471136  1509 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:35.471715  1611 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:35.471837  8090 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:35.472368  8090 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:35.472673  8245 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:35.473606  8090 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.367996  8238 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.405884  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:35.476722  1615 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:35.346205  8098 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:35.477982  8303 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:35.478550  8303 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:35.479435  8238 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:35.312335  1687 rpcz_store.cc:276] 0709 13:02:32.832597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.166224 (+333627us) service_pool.cc:224] Handling call
0709 13:02:33.166531 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.571044 (+1404513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.263609 (+692565us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b540003e154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:35.264064 (+   455us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2918056192}
I20260709 13:02:35.415490  1618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:40914
I20260709 13:02:35.482452  1618 raft_consensus.cc:606] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Received request to transfer leadership
I20260709 13:02:35.483040  1610 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:35.487776  8065 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:35.421862  1543 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:35.434707  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:35.455427  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:35.310750  1613 rpcz_store.cc:269] 0709 13:02:32.783148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.204714 (+421566us) service_pool.cc:224] Handling call
0709 13:02:35.242728 (+2038014us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54002dbed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:35.243018 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3085532928}
I20260709 13:02:35.439481  7926 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:35.444347  7883 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:35.475267  8090 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:35.509049  2235 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
W20260709 13:02:35.432862  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.388417  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 80893.6us
I20260709 13:02:35.466825  8243 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:35.469198  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:35.465261  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:35.514426  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:35.515360  1541 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:35.410936  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:35.516932  1539 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:35.366191  8150 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:35.412757  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:35.519085  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 106666us
I20260709 13:02:35.520246  1657 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:35.526077  8098 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:35.432205  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.506124  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:35.459012  8236 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:35.506645  1537 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:35.508307  1611 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:35.414548  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:35.313482  1690 rpcz_store.cc:276] 0709 13:02:33.427709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:33.592746 (+165037us) service_pool.cc:224] Handling call
0709 13:02:33.593185 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:34.843182 (+1249997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:35.280102 (+436920us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540028ebd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:35.280479 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2680389376}
I20260709 13:02:35.521239  8176 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:35.540149  1538 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:35.557542  1615 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:35.545766  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:35.553952  1610 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:02:35.561614  2230 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:35.551069  8131 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:35.564476  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.558166  1617 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:35.560642  8180 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:35.570746  1539 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:35.573609  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.572510  8009 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 13:02:35.543370  8226 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
W20260709 13:02:35.577350  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 163128us
I20260709 13:02:35.578140  1543 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:35.467070  8062 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:35.595948  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 68958.5us
I20260709 13:02:35.599278  8133 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.601373  1507 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:35.604490  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:35.605611  8202 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.605906  8249 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:35.606765  8297 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:35.607060  8249 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:35.607636  8249 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:35.601526  8187 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:02:35.606459  8202 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:35.609962  7753 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:35.610013  7980 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.611409  7980 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.614452  8080 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:35.615015  8243 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.615526  7753 raft_consensus.cc:3055] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:35.616199  7753 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:35.616542  8243 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:35.573010  7993 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
W20260709 13:02:35.623616  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 323527us
I20260709 13:02:35.616557  8192 raft_consensus.cc:993] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262: : Instructing follower cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 13:02:35.631896  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:35.637681  7993 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:35.601933  8133 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.638191  7993 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:35.641373  8202 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:35.642668  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:35.642892  8133 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:35.586534  8131 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.637087  8250 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:35.623248  8245 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:35.641391  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:35.643558  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:35.658584  1685 raft_consensus.cc:2410] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:02:35.665302  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:35.669543  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:35.670002  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:35.670357  1537 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:35.671901  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.672425  8287 raft_consensus.cc:1081] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Signalling peer cecad4f86a114a7696e398b3e8cc511e to start an election
I20260709 13:02:35.673832  8202 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:02:35.673931  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 78786.2us
I20260709 13:02:35.674464  8202 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.675001  8202 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:35.675123  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:35.646142  8180 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:35.678138  8180 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:35.679457  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.680171  2236 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:35.679863  7993 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.678238  8094 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:35.680462  2234 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:35.654830  8226 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:35.686730  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:35.687937  1582 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:35.690552  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 47977.3us
I20260709 13:02:35.671909  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:35.695451  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 48075.8us
I20260709 13:02:35.688896  7993 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:35.702907  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.692900  8192 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.711122  8075 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:35.711167  8192 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:35.707892  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.707985  1534 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:35.715001  8287 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:02:35.734191  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:35.734899  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.698256  7753 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.707273  8131 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:35.737309  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 60001us
I20260709 13:02:35.742282  2237 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:02:35.709020  8297 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 13:02:35.742683  1541 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
W20260709 13:02:35.749523  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 77797.8us
I20260709 13:02:35.760406  2235 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:02:35.764294  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 56572.1us
I20260709 13:02:35.749853  8180 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:35.765799  8287 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:02:35.770273  8192 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:35.770273  7753 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:35.778055  8226 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:35.778455  8250 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:35.779067  8286 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:35.781543  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:35.782256  8322 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.172s	user 0.004s	sys 0.000s
I20260709 13:02:35.784907  1610 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:35.786046  1617 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:35.787568  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:35.811630  1613 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:35.788225  8131 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:35.757337  7980 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:35.815152  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 102307us
W20260709 13:02:35.816009  8318 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 13:02:35.817092  7658 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:35.796396  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:35.797216  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:35.820320  1541 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 14.
I20260709 13:02:35.820432  1535 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:35.821487  1584 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:35.822503  8287 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:35.822961  8287 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.823333  8287 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:35.799180  8062 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:35.800424  2235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
 from {username='slave'} at 127.0.0.1:49846
I20260709 13:02:35.826495  2235 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:35.826994  2235 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:35.802686  8245 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:02:35.805699  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 45426.1us
I20260709 13:02:35.813714  8180 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:35.819361  7658 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:35.829600  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:35.815073  1611 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:35.825130  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:35.834395  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:35.811734  8250 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:35.835951  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:35.829994  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:35.834853  7658 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:35.844352  1610 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:35.831377  8237 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.854223  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:35.858029  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 44377us
I20260709 13:02:35.861760  1613 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:35.859622  8192 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:02:35.871428  8146 consensus_meta.cc:287] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.682s	user 0.014s	sys 0.002s
I20260709 13:02:35.873610  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:35.874912  8065 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:35.875180  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:35.875658  8065 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:35.876598  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:35.877888  7980 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:35.878619  7980 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:35.879253  8286 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:35.879588  8176 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:35.879761  8286 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:35.879829  7980 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:35.881635  8098 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:35.883574  8098 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:35.883733  8180 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:35.884209  8098 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:35.887040  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:35.882107  1618 raft_consensus.cc:1240] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:35.848726  8226 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:35.891650  8226 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:35.892618  1617 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:35.898341  1610 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:35.847442  8131 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:35.903558  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:35.904970  1615 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:35.851483  8090 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:35.908994  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.908591  7883 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:35.920612  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.915445  8098 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:35.920807  7883 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.921794  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:35.922798  7743 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:35.920660  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:35.924217  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 105208us
I20260709 13:02:35.905718  8131 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:35.926218  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.924964  1611 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:35.877025  8065 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:35.934379  2235 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.935067  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:35.861297  8303 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.917459  8207 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.939469  1583 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:35.940413  1613 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:35.944075  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:35.860061  8249 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:35.950973  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:35.952076  1687 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:35.930637  8131 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:35.955765  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:35.952824  1685 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:35.897681  8237 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:35.967026  2235 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:35.941126  7743 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:35.974538  1684 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:35.946255  8009 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:35.975162  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:35.980815  1615 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: []
W20260709 13:02:35.983145  1657 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.177s (SENT) [suppressed 37 similar messages]
I20260709 13:02:36.004958  2234 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
W20260709 13:02:36.043969  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:35.968125  8207 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.059656  1613 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:36.040867  8202 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.067294  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.070839  1691 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:02:36.073714  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 90599.8us
W20260709 13:02:36.077616  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.079282  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 140837us
I20260709 13:02:36.080622  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.081267  1610 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:36.046309  7743 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.091569  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.083261  8233 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:36.096537  1611 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:36.097797  8330 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.098265  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:36.102494  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:36.105576  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.109452  8338 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
W20260709 13:02:36.110994  8324 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.265s	user 0.004s	sys 0.000s
I20260709 13:02:36.098593  8303 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:36.117751  8207 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:36.118153  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.119185  8246 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:36.119730  8246 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:36.102032  8330 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.122774  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.131866  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.123662  1540 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:36.135378  8330 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.136543  8187 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:02:36.136648  8326 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 13:02:36.140899  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:36.143774  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:36.144629  1610 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:36.145748  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:36.146963  1614 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:36.148451  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.149681  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.152853  1612 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:36.154053  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:36.137064  8187 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:36.156284  8187 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.156739  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:36.157209  8237 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:36.157656  8187 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:36.157568  8329 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:02:36.158527  8243 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:36.158607  8237 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:36.159065  8329 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:36.159477  8245 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:36.159261  8243 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.159942  8245 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:36.160931  8245 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.161525  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.162168  8245 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:36.161545  7743 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.163085  7743 consensus_queue.cc:1059] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:36.164413  8303 raft_consensus.cc:3055] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:36.164813  8303 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:36.166095  8237 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:36.149168  1613 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:36.159654  8329 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:36.098600  8233 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:36.188575  1687 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
I20260709 13:02:36.195835  1687 pending_rounds.cc:85] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 9
I20260709 13:02:36.191169  1611 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:36.193940  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.120786  8246 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:36.194568  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.201901  1535 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:36.194612  8243 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:36.202471  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 45831.2us
I20260709 13:02:36.203476  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:36.205960  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 108939us
I20260709 13:02:36.206126  8329 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:36.207230  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 130839us
I20260709 13:02:36.207516  7744 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:36.208141  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:36.208585  8330 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:36.209152  8330 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:36.208827  7744 consensus_queue.cc:1059] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:36.209523  1610 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 13:02:36.209683  8330 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.193161  1617 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:36.211587  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:36.199509  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:36.212906  1541 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:36.117862  7883 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:36.200613  1582 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.210897  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.219444  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:36.220188  1686 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:36.196195  1687 pending_rounds.cc:107] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:02:36.222851  1687 raft_consensus.cc:2889] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:36.175690  8249 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.228816  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.231343  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:36.226037  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.235620  1509 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.237558  8025 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:36.238250  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 1385) took 1903 ms. Trace:
I20260709 13:02:36.237957  8303 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.238759  1609 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 13:02:36.239135  8325 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.307s	user 0.001s	sys 0.001s
I20260709 13:02:36.238706  1540 rpcz_store.cc:276] 0709 13:02:34.335313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:34.337931 (+  2618us) service_pool.cc:224] Handling call
0709 13:02:36.237994 (+1900063us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b540030a7d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:36.238170 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3779843456}
I20260709 13:02:36.239842  8303 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:36.240062  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:36.240924  1609 raft_consensus.cc:2393] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:36.237854  8009 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:36.246026  1584 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:36.238035  8025 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:36.246165  7743 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:36.247387  8133 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:36.247303  8025 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.247917  8133 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.248364  8133 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:36.248694  8025 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.249836  1542 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:36.252067  6976 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:36.255000  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.230163  1690 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
W20260709 13:02:36.257333  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 53609.1us
I20260709 13:02:36.217280  8084 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.225226  7980 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:36.266878  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:36.275368  6976 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.282450  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.283563  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.289546  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.290441  1617 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:36.297580  8062 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:36.218740  8243 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 13:02:36.298949  7980 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:36.302241  8207 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:36.306141  1540 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc"
dest_uuid: "83f6f5810950417195f707ff308a8125"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:50578
I20260709 13:02:36.306856  1540 raft_consensus.cc:606] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Received request to transfer leadership
I20260709 13:02:36.306335  8207 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.310214  8329 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.310855  8329 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.311952  7658 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.259507  7883 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 13:02:36.274609  8249 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:36.314736  8249 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:36.315171  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:36.315433  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.307195  1540 raft_consensus.cc:614] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:36.315943  1613 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:36.313277  8329 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.315373  8249 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:36.306181  7980 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:36.274187  8297 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.298185  8062 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:36.322628  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 105846us
I20260709 13:02:36.322703  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:02:36.324159  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 116455us
I20260709 13:02:36.326838  1509 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:02:36.250587  8192 consensus_queue.cc:1175] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "984c9089d9804652846f3caeb609d56e" responder_term: 14 status { last_received { term: 14 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 14 index: 9 } }
I20260709 13:02:36.330466  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.337674  1582 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.344236  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:36.392433  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.302929  8065 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:36.390916  7743 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 13:02:36.397081  8009 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.397529  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.398959  8328 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:36.299887  8084 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.401593  8297 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:36.402849  8250 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.554s	user 0.005s	sys 0.005s
I20260709 13:02:36.405521  7883 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:36.406325  8237 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:36.409973  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 72440.3us
I20260709 13:02:36.410604  8238 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:36.413233  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 70792.8us
W20260709 13:02:36.350392  7993 consensus_meta.cc:287] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.635s	user 0.012s	sys 0.000s
I20260709 13:02:36.419050  1535 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:36.339040  8062 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:36.419939  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.323236  8207 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:36.423480  1686 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:36.434933  8328 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:36.411450  8330 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.340498  8329 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:36.491626  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.462318  8098 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:36.478312  8187 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.466281  8250 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.529850  8187 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.531798  8250 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:36.447072  8009 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.534032  8009 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:36.502449  8287 consensus_meta.cc:287] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.679s	user 0.012s	sys 0.000s
I20260709 13:02:36.490882  8328 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:36.536132  8131 consensus_meta.cc:287] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.568s	user 0.009s	sys 0.001s
I20260709 13:02:36.536580  8131 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.419559  8233 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.537281  6976 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.480654  8245 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.538204  8245 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.538877  8131 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.539487  8249 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.509804  1657 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.491317  8329 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.540230  8287 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.506994  8337 raft_consensus.cc:3055] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:36.541200  8176 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:36.541769  8098 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.541304  8329 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:36.544663  8341 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.286s	user 0.000s	sys 0.003s
W20260709 13:02:36.510169  8344 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.211s	user 0.003s	sys 0.000s
I20260709 13:02:36.509850  8330 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:36.515193  8339 log.cc:927] Time spent T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:02:36.511304  8333 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 13:02:36.526304  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:36.498826  8297 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.553787  8207 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:36.554582  8207 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.526935  8214 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:36.540242  8286 consensus_meta.cc:287] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.660s	user 0.013s	sys 0.003s
I20260709 13:02:36.554944  8207 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.558910  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:36.542147  8337 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:36.521579  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.531929  8187 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.559759  1685 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:36.546983  1438 catalog_manager.cc:5697] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 8 to 10, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 10 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:02:36.534734  8238 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:36.578465  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:36.562451  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 52836.7us
I20260709 13:02:36.626096  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.576946  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:36.570063  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.635262  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.485237  8202 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:36.639891  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.570472  8084 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:36.636052  1683 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:36.646531  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:36.642771  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.573699  8287 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.556694  7658 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.639348  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.561239  8098 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:36.642730  1690 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:36.647060  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.647593  1686 raft_consensus.cc:2410] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 13:02:36.563961  8245 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:36.650683  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:36.563083  1434 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40172 (request call id 234) took 17239 ms (client timeout 15000 ms). Trace:
I20260709 13:02:36.556979  8233 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:36.559831  8133 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.562071  8176 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.089s
W20260709 13:02:36.673733  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 103747us
I20260709 13:02:36.557096  8286 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.645810  8250 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:36.648306  8238 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.675323  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.683789  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.651976  8131 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.688409  8098 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:36.688792  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 47610.7us
I20260709 13:02:36.690104  1657 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.556790  8214 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.684180  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:36.687240  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:36.687688  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.681493  1438 catalog_manager.cc:5697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:02:36.690438  8337 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.697678  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:36.698613  8202 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:36.698779  8238 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.699697  1438 catalog_manager.cc:5697] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:02:36.701824  8250 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:36.703480  8286 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:36.703902  8250 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.704185  8297 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:36.704455  8250 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:36.706641  8214 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.708453  2232 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:36.702183  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.688444  8131 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.710194  1688 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:36.712145  8131 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:36.702857  8350 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:02:36.674130  8025 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:36.695525  8329 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.701448  1438 catalog_manager.cc:5697] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:02:36.715629  8329 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.717902  1541 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:02:36.670562  1434 rpcz_store.cc:269] 0709 13:02:19.323824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:19.324120 (+   296us) service_pool.cc:224] Handling call
0709 13:02:36.562821 (+17238701us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:29.375327 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:29.376361 (+  1034us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:29.376440 (+    79us) write_op.cc:435] Acquired schema lock
0709 13:02:29.376469 (+    29us) tablet.cc:662] Decoding operations
0709 13:02:29.438199 (+ 61730us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:29.438313 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:29.438340 (+    27us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:02:29.440330 (+  1990us) tablet.cc:701] Row locks acquired
0709 13:02:29.440400 (+    70us) write_op.cc:260] PREPARE: finished
0709 13:02:29.440638 (+   238us) write_op.cc:270] Start()
0709 13:02:29.440801 (+   163us) write_op.cc:276] Timestamp: P: 1783602149440559 usec, L: 0
0709 13:02:29.440833 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:02:29.451488 (+ 10655us) log.cc:844] Serialized 8319 byte log entry
0709 13:02:30.510304 (+1058816us) op_driver.cc:464] REPLICATION: finished
0709 13:02:30.876295 (+365991us) write_op.cc:301] APPLY: starting
0709 13:02:30.876407 (+   112us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:32.420145 (+1543738us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:32.455682 (+ 35537us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:33.952773 (+1497091us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:33.953178 (+   405us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:02:33.953240 (+    62us) write_op.cc:312] APPLY: finished
0709 13:02:33.953789 (+   549us) log.cc:844] Serialized 188 byte log entry
0709 13:02:34.105401 (+151612us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:34.136078 (+ 30677us) write_op.cc:454] Released schema lock
0709 13:02:34.186548 (+ 50470us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:34.238937 (+ 52389us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:34.247584 (+  8647us) write_op.cc:454] Released schema lock
0709 13:02:35.174740 (+927156us) spinlock_profiling.cc:243] Waited 4.85 ms on lock 0x7b18000b3aa0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9cd2a6 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db
0709 13:02:35.179691 (+  4951us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
Metrics: {"mutex_wait_us":952453,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":365644,"apply.run_cpu_time_us":66130,"apply.run_wall_time_us":3346177,"cfile_cache_hit":86,"cfile_cache_hit_bytes":92033,"num_ops":21,"prepare.queue_time_us":351005,"prepare.run_cpu_time_us":53268,"prepare.run_wall_time_us":279677,"raft.queue_time_us":331285,"raft.run_cpu_time_us":40789,"raft.run_wall_time_us":235340,"replication_time_us":1069249,"spinlock_wait_cycles":17811072,"thread_start_us":985404,"threads_started":5,"wal-append.queue_time_us":414767,"wal-append.run_cpu_time_us":36936,"wal-append.run_wall_time_us":3705774}]]}
I20260709 13:02:36.718844  8133 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.721848  1508 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:36.721990  1616 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:36.701072  8238 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.723794  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:36.715644  1438 catalog_manager.cc:5697] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 10 to 16, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 16 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:36.703192  1507 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:36.734400  1507 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.692220  8245 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:36.730584  8328 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.738356  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:36.769186  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.769547  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:36.769124  1657 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.822639  8237 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:36.800489  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 66161.2us
W20260709 13:02:36.773192  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:36.825256  8131 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.825759  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:36.825840  8131 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.827437  1538 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:36.827706  8303 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.828189  8131 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:36.828351  8303 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.830348  1618 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:36.830825  8303 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:36.831409  8303 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.831724  1612 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:36.831907  8303 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.833588  1536 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:02:36.833818  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.834424  1615 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:36.834587  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:36.835168  1536 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:36.835290  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:36.835566  8287 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.836005  2230 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:36.836097  8287 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.836452  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.837239  1613 raft_consensus.cc:2393] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:36.838136  8009 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.838387  8287 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:36.838577  1617 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:36.838680  8009 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:36.840518  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:36.840940  8009 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:36.837109  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:36.841681  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:36.841933  1538 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:36.842295  1537 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.843204  1538 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:36.844074  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:36.844677  1613 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.845546  8009 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:36.845932  8009 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.846289  8009 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:36.841629  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.847165  1612 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:36.847920  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.848277  1612 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:36.848764  1617 raft_consensus.cc:2393] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:36.843323  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:36.843962  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:36.849880  2230 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:36.845273  1538 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:36.821935  1685 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:36.853901  1541 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:02:36.854669  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:36.863656  1537 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:36.867185  1540 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:02:36.696592  8065 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:36.855480  2232 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:36.720680  8250 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:36.858529  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:36.852326  1438 catalog_manager.cc:5697] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 10 to 12, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 12 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:36.875631  1610 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:02:36.898204  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 55364.3us
I20260709 13:02:36.904160  1609 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:36.908618  1541 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:36.913102  1538 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:36.914125  7883 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.914729  7883 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:36.917047  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 58580us
W20260709 13:02:36.918447  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 451454us
I20260709 13:02:36.917816  1438 catalog_manager.cc:5697] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 6 to 10, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 10 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:02:36.921751  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:36.922256  1616 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:36.921259  1613 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:36.927049  1507 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:36.739323  8187 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:36.736984  8354 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:36.938050  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:36.961126  1540 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:36.944679  1535 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 13:02:36.868602  8237 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:36.960541  1542 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:36.939666  8297 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:36.951702  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:36.966521  1538 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:36.917193  7883 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:36.731547  8098 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.201s
I20260709 13:02:36.963792  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:36.961228  1582 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:36.948017  7743 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:36.948020  8306 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:36.946275  8249 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:36.974594  1537 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:36.980520  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:36.967973  8354 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:36.981006  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:37.015892  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.026584  8306 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:37.040421  8062 consensus_meta.cc:287] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.508s	user 0.006s	sys 0.003s
I20260709 13:02:37.040933  8062 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.032310  1583 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:37.041846  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:37.042490  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.043118  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:37.022926  8098 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:37.045288  1438 catalog_manager.cc:5697] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 11 to 14, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 14 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:02:37.052793  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.043772  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 99099.8us
I20260709 13:02:37.044564  2230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:37.054497  2230 raft_consensus.cc:606] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:37.047256  1536 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:36.953023  8176 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:02:37.050130  1684 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:37.051407  6976 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.012388  8352 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:02:37.056983  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:36.970057  8297 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:37.075431  1428 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab5b971190d342d2b2568c30f3c0c81f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8a6a5ae1b7f240408b3ddd30d8a18262 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:37.076740  6976 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.009332  8238 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:37.076337  1538 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:37.011555  8329 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.082050  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 435561us
W20260709 13:02:37.050463  8349 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:02:37.089540  1543 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:37.041290  8306 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.091188  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:37.093835  1688 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
W20260709 13:02:37.106985  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.042754  8354 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:37.114678  1691 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:37.051311  8303 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.128863  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:37.129616  1686 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:37.110538  1542 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:37.106773  1438 catalog_manager.cc:5697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 11 to 12, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 12 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:37.008100  7980 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:02:37.079828  8297 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:37.122452  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:37.131105  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:37.131835  1538 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:36.971407  8237 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.120061  7743 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:37.117050  8131 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.141974  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.143419  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:37.143929  1691 raft_consensus.cc:2410] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:02:37.144582  8363 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.200s
W20260709 13:02:37.145027  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 376010us
I20260709 13:02:37.144871  8131 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.144616  1692 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:37.146160  6976 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.147799  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.148458  1688 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:37.150647  1438 catalog_manager.cc:5697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 7 to 8, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 8 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:02:37.099536  8329 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.134229  8355 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:37.157703  8356 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.219s
I20260709 13:02:37.167665  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:37.160125  8357 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.170873  2230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:37.170674  8357 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.171511  2230 raft_consensus.cc:606] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:02:37.171902  2230 raft_consensus.cc:614] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:37.172032  8098 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:37.169790  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:37.173632  1535 raft_consensus.cc:2410] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:02:37.159823  8250 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.161844  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:37.175935  1542 raft_consensus.cc:2410] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 13:02:37.162575  8328 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.181516  1584 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:37.182780  8337 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.183768  8192 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:37.184219  8192 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.184576  8192 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:37.149410  8150 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:37.173061  8357 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.134477  8358 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:37.189868  8358 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.191092  1683 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:37.165922  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:37.193805  1538 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:37.196619  1657 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:37.197759  1537 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:37.206372  1540 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:37.140946  7743 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.213310  7743 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:37.214077  8369 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:02:37.214938  1584 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:37.215844  1584 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:37.217967  8303 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:37.218623  8303 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:37.218746  6976 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:37.219249  6976 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.192478  8358 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:37.219691  6976 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.220073  8358 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:37.220186  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:37.220614  8358 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:37.207724  8345 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.221241  8133 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:37.221717  8133 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.222281  8133 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.170027  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.223088  8329 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.223421  1689 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:37.221994  8237 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.194816  8176 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:37.224640  8176 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.225061  8176 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.211714  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:37.225808  2237 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:37.219714  8303 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:37.164721  8207 consensus_meta.cc:287] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.607s	user 0.008s	sys 0.000s
I20260709 13:02:37.221728  8345 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.211714  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.087982  8065 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.208390  1438 catalog_manager.cc:5697] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 9 to 11, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 11 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:37.229768  8207 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.244935  8306 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.245096  8356 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.245669  8207 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:37.246342  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 160172us
I20260709 13:02:37.246309  8356 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.247386  8238 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:37.247694  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:37.248824  8356 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.249626  8150 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:37.250082  8150 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:37.250453  8150 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:37.253641  1687 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:37.255307  8009 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.257015  1615 raft_consensus.cc:2368] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19.
I20260709 13:02:37.258074  8131 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.194581  8355 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.258788  8355 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.259934  8009 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.259608  8192 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.274413  1438 catalog_manager.cc:5697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:02:37.275815  8192 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.276116  1438 catalog_manager.cc:5697] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 5 to 8, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 8 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:02:37.263217  8187 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:37.265626  6976 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.221764  8358 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.282203  2237 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:37.283126  6976 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.266463  2232 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40336
W20260709 13:02:37.283648  8202 consensus_meta.cc:287] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.584s	user 0.003s	sys 0.007s
I20260709 13:02:37.283890  2232 raft_consensus.cc:606] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:02:37.284127  8202 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:37.244967  8345 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.257088  8062 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.270757  8246 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.285600  8246 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.245852  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:37.273734  1584 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:37.286932  6976 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:37.287269  6976 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.284969  1686 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:37.287801  6976 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:37.287139  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:37.290166  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:37.291728  2232 raft_consensus.cc:3055] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:37.292230  2232 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:37.319660  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:37.326398  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:37.327564  1543 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:37.337769  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 305548us
I20260709 13:02:37.346578  1438 catalog_manager.cc:5697] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 11 to 12, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 12 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:37.330506  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:37.350160  1539 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.294059  8009 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:37.357249  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:37.357288  6976 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.360306  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:37.361260  2237 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:37.361593  2230 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:37.360558  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 277524us
I20260709 13:02:37.328879  2232 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.317085  8192 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:37.245729  8306 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.364096  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:37.369257  1508 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:37.371129  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:37.371763  2232 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.376569  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.366314  2234 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:37.373713  1508 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:37.319422  8233 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:37.386037  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.419065  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:37.419103  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:37.423491  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 59491.1us
I20260709 13:02:37.424525  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:37.421522  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:37.420904  1508 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:37.428232  6976 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.429011  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:37.429327  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:37.429786  1438 catalog_manager.cc:5697] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:37.426249  1687 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:37.425871  2236 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:37.432327  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:37.432924  2234 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.377893  8192 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.437616  2235 raft_consensus.cc:2410] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:37.439590  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:37.440414  1609 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:37.375831  8338 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:37.374974  8202 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.382372  8187 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:37.451023  1613 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:37.447047  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:37.453842  1618 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:37.440768  1438 catalog_manager.cc:5697] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:37.442710  1616 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 17: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:37.444144  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:37.445627  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:37.446064  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:37.374205  8009 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:37.422158  8303 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.438558  8363 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:02:37.427866  8237 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:37.463891  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:37.365322  7743 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.464632  8356 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.441675  8329 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:37.496419  8389 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:37.497030  8329 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.497502  8329 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:37.497623  8389 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:37.497686  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:37.469439  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.498263  8389 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.498739  8389 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:37.471451  8354 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.472455  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.472707  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.474324  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:37.474758  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:37.476058  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:37.387068  8098 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:02:37.478075  8090 raft_consensus.cc:993] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: : Instructing follower 984c9089d9804652846f3caeb609d56e to start an election
I20260709 13:02:37.477942  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:37.377558  8133 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.506922  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:37.507103  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:37.387648  8062 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.091s
I20260709 13:02:37.479995  8246 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.480669  6976 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.509307  1611 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:37.509287  6976 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.481577  8287 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:37.482043  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 165758us
I20260709 13:02:37.482537  8338 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:37.494364  8237 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:37.427866  8233 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.427721  8245 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.463641  2231 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "474d3b57c33542aba459b97733122bf0"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:37.489527  1582 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:37.493718  1543 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:37.496526  8356 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.471508  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.468029  8306 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.448814  8192 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:37.458652  8202 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.538496  1610 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:02:37.499914  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:37.540210  1616 raft_consensus.cc:2368] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20.
I20260709 13:02:37.502678  1688 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.501806  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:37.502202  1689 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.502456  1685 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:37.501212  8084 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:37.507855  1617 raft_consensus.cc:2410] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:02:37.507876  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: true dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:37.540927  2201 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:37.545183  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 120742us
I20260709 13:02:37.488025  1539 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:37.517199  8133 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.495079  8303 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.517946  8287 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.518524  8176 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.518885  8297 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.509135  7980 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:37.520715  2203 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:37.522815  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:37.540038  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:37.459002  8187 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:37.563505  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 74118.4us
I20260709 13:02:37.525254  1611 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:37.541813  2231 raft_consensus.cc:606] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:02:37.500978  8328 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:37.541718  1618 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:37.549834  1610 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:37.566129  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:37.566651  2236 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:37.567118  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:37.527209  8237 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:37.551532  8238 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.513828  7744 raft_consensus.cc:1081] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Signalling peer 984c9089d9804652846f3caeb609d56e to start an election
I20260709 13:02:37.559173  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.562285  1540 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.565100  2231 raft_consensus.cc:614] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:37.568292  1612 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:37.550472  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:37.575915  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:37.553172  8084 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.576148  1613 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:37.576606  1537 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:37.525514  8379 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:02:37.579216  1541 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:37.498929  8009 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.581221  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:37.583756  1692 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:37.562988  8334 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:37.578811  8238 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.567584  7980 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:37.549959  8245 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.587608  1582 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:37.597839  8176 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:37.601262  8358 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:02:37.601589  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.601805  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 81145.5us
I20260709 13:02:37.602911  8238 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.603349  8084 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:37.603454  8345 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:37.604702  8345 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:37.605991  7744 raft_consensus.cc:3055] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:37.606438  7744 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:37.606662  8345 raft_consensus.cc:993] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: : Instructing follower 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 13:02:37.607184  8345 raft_consensus.cc:1081] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Signalling peer 8a6a5ae1b7f240408b3ddd30d8a18262 to start an election
I20260709 13:02:37.607391  7744 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.608791  7744 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:37.569887  8384 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:37.609699  1683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40014
I20260709 13:02:37.609954  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:37.610280  1683 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:37.610687  1683 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:37.579335  8302 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:37.610706  2233 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:37.571552  8385 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:37.609925  1685 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:37.586747  8363 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:02:37.613214  8363 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:37.613626  8363 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:37.614387  2235 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 13:02:37.614840  2235 pending_rounds.cc:85] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 10
I20260709 13:02:37.614506  8363 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.609715  8384 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.615721  8384 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:37.615844  8363 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:37.570050  8187 raft_consensus.cc:3060] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:37.621212  2232 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:37.613626  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:37.622148  2237 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:37.623127  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:37.623903  2228 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:37.594086  8334 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.624460  8334 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:37.615108  2235 pending_rounds.cc:107] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:02:37.626071  2235 raft_consensus.cc:2889] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:37.626345  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:37.627933  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:37.628150  1609 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
W20260709 13:02:37.628415  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 47250.3us
I20260709 13:02:37.601820  8358 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:37.630367  8358 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:37.630457  7980 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.631887  7980 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:37.630482  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:37.636549  1540 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:37.634167  1688 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:37.623931  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:37.638248  1689 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:37.639031  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:37.639822  1582 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:37.639726  2230 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:37.641702  1614 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:37.641835  1508 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:37.584601  8391 raft_consensus.cc:2764] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:02:37.591606  8131 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:37.643113  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:37.650214  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:37.643970  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:37.651027  2232 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:37.645215  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:37.642360  8381 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:37.597913  8287 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.647110  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:37.661616  2237 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.662520  7743 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.667901  6976 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.668627  8237 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.648903  8250 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:37.649238  8133 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:37.681977  8133 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:37.682977  8133 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.683269  8150 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.649663  8387 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:02:37.686874  8237 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.689646  8356 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.690400  8389 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:37.702040  8390 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:02:37.702837  8303 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.535252  8338 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.571477  8328 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.646689  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:37.638432  8385 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:37.671077  8375 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:37.725927  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:37.650362  8357 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:37.672952  2231 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:02:37.673813  2233 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:37.671271  8384 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.674387  8176 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:37.640295  8084 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:02:37.681337  8250 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:37.703315  8374 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.061s
W20260709 13:02:37.711957  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:37.752712  1489 auto_rebalancer.cc:893] Removing replace marker failed: Replica 83f6f5810950417195f707ff308a8125 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:37.726552  8379 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:37.753271  1489 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 13:02:37.711908  2236 raft_consensus.cc:2468] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
W20260709 13:02:37.753535  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.757875  1658 leader_election.cc:400] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19.
I20260709 13:02:37.758370  1658 leader_election.cc:403] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:37.761044  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:37.761842  1540 raft_consensus.cc:2368] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:37.761674  1507 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:37.716989  2237 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:37.645215  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:37.722582  2228 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 15: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:02:37.772338  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 127219us
I20260709 13:02:37.773332  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:37.601312  8329 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.724248  2234 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:37.726542  2232 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:37.648411  8337 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:37.647998  8238 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:37.732368  2229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eed65fd1add844e7821095e2215fb393"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:37.785620  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:37.785944  2229 raft_consensus.cc:606] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:37.638164  8246 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 13:02:37.786422  2229 raft_consensus.cc:614] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:37.787199  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:37.725127  1683 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:37.734644  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 102176us
I20260709 13:02:37.744791  8176 raft_consensus.cc:697] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:37.810642  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:37.746435  8384 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:37.748037  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 124213us
I20260709 13:02:37.815493  1534 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:02:37.748546  8303 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:37.779054  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:37.784029  1538 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:02:37.819095  1683 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.819733  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:37.820261  1538 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 13:02:37.754523  1489 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:02:37.822207  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:37.822496  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.822952  1543 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:37.749145  8354 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.786402  1535 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:37.823374  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:37.725895  8375 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:37.823935  1688 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:37.733346  8062 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:37.807574  8355 consensus_meta.cc:287] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.548s	user 0.005s	sys 0.005s
I20260709 13:02:37.820343  2231 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:02:37.775240  8287 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.826762  1539 raft_consensus.cc:2368] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16.
I20260709 13:02:37.829257  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.831945  1690 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
W20260709 13:02:37.834797  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:37.836680  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:37.837471  2234 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:37.828227  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:37.838726  1692 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.829670  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:37.840329  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:37.835803  1583 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:37.841334  2228 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:37.783262  8237 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:37.785544  8387 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19.
I20260709 13:02:37.788158  8337 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:37.733340  8098 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 13:02:37.860347  2232 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:37.809876  8356 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:37.849776  1535 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:02:37.861512  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:37.862373  1610 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:37.852950  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:37.854794  1609 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:02:37.725436  8391 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.104s
I20260709 13:02:37.858979  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.859262  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:37.888417  1687 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:37.817020  8385 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:37.825342  8306 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:37.866706  1609 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:40914
I20260709 13:02:37.874706  6976 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.893471  1609 raft_consensus.cc:606] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Received request to transfer leadership
I20260709 13:02:37.876199  1688 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:37.893869  1609 raft_consensus.cc:614] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:37.878123  1618 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:37.878614  1616 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:37.894914  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:37.895604  1543 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:37.753343  8187 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.890105  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:37.865530  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:37.897073  1612 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:37.756842  8379 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.893643  6976 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:37.840618  8150 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:37.840699  8131 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:37.909456  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:37.911094  1611 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:37.912473  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:37.859686  8354 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:37.868129  8337 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:37.867255  8384 raft_consensus.cc:2804] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:02:37.924034  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 145111us
I20260709 13:02:37.924660  6976 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:37.823999  8238 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.924955  1509 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:37.753682  8250 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.908588  8356 raft_consensus.cc:697] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:37.840905  8062 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.888695  8389 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.927021  1536 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 23: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:37.929853  2234 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:37.908299  8233 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.910006  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:37.830888  8355 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.918581  8374 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:37.920198  1617 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:37.922699  8084 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:37.930800  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 169204us
I20260709 13:02:37.845533  8287 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.887194  8237 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.860126  8303 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.941962  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:37.943626  1610 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:37.975798  8385 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:37.991309  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 104763us
I20260709 13:02:38.006464  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:37.992808  1582 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:38.007215  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 97296.5us
I20260709 13:02:38.007705  6976 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.008262  6976 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.947937  8202 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.902093  8306 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.009622  1611 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:38.011456  1542 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:37.947438  8395 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:38.013039  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 52769us
I20260709 13:02:37.954962  8354 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:38.026542  6976 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:37.936167  8150 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:02:38.030215  8385 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.031416  6976 consensus_queue.cc:1059] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:02:38.034484  6976 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.005748  8065 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:38.035059  6976 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.032796   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:38.042893  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:38.048053  8233 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:38.048437  8354 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:38.050462  8233 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:38.051244  8233 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.051942  8328 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.051813  8233 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:37.928970  8375 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:37.994644  8090 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.040s
I20260709 13:02:38.013547  8245 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:38.013253  8384 raft_consensus.cc:697] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:37.959787  8374 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.004071  8084 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:38.046216  8394 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:38.040994  8065 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:38.045089   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:38.053362  2231 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:38.028280  7743 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.062989  8287 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.078848  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:38.055718  8098 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.059479  7744 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:38.061098  8192 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.065346  8357 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:38.133744  8357 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:38.134562  8410 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 13:02:38.134971  8363 consensus_meta.cc:287] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.519s	user 0.009s	sys 0.003s
I20260709 13:02:38.135613  7743 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.083595  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:38.017889  8133 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:38.057636  8238 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.170652  1692 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:38.171409  1534 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:38.029313  8395 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.020859  8406 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:38.121202  6976 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.124305  8355 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:38.126453  8358 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:38.116853  8246 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:38.180068  1684 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:37.957779  8176 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.185552  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:38.061910  8131 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:38.056401  8379 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.057210  8150 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:02:38.087491  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 145609us
I20260709 13:02:38.193105  1690 raft_consensus.cc:2368] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20.
I20260709 13:02:38.192622  1542 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 13:02:38.193475  1509 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:38.053840  8303 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.064509  8062 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.058269  8297 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:38.119068  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 126355us
I20260709 13:02:38.147037  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:38.058586  8237 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:38.179459  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.204814  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:38.136986  7744 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:38.183156  1583 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:38.211743  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:38.130100  8084 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:38.112663  8403 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.189217  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:38.133927  8381 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:38.191355  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:38.133280  8399 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:02:38.195726  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:38.236467  1691 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:38.198289  1543 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
W20260709 13:02:38.198912  1657 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.020s (SENT)
I20260709 13:02:38.170869  8375 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:38.204666  1539 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:38.206944  1685 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:38.179929  1683 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:38.211783  1509 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:38.187362  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.214128  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:38.215960  2234 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:38.241925  1535 raft_consensus.cc:2368] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20.
I20260709 13:02:38.216117  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.215538  2203 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:38.217489  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.180711  8395 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:38.230437  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:38.231655  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:38.188758  8133 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:38.120690  8329 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.080684  8356 consensus_queue.cc:237] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:38.255769  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.251554  1542 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:38.240808  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.245390  1538 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:38.210450  8150 raft_consensus.cc:697] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:38.260941  1509 leader_election.cc:400] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20.
I20260709 13:02:38.263913  1509 leader_election.cc:403] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:02:38.266155  1509 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:38.267099  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:38.268203  1689 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:38.269111  1686 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:38.192963  8246 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:38.136157  8357 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:38.222972  8287 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:38.176800  8098 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.270979  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:38.258311  8354 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:02:38.233371  8187 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:38.278285  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 62847.1us
I20260709 13:02:38.248617  8401 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:38.217900  8062 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:38.289130  6976 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.237872  8303 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:38.292073  6976 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.289408  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:38.293958  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:38.247718  8385 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:38.282770  8287 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.278420  8009 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.303630  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.278939  8355 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:38.286499  8237 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:02:38.310330  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.256110  8065 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.263365  8406 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:02:38.312649  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:38.312994  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:38.323992  2237 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:38.308022  8385 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:38.283666  8329 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:02:38.243815  8403 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.328420  2234 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:38.338249  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.338394  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:38.339473  2235 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:38.340655  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:38.383239  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:38.392452  1538 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:38.341557  8366 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:38.343961  1540 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.010s	sys 0.007s
W20260709 13:02:38.395803  1540 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.010s	sys 0.007s
W20260709 13:02:38.345517  8396 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.284s	user 0.000s	sys 0.002s
I20260709 13:02:38.305104  8420 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:38.347972  8133 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:38.298018  8401 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:38.262818  8384 consensus_queue.cc:237] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.352386  8414 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:38.352834  8192 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.353900  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:38.313915  8287 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:38.355814  8098 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.393390  8065 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:38.361786  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 162906us
I20260709 13:02:38.369730  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:38.307946  8303 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:38.373898  2228 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:38.326236  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.375425  2237 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:38.374861  8150 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.429818  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:38.430526  1657 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.177s (SENT)
I20260709 13:02:38.430732  1541 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 23: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:38.299456  7744 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.431799  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:38.432642  1534 raft_consensus.cc:2393] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:38.432591  7744 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:38.394375  2232 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:38.371271  8233 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.382840  8237 raft_consensus.cc:697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:38.341565  6976 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.438503  1507 leader_election.cc:400] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20.
I20260709 13:02:38.407855  2236 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:38.438511  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:38.335938  8009 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.382630  8306 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:38.414865  8401 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:38.416407  7980 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.784s	user 0.006s	sys 0.006s
I20260709 13:02:38.411902  2233 raft_consensus.cc:2376] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19. Re-sending same reply.
I20260709 13:02:38.421499  7743 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.397547  8374 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:38.426229  8408 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:02:38.426644  8400 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:38.428143  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:38.429508  8334 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.805s	user 0.005s	sys 0.008s
I20260709 13:02:38.399446  8358 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:38.434583  1508 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:38.435387  1688 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:02:38.436225  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.436414  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:38.429816  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:38.496409  1534 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:38.496636  1689 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:38.401602  8090 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 13:02:38.402779  8366 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:38.438799  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 113574us
I20260709 13:02:38.436179  2228 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:38.442333  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:38.443238  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:38.500304  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 65806.5us
I20260709 13:02:38.500753  1683 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:38.449332  1584 leader_election.cc:400] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16.
I20260709 13:02:38.450503  1657 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:38.501508  1584 leader_election.cc:403] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:02:38.501855  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 51495.3us
I20260709 13:02:38.459748  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:38.502310  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 55204us
I20260709 13:02:38.502964  1538 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
I20260709 13:02:38.504237  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:38.506618  1584 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:38.463011  8413 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:02:38.465837  1687 raft_consensus.cc:2410] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:02:38.471020  8009 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.473377  8401 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:38.498572  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:38.462055  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 134868us
W20260709 13:02:38.512709  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.518675  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:38.519644  1538 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:38.514480  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:38.529816  1541 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:38.530061  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:38.530656  1542 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:38.534842  1538 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:02:38.462409  8192 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:02:38.515502  7980 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:38.514842  8131 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:38.545184  8357 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:38.546047  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.574519  2233 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:38.546830  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.510219  7743 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:02:38.547731  8237 consensus_queue.cc:237] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:38.548717  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.579012  2228 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:38.549778  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.581480  2232 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
I20260709 13:02:38.550511  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:38.551285  8337 consensus_meta.cc:287] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.620s	user 0.010s	sys 0.000s
I20260709 13:02:38.530985  8356 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.551184  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.582793  2235 raft_consensus.cc:2410] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 13:02:38.553686  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 346862us
I20260709 13:02:38.583704  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:38.558980  8403 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.562284  8025 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:02:38.562911  8246 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:38.591035  1508 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:38.598048  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:38.599244  2237 raft_consensus.cc:2393] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:38.563287  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.568913  1683 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:38.515676  8406 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:38.602195  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:38.602881  2232 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:02:38.596258  8384 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:38.495572  8374 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:02:38.494997  8084 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:02:38.608000  8025 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:38.608467  8025 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:38.576648  8131 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.609213  8131 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:38.609685  8355 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20.
I20260709 13:02:38.609292  8025 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.611277  8025 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:38.586159  7980 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:38.589772  8337 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.614248  8366 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:38.615710  8337 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:38.616784  2231 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:38.617597  8406 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:38.612762  7980 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.618597  8406 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.596257  8250 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:02:38.622579  8409 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:38.623023  8409 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:38.623665  2230 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:38.624063  2230 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:38.623893  8409 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.586476  1582 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:38.583191  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:38.626502  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.497530  8334 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.540705  8133 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:38.599998  2236 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:38.601044  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.629741  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:38.606186  8395 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:38.627146  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 44052.3us
I20260709 13:02:38.627141  2233 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:38.628038  1615 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:38.627331  1534 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:38.476202  8385 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:38.631436  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 45505us
I20260709 13:02:38.632181  1542 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:38.494936  8400 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.634341  2232 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:38.624868  2230 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:38.584604  8345 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.015s	sys 0.021s
I20260709 13:02:38.637566  1583 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:38.640498  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 49529.3us
I20260709 13:02:38.642791  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:38.659334  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 168452us
I20260709 13:02:38.659590  1613 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:38.646145  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:38.634902  8238 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:38.677271  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:38.643496  8384 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:38.693807  6976 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:38.695771  2230 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:38.688609  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:38.701956  1692 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:38.649901  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:38.654157  2235 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:38.644948  8250 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:38.655781  2229 raft_consensus.cc:2368] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 20. Current term is 21.
I20260709 13:02:38.656504  1685 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:02:38.716751  1685 pending_rounds.cc:85] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 7
I20260709 13:02:38.717051  1685 pending_rounds.cc:107] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:02:38.717306  1685 raft_consensus.cc:2889] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:38.660161  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:38.718634  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 58582.6us
I20260709 13:02:38.719852  1658 leader_election.cc:400] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:38.720207  1658 leader_election.cc:403] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:38.642341  8355 raft_consensus.cc:3055] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:38.664867  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:38.655364  1614 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:38.590459  8356 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:38.697999  6976 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:38.721499  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 84027.5us
I20260709 13:02:38.730203  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:38.730790  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:38.731693  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:38.696754  1508 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:38.736502  1613 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:38.733314  1508 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:38.726966  1686 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
W20260709 13:02:38.684764  8345 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.015s	sys 0.022s
I20260709 13:02:38.643867  8133 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:38.751937  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:38.753109  1583 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:38.742733  2230 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:38.588199  8303 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.739518  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:38.676944  8238 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.695310  8297 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:38.711656  8384 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.715317  8250 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:38.731128  8419 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:38.729683  8084 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:38.725557  8355 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:38.734725  8358 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:38.750960  8329 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:38.743783  8009 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:38.765832  8403 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:38.749403  8187 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:38.749979  8375 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:02:38.790634  1658 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:38.791278  8334 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:38.794834  8133 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:38.770170  8384 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:38.771428  8192 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:38.760706  8407 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:38.743729  8418 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:38.763274  8238 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:38.772205  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:38.816030  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 1309) took 1006 ms. Trace:
I20260709 13:02:38.769232  8250 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:38.818192  8355 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.763185  8297 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:38.821807  6976 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.789033  8131 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:38.833294  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:38.802495  8395 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:38.822252  8009 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.851854  8400 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:38.827744  8187 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.820924  8403 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:38.861634  2229 rpcz_store.cc:276] 0709 13:02:37.809877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.810368 (+   491us) service_pool.cc:224] Handling call
0709 13:02:38.815742 (+1005374us) spinlock_profiling.cc:243] Waited 842 ms on lock 0x7b54004bb254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:38.815945 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1936509312}
I20260709 13:02:38.828016  8329 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:02:38.872660  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 309198us
I20260709 13:02:38.896464  2203 leader_election.cc:400] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20.
I20260709 13:02:38.896793  2203 leader_election.cc:403] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:38.884974  8381 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:38.885039  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 51843.6us
I20260709 13:02:38.862828  8418 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:38.883805  8297 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:38.849180  8192 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.892015  8009 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:38.893287  8187 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:38.894960  8403 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:38.870944  8287 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:38.896683  8065 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:02:38.909441  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 156373us
I20260709 13:02:38.907998  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:38.904949  8355 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:38.901501  8381 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:38.900859  8084 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:38.916503  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:38.920959  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:38.896128  8375 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 13:02:38.908238  8192 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:38.919127  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 157076us
I20260709 13:02:38.921538  1541 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:38.904222  8418 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:38.930460  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:38.931248  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:38.931931  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:38.901108  8302 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20.
I20260709 13:02:38.923254  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:38.932710  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:38.928715  6976 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:38.919394  1583 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:38.933574  1609 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:38.935791  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:38.936676  1609 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:38.926433  1584 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:38.939814  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:38.949687  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.950323  2230 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:38.927044  8381 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:38.941516  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:38.939754  6976 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.952986  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:38.953298  1611 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:38.952466  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:38.953890  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:38.954751  1613 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:38.957737  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:38.958344  1536 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:38.959184  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:38.920796  8334 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:38.961331  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.919772  8394 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:38.909533  8345 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 18.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 13:02:38.921501  8387 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:38.977487  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:38.935063  8407 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:39.001796  8453 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:38.963656  8356 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:39.003142  8287 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:38.961476  8133 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:39.003449  8453 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:39.003790  6976 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.004130  8133 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:39.005266  8133 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.007189  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:39.008092  8443 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 13:02:39.009147  8451 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:39.009593  8451 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:39.009758  8345 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:39.010135  8345 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:39.010438  8451 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.010975  8345 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.011767  8451 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:39.012176  8345 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:39.001780  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:39.017516  1541 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:39.018438  1543 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:02:39.019825  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:38.968040  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:38.927644  8414 raft_consensus.cc:2764] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:02:38.973706  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:39.026221  1609 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:38.962131  2229 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:38.978744  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:39.027848  1613 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:02:38.978603  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 55433.9us
W20260709 13:02:38.979760  8424 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.539s	user 0.005s	sys 0.000s
W20260709 13:02:38.980167  8437 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 13:02:38.977242  8387 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:39.019977  1509 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:39.041256  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:38.988543  7743 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16.
W20260709 13:02:38.988739  8444 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:02:38.990756  8395 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:38.992899  8131 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:38.940977  8329 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.994526  8394 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:38.998173  8375 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:38.975814  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:38.998199  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:39.066792  2231 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:39.038199  1658 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:39.033427  8287 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:39.067782  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:39.069382  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:39.070977  1536 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 22: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:39.048455  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:39.050166  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:39.051769  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:39.077369  1609 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:39.057377  1542 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:02:39.054636  8062 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:02:38.990805  8456 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 13:02:39.092257  8238 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.093657  8461 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:39.095309  8250 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.096221  8384 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:39.114811  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 143624us
I20260709 13:02:39.100193  1610 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:39.062052  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:39.103708  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 53652us
I20260709 13:02:39.115805  1657 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:39.062384  8400 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:39.059361  8150 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:39.112792  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 43533.1us
I20260709 13:02:39.102572  7980 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:39.074502  8287 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.133404  1541 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:39.074802  8385 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.122725  1615 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:39.066560  8065 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 18 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:39.130545  1582 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:39.062630  8395 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.116987  8302 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:39.144645  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2779) took 1249 ms. Trace:
I20260709 13:02:39.115079  8375 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:39.146433  1616 rpcz_store.cc:276] 0709 13:02:37.888526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.898065 (+  9539us) service_pool.cc:224] Handling call
0709 13:02:37.900667 (+  2602us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:37.909604 (+  8937us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:37.909637 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:37.925167 (+ 15530us) log.cc:844] Serialized 52 byte log entry
0709 13:02:38.196245 (+271078us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:38.196292 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:38.196366 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:38.769778 (+573412us) spinlock_profiling.cc:243] Waited 5.82 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:39.123309 (+353531us) raft_consensus.cc:1685] finished
0709 13:02:39.123332 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:39.137328 (+ 13996us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81387,"raft.queue_time_us":32998,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"spinlock_wait_cycles":13392128,"thread_start_us":241353,"threads_started":1,"wal-append.queue_time_us":339395}
I20260709 13:02:39.150759  1616 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:02:39.151659  8463 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:02:39.154822  8238 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.156658  8250 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:39.158058  8446 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 13:02:39.158406  1611 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa"
dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:45356
I20260709 13:02:39.159374  8381 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.162756  1611 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:39.163169  1611 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:39.124207  7980 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.123679  8456 consensus_queue.cc:1243] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:02:39.137049  8461 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.190559  1614 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:39.163854  1657 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:39.158349  8394 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.160666  8356 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:39.170982  8460 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:39.197674  8461 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.203603  8462 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.204069  8454 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:02:39.108057  8084 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.205858  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:39.206580  1616 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:39.187510  7980 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:39.137835  8187 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.161162  8419 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:39.222338  1612 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:39.223086  1614 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:39.231220  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:39.142226  8385 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.232237  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:39.233032  1543 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:39.235006  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:39.236094  1539 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:39.237792  1542 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:39.213181  8356 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.239226  1542 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:39.212499  8394 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.242724  1537 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:39.243525  1542 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:02:39.213593  8375 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:39.222188  1611 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.249087  1535 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:39.138005  8387 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.252103  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:39.201875  8453 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 13:02:39.252004  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:39.224449  8462 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.252939  8460 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:02:39.261611   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:39.264353  8297 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.267673  1539 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:39.284829  1611 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.246927  8356 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:39.248914  8375 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:39.295264  1543 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:39.298629  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:39.302325  6976 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:39.304272  1541 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:39.308537  6976 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:39.271642  8385 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.314159  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:39.290813  8355 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:39.299316  8375 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:39.320962  6976 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:39.321485  6976 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:39.302513  8098 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:39.325711  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:39.327244  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:39.328066  1542 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:39.247821  8065 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.254184  8394 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.336094  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:39.291424  8192 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.330070  8441 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:39.303728  8381 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.331537  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:39.337302  1538 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:39.292838  8131 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 13:02:39.302161  8009 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.321445  8334 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.346065  6976 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.325332  8098 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:39.368251  1612 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:39.350639  8385 raft_consensus.cc:2764] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:02:39.355316  8384 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:39.397409  8472 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 13:02:39.398003  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:39.430888  1538 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:39.401859  8400 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:39.330655  8355 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.409124  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:39.416533  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 975568us
I20260709 13:02:39.392570  8448 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 13:02:39.391687  8474 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 13:02:39.372058  8395 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.356778  8441 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.478856  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:39.369555  8334 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.372771  8418 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.542261  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:39.540568  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:39.581027  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:39.476904  1688 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:39.543673  1687 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:39.542996  1692 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:39.602463  1689 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
W20260709 13:02:39.603338  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 124547us
I20260709 13:02:39.557374  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:39.604169  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:39.545827  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:39.539830  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:39.612066  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:39.473694  8403 consensus_meta.cc:287] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.552s	user 0.008s	sys 0.009s
I20260709 13:02:39.479023  8084 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:39.504357  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 89150us
W20260709 13:02:39.479794  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 63726.7us
I20260709 13:02:39.722817  1611 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:39.398051  8461 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.606539  1688 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:02:39.724296  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:39.655098  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:39.543993  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:39.612687  1536 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:39.474139  8381 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:39.662467  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.739467  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:39.521029  8357 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.940s	user 0.006s	sys 0.015s
W20260709 13:02:39.743028  1613 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.003s	sys 0.079s
W20260709 13:02:39.537833  8429 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
W20260709 13:02:39.754737  1613 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.003s	sys 0.084s
I20260709 13:02:39.477272  8451 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:39.476205  8400 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:39.549674  8187 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:39.545778  8345 consensus_meta.cc:287] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.003s
I20260709 13:02:39.719880  8150 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.544773  8394 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.568604  8483 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:39.755558  8475 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.278s	user 0.001s	sys 0.001s
I20260709 13:02:39.654903  8250 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:39.658048  8297 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:39.479708  8176 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:39.790882  1687 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:02:39.609496  8464 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.608759  8131 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.166s
I20260709 13:02:39.609581  8237 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:39.743191  8462 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:39.755824  8401 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.816185  1687 pending_rounds.cc:85] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 9
I20260709 13:02:39.822687  1687 pending_rounds.cc:107] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:02:39.822983  1687 raft_consensus.cc:2889] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:39.568537  8356 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:39.821587  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 209613us
I20260709 13:02:39.824069  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:39.795432  8150 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.754698  8381 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.807515  8345 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.825383  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:39.740302  8084 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.826061  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:39.810161  8187 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:39.521284  8098 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.743280  8403 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.831061  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:39.777940  8451 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.816910  8009 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.823585  8358 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.834671  1684 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:39.842692  1507 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:39.831233  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 218168us
W20260709 13:02:39.847047  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.833685  1617 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:39.835306  1616 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:39.833814  8395 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:02:39.830422  8297 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:39.850999  1689 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:39.810075  8483 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.850307  8358 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.858707  1618 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:39.861877  8477 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.382s	user 0.001s	sys 0.000s
I20260709 13:02:39.845782  8250 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:02:39.865617  8403 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:39.866855  8460 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.867448  8460 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.869967  8460 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:39.831830  8345 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.872308  8345 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:39.874660  8150 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:39.874759  8451 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.875880  8329 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.876382  8356 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.876520  8329 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:39.856418  8485 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:02:39.865692  8187 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:39.870038  8441 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:39.870364  8381 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:39.824273  8176 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.854698  8355 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:39.850487  8476 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.444s	user 0.002s	sys 0.000s
W20260709 13:02:39.880623  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:39.894686  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:39.827781  8464 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.849481  8407 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:02:39.827373  8462 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:39.838076  8237 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:39.826375  8401 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.881389  8418 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.883769  8334 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:39.883841  1609 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:39.892431  8403 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.833251  8394 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:39.895744  1689 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:39.854370  8098 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:39.925688  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:39.927780  2228 raft_consensus.cc:1275] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:02:39.883091  8250 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:39.929087  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:39.905316  1687 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:02:39.930740  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:39.931515  1687 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:39.865342  8084 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.828300  8474 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:02:39.908085  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:39.910378  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:39.830649  8448 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:02:39.882040  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:39.923365  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:39.921095  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:39.886430  7980 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.001s
I20260709 13:02:39.927644  2235 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:39.928637  8250 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.906615  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:39.957662  2234 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:39.901052  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:39.932451  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:39.959983  2231 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:39.939623  2228 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:39.940802  2233 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:39.914273  8345 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:02:39.964125  2234 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:39.955602  2236 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:39.962558  2233 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:39.968813  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:39.976944  1618 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:39.977669  2233 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.979180  2233 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:39.980439  1509 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:39.981475  8237 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:39.917603  8462 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:39.958671  8464 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.931891  8403 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:39.990008  1613 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:39.989095  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:39.982043  8237 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:39.993834  8237 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:39.994848  8192 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.004849  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:39.964187  8394 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:02:40.006104  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:40.006182  2233 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:40.007292  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:40.007977  8375 consensus_meta.cc:287] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.000s
I20260709 13:02:40.008579  8025 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:40.009347  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:40.008049  2233 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:40.011361  2233 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:02:40.012038  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.012038  8441 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:40.013782  8441 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:40.014261  1613 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:02:40.014756  8441 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:40.018565  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.019191  2231 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:40.008652  8375 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.007292  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:40.023489  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.024834  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.027405  8329 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:39.962941  7980 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.029563  7980 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.021801  8374 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:40.030539  8374 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:40.031692  8489 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:02:40.033774  1685 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 22: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:40.026401  2230 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:40.040331  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.041283  1538 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:40.037475  1687 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:40.042752  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1228) took 1105 ms. Trace:
I20260709 13:02:40.045101  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.032713  8374 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.043208  1610 rpcz_store.cc:276] 0709 13:02:38.937826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.156392 (+218566us) service_pool.cc:224] Handling call
0709 13:02:39.156734 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.165929 (+  9195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.165974 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.186306 (+ 20332us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.250277 (+ 63971us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:39.355078 (+104801us) log.cc:844] Serialized 14 byte log entry
0709 13:02:39.409079 (+ 54001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.409159 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.042542 (+633383us) raft_consensus.cc:1685] finished
0709 13:02:40.042561 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.042697 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8713,"prepare.queue_time_us":315490,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":28200,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":27965,"spinlock_wait_cycles":474368,"thread_start_us":161142,"threads_started":3,"wal-append.queue_time_us":156077}
I20260709 13:02:40.047964  1692 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:40.048790  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:40.049799  1538 raft_consensus.cc:2368] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 20. Current term is 21.
I20260709 13:02:39.981822  8492 consensus_queue.cc:1059] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:02:40.051448  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.048686  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:40.052284  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.055505  1538 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:02:40.056110  1542 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:40.058693  1687 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:40.059193  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.061172  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:40.060977  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.069034  1542 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:40.056520  7980 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:40.071267  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.071599  1538 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 13:02:40.072021  1610 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:40.072899  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.072924  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:40.074880  1687 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:40.076074  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.064481  8355 raft_consensus.cc:3055] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:40.077391  1538 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:40.078542  1610 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:40.073995  8491 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:40.079316  8355 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:40.080384  8401 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:40.080873  8491 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.081640  8355 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:40.080449  8493 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:02:40.083066  8355 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:40.083873  8491 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.081214  8401 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:40.085090  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.067919  8176 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:40.077862  7980 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.121551  7980 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.084765  8401 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.124399  8401 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.124743  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1231) took 1156 ms. Trace:
I20260709 13:02:40.125280  1611 rpcz_store.cc:276] 0709 13:02:38.968829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.286145 (+317316us) service_pool.cc:224] Handling call
0709 13:02:39.286605 (+   460us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.723300 (+436695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.124480 (+401180us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b5400104d54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:40.124679 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1002716928}
I20260709 13:02:40.126663  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:40.127545  1611 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 22: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:40.128559  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.130386  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:40.130528  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:40.131759  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:40.131858  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:40.132396  2233 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:40.132725  1688 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 21: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:40.134742  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:40.137943  1611 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:40.139050  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:40.104511  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.105988  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.106355  8250 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:40.110096  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.113969  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2786) took 1461 ms. Trace:
I20260709 13:02:40.091069  8490 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:02:40.096755  8400 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:40.143859  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:40.145529  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 122365us
I20260709 13:02:40.145673  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:40.146416  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.147145  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:40.147802  1688 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:40.105158  1687 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:40.149984  1691 raft_consensus.cc:2410] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:40.089632  2230 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:40.140069  1658 leader_election.cc:400] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 20. Current term is 21.
I20260709 13:02:40.142622  1615 rpcz_store.cc:276] 0709 13:02:38.652555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.652691 (+   136us) service_pool.cc:224] Handling call
0709 13:02:40.113686 (+1460995us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b54002cb854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:40.113883 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1018307968}
I20260709 13:02:40.150802  1658 leader_election.cc:403] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:02:40.152133  1582 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:40.139793  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:40.153194  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:40.153971  2231 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 21: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:40.144793  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.155730  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:40.102373  8375 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.157622  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:40.160351  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:40.150760  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:40.162557  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.162716  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.163719  1688 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:40.141511  8492 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.165290  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:40.143249  8297 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.168179  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.191639  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.192127  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:40.190676  1688 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 22: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:40.201231  1687 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:40.192304  1615 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:40.202557  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:40.202836  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.205470  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.157706  8394 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:40.207286  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:40.218101  1688 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:40.219061  1611 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:40.207669  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:40.164505  8474 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:40.205832  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:40.204960  8306 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:02:40.211051  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.221841  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2793) took 1006 ms. Trace:
I20260709 13:02:40.228072  1611 rpcz_store.cc:276] 0709 13:02:39.215037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.218391 (+1003354us) service_pool.cc:224] Handling call
0709 13:02:40.218809 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.221549 (+  2740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.221778 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:40.204929  8492 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.229768  8187 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:40.230245  8187 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.230662  8187 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:40.232784  8492 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.229375  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2791) took 1015 ms. Trace:
I20260709 13:02:40.199847  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:40.233319  8306 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.235188  8387 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:40.235806  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.235682  8306 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.238263  8306 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:40.238780  1615 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:40.190982  8237 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:40.240725  1540 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:40.239611  8306 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:40.243248  8237 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:40.246766  1688 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:40.251362  1582 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:40.251875  8462 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.252401  8463 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:40.252985  8463 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:40.249191  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:40.254909  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 47332.8us
I20260709 13:02:40.163096  8401 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 20. Current term is 21.
I20260709 13:02:40.195084  8098 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:40.257107  8493 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:40.257360  8358 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:40.234540  1613 rpcz_store.cc:276] 0709 13:02:39.213743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.015159 (+801416us) service_pool.cc:224] Handling call
0709 13:02:40.015498 (+   339us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.015927 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.015968 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:40.016597 (+   629us) log.cc:844] Serialized 52 byte log entry
0709 13:02:40.086144 (+ 69547us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:40.086193 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.086279 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.229160 (+142881us) raft_consensus.cc:1685] finished
0709 13:02:40.229184 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.229318 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4943,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":2676,"thread_start_us":69091,"threads_started":1,"wal-append.queue_time_us":178212}
I20260709 13:02:40.210275  8176 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:40.266065  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:40.209918  8449 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:40.255332  1509 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:02:40.272032  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 129633us
I20260709 13:02:40.167786  7980 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:40.275094  1613 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:02:40.270251  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:40.270689  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.273492  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.276448  1535 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:40.258251  1688 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:02:40.275516  7980 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.279187  7980 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:40.261801  8302 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:40.272763  8491 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 13:02:40.281745  8491 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:40.282140  8491 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:40.282868  1507 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:40.283015  8491 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:40.283963  8250 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:40.284410  8250 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.284827  8250 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:40.285679  8497 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:40.286212  8497 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:40.286667  8497 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:40.270802  8496 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.288028  8496 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.220541  8297 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.289726  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2792) took 1075 ms. Trace:
I20260709 13:02:40.290506  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 2390) took 1358 ms. Trace:
I20260709 13:02:40.290180  1610 rpcz_store.cc:276] 0709 13:02:39.214039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.090118 (+876079us) service_pool.cc:224] Handling call
0709 13:02:40.090547 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.091260 (+   713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.091298 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.091815 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 13:02:40.217700 (+125885us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:40.217743 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.217821 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.289495 (+ 71674us) raft_consensus.cc:1685] finished
0709 13:02:40.289517 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.289661 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":126,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":125413,"threads_started":1,"wal-append.queue_time_us":184046}
I20260709 13:02:40.292399  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.291121  1617 rpcz_store.cc:276] 0709 13:02:38.932031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:38.932167 (+   136us) service_pool.cc:224] Handling call
0709 13:02:40.290331 (+1358164us) spinlock_profiling.cc:243] Waited 815 ms on lock 0x7b540008b6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:40.290439 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1874445184}
I20260709 13:02:40.294960  1615 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 22: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:40.294776  8192 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:40.295264  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.296810  1617 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:02:40.278473  8474 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:40.298612  8474 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.270898  8453 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.029s
W20260709 13:02:40.300426  1614 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 1230) took 1332 ms (client timeout 1503 ms). Trace:
I20260709 13:02:40.285729  8498 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:40.300880  1614 rpcz_store.cc:269] 0709 13:02:38.968044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.006911 (+ 38867us) service_pool.cc:224] Handling call
0709 13:02:40.300263 (+1293352us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b540008cfd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:40.300370 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":420046080}
I20260709 13:02:40.263839  8098 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.262389  8493 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.302778  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:40.308074  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:40.253953  8463 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.311811  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.329973  1610 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:02:40.312361  1614 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:40.316581  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.338150  1542 raft_consensus.cc:2393] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:40.326843  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:40.285251  8413 raft_consensus.cc:3055] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:40.343602  1536 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 23: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:40.292595  8237 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:40.290369  8496 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:40.284430  8491 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:40.311488  8098 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:40.272239  8462 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 13:02:40.220012  8394 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.349026  8413 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:40.315989  8493 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:40.349457  8355 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:40.277705  8449 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.369067  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:40.353962  8237 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.357831  8297 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.301184  8498 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:40.379136  2230 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:40.385360  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:40.390688  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:40.391688  2233 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:02:40.349711  8329 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:02:40.394162  2231 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:40.396678  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:40.397375  2237 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
W20260709 13:02:40.399721  8503 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 13:02:40.410017  8400 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:40.410383  8506 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:40.415064  8483 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:40.420143  8487 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.593s	user 0.000s	sys 0.002s
I20260709 13:02:40.367553  8084 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:02:40.371788  8462 consensus_queue.cc:1243] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:02:40.424623  2201 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:40.421626  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3377) took 1096 ms. Trace:
I20260709 13:02:40.434128  1610 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 13:02:40.376396  8237 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:40.438346  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:40.440688  2201 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:40.383191  8492 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.431952  1537 rpcz_store.cc:276] 0709 13:02:39.325555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.325662 (+   107us) service_pool.cc:224] Handling call
0709 13:02:39.325940 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.333517 (+  7577us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.333552 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.341797 (+  8245us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.552302 (+210505us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:39.552418 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.552563 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:39.875627 (+323064us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b480030c858. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:02:40.419520 (+543893us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:40.421374 (+  1854us) raft_consensus.cc:1685] finished
0709 13:02:40.421397 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.421546 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":3060,"raft.run_wall_time_us":544366,"spinlock_wait_cycles":10474240,"thread_start_us":141986,"threads_started":1,"wal-append.queue_time_us":418363}
I20260709 13:02:40.388357  8403 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:40.469911  1610 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:40.443164  8302 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:40.449447  7744 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:40.450460  8329 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:40.461040  8462 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:02:40.497289  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1232) took 1528 ms. Trace:
I20260709 13:02:40.498569  1616 rpcz_store.cc:276] 0709 13:02:38.969609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.543551 (+573942us) service_pool.cc:224] Handling call
0709 13:02:39.543870 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.849268 (+305398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.496947 (+647679us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b5400087ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:40.497204 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":669581568}
I20260709 13:02:40.480229  8345 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:40.483829  8401 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:40.504161  8394 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.486990  8250 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:40.508678  1535 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:02:40.509230  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:40.510416  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:40.512441  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:40.514096  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:40.515023  8498 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:40.516148  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:40.516747  2230 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:40.530915  8430 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:40.531162  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:40.463979  8492 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.532509  1657 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:40.494446  7744 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:40.514125  8502 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.487s	user 0.001s	sys 0.000s
I20260709 13:02:40.538079  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:40.535527  2230 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:02:40.538690  1657 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:40.538789  2237 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:40.497820  8462 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:40.510686  8250 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:40.542253  2231 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:02:40.534238  8297 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:40.546398  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:40.547670  8511 log.cc:927] Time spent T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:02:40.548923  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:40.540028  7744 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:40.500397  8329 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.540874  8065 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:40.541038  8515 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:40.544956  8374 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:40.524713  8345 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.548476  8403 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.583341  8413 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.576803  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:40.571954  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:40.590883  2237 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:40.593755  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:40.595449  8345 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:40.597237  8329 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.597977  8409 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:40.602736  8510 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:02:40.604022  8461 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:40.604647  8461 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:40.609656  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.610637  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.613898  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:40.548386  7980 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:40.577891  8358 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:40.614396  8512 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:02:40.590481  8403 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:40.616833  1610 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 13:02:40.609189  1584 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:40.618976  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3301) took 1080 ms. Trace:
I20260709 13:02:40.568657  8490 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:40.617094  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:40.629859  1686 rpcz_store.cc:276] 0709 13:02:39.538723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.541513 (+  2790us) service_pool.cc:224] Handling call
0709 13:02:39.541820 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.853675 (+311855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.853724 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.853790 (+    66us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:39.853848 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.618708 (+764860us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b5400355054. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:40.618770 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.618920 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716144896}
I20260709 13:02:40.648834  1687 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:40.650396  8492 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.654808  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:40.660825  1686 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:40.662293  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:40.650941  8449 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.658192  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2767) took 1358 ms. Trace:
I20260709 13:02:40.671746  1543 rpcz_store.cc:276] 0709 13:02:39.299877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.300421 (+   544us) service_pool.cc:224] Handling call
0709 13:02:39.312651 (+ 12230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:39.526508 (+213857us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:39.526551 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:39.526606 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:39.526652 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:40.657860 (+1131208us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54003453d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:40.657936 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.658112 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":491297024}
W20260709 13:02:40.655790  8521 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:02:40.588111  8495 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:02:40.605151  8237 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:40.690214  1612 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2790) took 1476 ms (client timeout 1693 ms). Trace:
W20260709 13:02:40.690918  1612 rpcz_store.cc:269] 0709 13:02:39.213465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.236919 (+ 23454us) service_pool.cc:224] Handling call
0709 13:02:40.673864 (+1436945us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54000875d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:40.690128 (+ 16264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241826176}
I20260709 13:02:40.624782  8413 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:40.754518  1686 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:02:40.768006  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 3305) took 1160 ms. Trace:
I20260709 13:02:40.770643  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:40.773517  8525 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.774813  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 164749us
I20260709 13:02:40.776085  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:40.777464  8527 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:40.777894  8527 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:40.778280  8403 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:40.778241  8527 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:40.773208  8355 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:40.778896  8430 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:40.651628  8065 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.128s
I20260709 13:02:40.780936  8355 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:40.768571  1684 rpcz_store.cc:276] 0709 13:02:39.539863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.543753 (+  3890us) service_pool.cc:224] Handling call
0709 13:02:40.674193 (+1130440us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540003e154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:40.699858 (+ 25665us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249686912}
I20260709 13:02:40.782045  8497 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.784397  8497 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.774334  8525 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:40.788410  8098 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:40.790680  8098 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:40.784825  7744 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.788873  8464 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:40.792336  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 220509us
W20260709 13:02:40.793098  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 75803.5us
I20260709 13:02:40.587944  8515 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.639693  8374 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.795259  7744 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:40.793143  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:40.652205  8385 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:40.800981  1657 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:40.790632  8525 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:40.589382  8462 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:40.815408  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:40.822166  8484 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.049s	user 0.004s	sys 0.000s
I20260709 13:02:40.823292  1692 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:40.823917  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3375) took 1500 ms. Trace:
I20260709 13:02:40.824460  1539 rpcz_store.cc:276] 0709 13:02:39.323657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.323796 (+   139us) service_pool.cc:224] Handling call
0709 13:02:39.324110 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:39.324334 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:39.324358 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:39.333378 (+  9020us) log.cc:844] Serialized 26 byte log entry
0709 13:02:39.479514 (+146136us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:39.479641 (+   127us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:39.479723 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:40.798652 (+1318929us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:40.823520 (+ 24868us) raft_consensus.cc:1685] finished
0709 13:02:40.823557 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:40.823838 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6766848,"thread_start_us":110113,"threads_started":1,"wal-append.queue_time_us":255548}
I20260709 13:02:40.828112  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:40.829476  8515 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:40.831656  8529 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:02:40.834116  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:40.804100  8374 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.839304  8374 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.801409  8385 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:40.840185  8385 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:40.849246  8237 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:40.865751  1684 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 13:02:40.866379  1684 pending_rounds.cc:85] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 13
I20260709 13:02:40.866645  1684 pending_rounds.cc:107] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 13:02:40.866930  1684 raft_consensus.cc:2889] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:40.651343  8490 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:40.881333  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:40.884490  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.804808  8456 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:02:40.885665  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 109485us
W20260709 13:02:40.889132  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:40.696659  8187 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.894726  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.896361  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:40.896241  1507 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:40.901594  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.800575  8464 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.903848  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:40.934549  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:40.934741  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:40.935261  2237 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:40.689078  8461 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.885881  8462 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:40.943351  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:40.946972  1610 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:40.808321  8463 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 13:02:40.899533  8430 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.935688  8526 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:40.930135  8497 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:40.902824  8525 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:40.980873  8525 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:40.984117  8525 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:40.939793  8456 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 22 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:40.897907  8237 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
W20260709 13:02:41.039235  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 136166us
I20260709 13:02:41.040488  8098 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:40.976018  8025 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:41.027403  8484 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:02:41.047183  7744 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.048421  2233 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:41.050796  2237 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:40.948396  8384 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.082s
I20260709 13:02:41.052793  7744 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.046687  1583 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:41.063331  1610 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:41.065665  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:41.066936  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:41.072914  1540 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:41.078445  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:41.080403  1611 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:41.078652  1508 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.225s (SENT) [suppressed 6 similar messages]
I20260709 13:02:41.079324  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:41.085258  2236 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:41.079749  1618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 2400) took 1553 ms (client timeout 1742 ms). Trace:
I20260709 13:02:41.071976  1583 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:02:41.086097  1618 rpcz_store.cc:269] 0709 13:02:39.526823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.539495 (+ 12672us) service_pool.cc:224] Handling call
0709 13:02:41.079544 (+1540049us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400398ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:41.079688 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469975296}
I20260709 13:02:41.042309  8338 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.090293  7744 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:41.093951  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:40.979281  8430 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.101826  2237 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 22: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:41.056344  8306 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:41.080026  8519 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.429s	user 0.000s	sys 0.001s
I20260709 13:02:41.114446  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:41.115267  1542 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:41.116187  2237 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:41.096869  8358 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:02:41.043071  8474 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:02:41.131287  2236 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
W20260709 13:02:41.095495  8493 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.711s	user 0.013s	sys 0.000s
I20260709 13:02:41.032320  8526 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:41.145541  2228 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
I20260709 13:02:41.112377  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:41.146935  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:41.156123  2235 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:41.119269  8375 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:41.156507  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:41.055361  8025 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.158634  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:41.159523  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:41.165716  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:41.167100  1542 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:02:41.168422  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:41.169768  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:41.171046  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:41.174938  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:41.178303  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:41.179829  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:41.120365  8515 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:41.122750  8355 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:41.186321  1540 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:41.195832  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:41.198246  8358 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:41.198529  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:41.154528  8495 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:41.101284  8338 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.176872  8461 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:41.149578  8493 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:41.132656  8306 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.183828  7743 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:02:41.206905  8523 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:02:41.183585  8419 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:41.244944  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 47254.8us
I20260709 13:02:41.198948  8358 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:41.296116  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 224231us
I20260709 13:02:41.337685  8358 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:41.175668  8375 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.245240  1540 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:41.249776  8374 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:41.172926  8483 consensus_queue.cc:1048] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.100s
I20260709 13:02:41.247217  8491 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:41.248454  8355 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:41.366881  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:41.155058  8456 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.292297  8490 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:41.365365  1688 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:41.250087  8539 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:41.338611  8025 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:41.342589  8329 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.275902  8515 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:41.301259  7743 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:41.344262  8187 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:41.376291  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:41.342315  8409 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:41.383086  1683 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:41.384636  1689 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 3311) took 1798 ms (client timeout 1518 ms). Trace:
W20260709 13:02:41.370752  8530 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
W20260709 13:02:41.386687  1689 rpcz_store.cc:269] 0709 13:02:39.586791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.603582 (+ 16791us) service_pool.cc:224] Handling call
0709 13:02:41.384378 (+1780796us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54000aafd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:41.384529 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288072064}
I20260709 13:02:41.387935  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:41.369189  8374 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:41.388518  1689 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:41.389108  8374 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:41.390195  8409 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.391279  1542 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
W20260709 13:02:41.393041  8532 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
I20260709 13:02:41.389148  8515 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:41.394156  7743 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.374614  8539 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.395654  8539 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:41.398931  7744 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.399446  7744 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.401337  7744 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:41.393210  8409 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:41.402541  2236 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 13:02:41.403331  8538 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.284s	user 0.000s	sys 0.001s
I20260709 13:02:41.404505  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2345) took 1358 ms. Trace:
I20260709 13:02:41.405272  8525 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:41.405752  8525 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:41.405953  8409 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:41.406586  8409 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.406651  8525 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:41.409538  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:41.413786  8536 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 13:02:41.413244  1688 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:02:41.418563  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3326) took 1300 ms. Trace:
I20260709 13:02:41.401599  8025 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.409628  8409 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:41.404910  1692 rpcz_store.cc:276] 0709 13:02:40.045621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.828455 (+782834us) service_pool.cc:224] Handling call
0709 13:02:40.829925 (+  1470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.865187 (+ 35262us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.865253 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.894253 (+ 29000us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.039907 (+145654us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:41.039964 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.040052 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.404256 (+364204us) raft_consensus.cc:1685] finished
0709 13:02:41.404283 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.404445 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36737,"raft.queue_time_us":35753,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":7393,"thread_start_us":97509,"threads_started":1,"wal-append.queue_time_us":177120}
I20260709 13:02:41.425740  1692 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:02:41.423481  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.418915  1688 rpcz_store.cc:276] 0709 13:02:40.118204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.412669 (+1294465us) service_pool.cc:224] Handling call
0709 13:02:41.413019 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.418286 (+  5267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.418507 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.375660  8491 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:41.426348  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3327) took 1307 ms. Trace:
I20260709 13:02:41.439663  2229 raft_consensus.cc:1275] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:41.429940  1692 rpcz_store.cc:276] 0709 13:02:40.118496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.425036 (+1306540us) service_pool.cc:224] Handling call
0709 13:02:41.425347 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.426108 (+   761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.426295 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:02:41.463795  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.465045  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:41.463958  1509 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:02:41.468659  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 272906us
I20260709 13:02:41.462438  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2797) took 1707 ms. Trace:
I20260709 13:02:41.469545  1683 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:41.429491  8495 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.473348  1509 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:41.474045  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.470325  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:41.384397  8329 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:41.443851  8407 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:41.427419  8025 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:41.521215  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 182312us
I20260709 13:02:41.520790  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:41.492843  1489 auto_rebalancer.cc:893] Removing replace marker failed: Replica 8a6a5ae1b7f240408b3ddd30d8a18262 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:02:41.502950  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.536221  1489 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:02:41.536640  1489 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:02:41.536684  2234 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:41.469821  1615 rpcz_store.cc:276] 0709 13:02:39.755288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.295652 (+540364us) service_pool.cc:224] Handling call
0709 13:02:40.295934 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.386007 (+ 90073us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:40.445050 (+ 59043us) log.cc:844] Serialized 14 byte log entry
0709 13:02:40.555641 (+110591us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:40.555701 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:40.555747 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.462120 (+906373us) spinlock_profiling.cc:243] Waited 89.7 ms on lock 0x7b540008cad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:41.462177 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.462345 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1599,"prepare.queue_time_us":62214,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":3004,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":19134,"spinlock_wait_cycles":211744768,"thread_start_us":109938,"threads_started":1,"wal-append.queue_time_us":225551}
I20260709 13:02:41.537355  2234 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:41.484385  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:41.492735  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:41.492735  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:41.543227  1543 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
W20260709 13:02:41.477216  1582 connection.cc:256] RPC call timeout handler was delayed by -1.0718874900000515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15180000000000002s for a response
I20260709 13:02:41.542126  1583 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:41.538450  2234 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.548255  2234 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:41.549153  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 470488us
W20260709 13:02:41.537170  8463 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.003s	sys 0.009s
I20260709 13:02:41.550778  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:41.506096  8237 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:02:41.547008  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.553393  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:41.468529  8430 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:41.555148  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:41.475643  8187 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:41.492753  8297 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:41.541502  8419 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.520435  8287 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:41.566643  8374 consensus_queue.cc:1059] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:02:41.567863  8479 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 2.191s	user 0.006s	sys 0.000s
I20260709 13:02:41.575071  8306 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:41.568627  8544 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 13:02:41.569310  8533 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 13:02:41.570196  8552 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:41.573403  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:41.574904  8025 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:41.576579  8541 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:41.580268  8542 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:02:41.557058  8192 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:41.586961  8355 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:41.587440  8527 consensus_meta.cc:287] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.808s	user 0.000s	sys 0.010s
I20260709 13:02:41.575914  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:41.589859  8549 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:02:41.591830  8537 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.403s	user 0.001s	sys 0.000s
I20260709 13:02:41.590381  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:41.564834  8237 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:41.594460  8553 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:02:41.685694  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3325) took 1567 ms. Trace:
I20260709 13:02:41.685930  8357 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:02:41.686305  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 95994.8us
I20260709 13:02:41.686264  1690 rpcz_store.cc:276] 0709 13:02:40.117868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.835603 (+717735us) service_pool.cc:224] Handling call
0709 13:02:40.835991 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.889100 (+ 53109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.889148 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.899339 (+ 10191us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.151736 (+252397us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:41.151781 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.151923 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.685327 (+533404us) raft_consensus.cc:1685] finished
0709 13:02:41.685352 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.685633 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68795,"raft.queue_time_us":59510,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":31038,"thread_start_us":129387,"threads_started":1,"wal-append.queue_time_us":491844}
I20260709 13:02:41.687463  8133 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:02:41.596279  8413 consensus_meta.cc:287] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.803s	user 0.012s	sys 0.000s
I20260709 13:02:41.688139  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:41.688642  8133 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:41.597216  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 3317) took 1697 ms. Trace:
I20260709 13:02:41.545979  8547 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.685873  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3323) took 1701 ms. Trace:
I20260709 13:02:41.689024  8133 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:41.689867  1685 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 13:02:41.690268  1685 pending_rounds.cc:85] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 14
I20260709 13:02:41.690500  1685 pending_rounds.cc:107] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 13:02:41.690996  1685 raft_consensus.cc:2889] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:41.690438  8133 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.610232  2234 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:02:41.691711  2234 pending_rounds.cc:85] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 12
I20260709 13:02:41.691954  2234 pending_rounds.cc:107] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:02:41.692181  2234 raft_consensus.cc:2889] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:41.615942  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.549881  8526 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:41.557560  8463 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.004s	sys 0.009s
I20260709 13:02:41.651023  1683 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 14.
I20260709 13:02:41.550669  8515 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
W20260709 13:02:41.644469  8176 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.369s	user 0.014s	sys 0.000s
I20260709 13:02:41.542445  8539 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.544440  8407 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.688463  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:41.685642  8237 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:41.689829  1686 rpcz_store.cc:276] 0709 13:02:39.984841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.796272 (+811431us) service_pool.cc:224] Handling call
0709 13:02:40.796561 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:40.800197 (+  3636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.800227 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:40.805130 (+  4903us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.040153 (+235023us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:41.040230 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.040320 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.591196 (+550876us) spinlock_profiling.cc:243] Waited 8.25 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:41.685573 (+ 94377us) raft_consensus.cc:1685] finished
0709 13:02:41.685599 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.685816 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1258,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":6076,"spinlock_wait_cycles":18972928,"thread_start_us":134066,"threads_started":1,"wal-append.queue_time_us":216298}
I20260709 13:02:41.493707  8495 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:41.645130  8187 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:41.658800  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:41.724097  1692 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:41.648844  8493 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:41.650777  8338 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:41.667007  8524 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.667119  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2815) took 1058 ms. Trace:
W20260709 13:02:41.668116  8385 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.827s	user 0.011s	sys 0.002s
W20260709 13:02:41.668744  8462 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.712s	user 0.008s	sys 0.002s
I20260709 13:02:41.657262  8192 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:41.661720  1535 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:41.660825  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:41.669778  8522 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.021s	user 0.003s	sys 0.001s
I20260709 13:02:41.735631  1540 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:41.724153  8176 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:41.736050  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:41.665042  8297 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:41.662799  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 86950.8us
I20260709 13:02:41.726337  8187 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:41.649067  8419 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.668661  8549 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:41.675824  1691 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 3306) took 2135 ms (client timeout 2059 ms). Trace:
W20260709 13:02:41.679562  8518 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.898s	user 0.005s	sys 0.005s
I20260709 13:02:41.682690  8492 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:41.657706  8541 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:41.595165  8329 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:41.735347  8355 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.689229  1687 rpcz_store.cc:276] 0709 13:02:39.899886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.202244 (+302358us) service_pool.cc:224] Handling call
0709 13:02:41.558535 (+1356291us) spinlock_profiling.cc:243] Waited 445 ms on lock 0x7b54000749d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:41.597139 (+ 38604us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1024637312}
W20260709 13:02:41.689536  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:41.650682  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 173889us
I20260709 13:02:41.758082  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:41.689381  1583 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:41.721978  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:41.692345  8133 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:41.660606  8556 raft_consensus.cc:3055] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:41.689680  8483 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:41.727994  1616 rpcz_store.cc:276] 0709 13:02:40.608920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.615377 (+  6457us) service_pool.cc:224] Handling call
0709 13:02:40.623184 (+  7807us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.628719 (+  5535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.628770 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:40.691671 (+ 62901us) log.cc:844] Serialized 52 byte log entry
0709 13:02:41.032321 (+340650us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:41.032837 (+   516us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.033130 (+   293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.033245 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:41.666860 (+633615us) raft_consensus.cc:1685] finished
0709 13:02:41.666884 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.667059 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29965,"prepare.queue_time_us":10126,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":77836,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":26510,"spinlock_wait_cycles":9779072,"thread_start_us":227160,"threads_started":1,"wal-append.queue_time_us":458773}
I20260709 13:02:41.695844  8287 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:41.671509  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2796) took 2288 ms. Trace:
I20260709 13:02:41.738011  1507 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:41.737799  1582 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:41.739286  1690 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:41.735167  8192 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:41.750079  1692 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
W20260709 13:02:41.746683  1691 rpcz_store.cc:269] 0709 13:02:39.540171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.540315 (+   144us) service_pool.cc:224] Handling call
0709 13:02:41.675550 (+2135235us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b54000aaad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:41.675718 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1258627328}
I20260709 13:02:41.696722  8527 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.758967  1687 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
W20260709 13:02:41.766188  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 76869.6us
I20260709 13:02:41.763974  1613 rpcz_store.cc:276] 0709 13:02:39.383391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.276554 (+893163us) service_pool.cc:224] Handling call
0709 13:02:40.313144 (+ 36590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.430079 (+116935us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:02:40.773820 (+343741us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.078729 (+304909us) spinlock_profiling.cc:243] Waited 42.6 ms on lock 0x7b18001c6960. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:02:41.250717 (+171988us) log.cc:844] Serialized 14 byte log entry
0709 13:02:41.272989 (+ 22272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:41.273054 (+    65us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:41.273113 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.671262 (+398149us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400105254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:41.671318 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:41.671449 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109300,"prepare.queue_time_us":146106,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":3020,"raft.queue_time_us":219870,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":23873,"spinlock_wait_cycles":359011840,"thread_start_us":719040,"threads_started":3,"wal-append.queue_time_us":204893}
I20260709 13:02:41.771802  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:41.774549  7744 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:41.776340  1687 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:41.777081  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2349) took 1498 ms. Trace:
I20260709 13:02:41.777554  1687 rpcz_store.cc:276] 0709 13:02:40.278704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.775774 (+1497070us) service_pool.cc:224] Handling call
0709 13:02:41.776031 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.776778 (+   747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.777008 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.775177  7744 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.780664  1691 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 7. (term mismatch)
I20260709 13:02:41.782214  1691 pending_rounds.cc:85] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 6
I20260709 13:02:41.782500  1691 pending_rounds.cc:107] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 13:02:41.782779  1691 raft_consensus.cc:2889] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:41.783615  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2348) took 1505 ms. Trace:
I20260709 13:02:41.784004  1691 rpcz_store.cc:276] 0709 13:02:40.278446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.775286 (+1496840us) service_pool.cc:224] Handling call
0709 13:02:41.775533 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.783300 (+  7767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.783548 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:41.766752  8287 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:41.787379  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 49470.6us
I20260709 13:02:41.735522  8385 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.787736  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:41.814589  1691 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:02:41.818961  1690 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:41.763820  8025 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.820329  8025 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.823082  8025 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:41.780634  8415 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:41.787127  8463 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 13:02:41.794113  8560 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:41.795912  8541 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:41.796041  7744 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:41.792374  1507 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:41.767076  8483 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:41.827991  8483 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.798342  8495 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:41.799647  8418 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:41.799923  8419 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:41.760075  8556 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:41.831416  8556 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:41.832535  8556 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:41.802244  8497 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:41.834100  8497 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:41.803269  1541 raft_consensus.cc:1240] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:41.805821  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:41.689476  8547 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.837396  8547 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:41.793646  8098 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:41.838172  8098 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:41.826047  8463 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:02:41.839483  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 81173.2us
I20260709 13:02:41.731870  8524 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:41.849814  1691 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 13:02:41.742022  8297 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:41.769729  8357 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:41.845386  1690 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:41.776633  8176 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.781400  8338 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.856864  1609 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:41.781644  8430 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:41.859777  1690 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:41.809888  8287 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:41.760818  8492 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:41.882587  8520 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.211s	user 0.001s	sys 0.006s
I20260709 13:02:41.885362  1683 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:41.768172  8355 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.871483  7980 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:41.873679  8430 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.874830  8385 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:41.856493  8409 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:41.892980  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 75996.2us
I20260709 13:02:41.900400  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2794) took 2671 ms. Trace:
I20260709 13:02:41.877519  8297 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:41.901082  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 108814us
I20260709 13:02:41.881136  8492 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:41.902820  1611 rpcz_store.cc:276] 0709 13:02:39.228880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.229241 (+1000361us) service_pool.cc:224] Handling call
0709 13:02:41.080222 (+850981us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.083952 (+  3730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.893709 (+809757us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b5400398ad0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:41.900339 (+  6630us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1956220288}
I20260709 13:02:41.906744  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:41.907208  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:41.913187  2228 raft_consensus.cc:2410] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:41.907678  8547 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:41.790813  8563 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.117s
I20260709 13:02:41.908684  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:41.791963  8009 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.095s
I20260709 13:02:41.908026  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:41.914415  2229 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:02:41.918401  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 166183us
I20260709 13:02:41.918745  2234 raft_consensus.cc:2376] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18. Re-sending same reply.
I20260709 13:02:41.919215  1508 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:41.911937  1507 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:41.916193  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 127634us
W20260709 13:02:41.914893  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 141909us
I20260709 13:02:41.921692  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:41.866271  8176 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.922778  1582 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:41.926690  1508 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:41.930426  2228 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: [17.15-17.15]
W20260709 13:02:41.936414  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:41.940069  1690 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:41.942045  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:41.934783  1583 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:41.958298  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:41.899869  8565 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 13:02:41.957566  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:41.958371  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:41.942834  2232 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:41.960215  1611 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 20. Current term is 21. Ops: [19.11-20.12]
I20260709 13:02:41.964874  8567 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:41.964903  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:41.927332  8492 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:41.965633  8541 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:41.926208  8297 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:41.971127  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:41.895782  8338 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.971621  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:41.913285  8409 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.934015  8577 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:41.980592  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:41.981185  1537 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:02:41.986099  8566 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:02:41.981300  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:41.992933  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:41.997591  2232 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:41.997493  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:41.963158  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:41.929077  8574 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:41.918184  8547 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:41.942929  8329 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:41.928409  8483 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:42.090600  1610 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
W20260709 13:02:42.089924  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 155165us
I20260709 13:02:41.912581  8549 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.093703  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:41.942047  8582 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:02:42.094841  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 93615.9us
W20260709 13:02:42.120671  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.119125  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:42.096668  1616 raft_consensus.cc:2376] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16. Re-sending same reply.
W20260709 13:02:42.122018  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.097950  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 139737us
I20260709 13:02:42.097970  1690 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:42.097261  1611 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:41.970715  8418 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.135s
I20260709 13:02:42.123863  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:42.122954  1657 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:42.101792  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:42.101982  8584 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:42.102447  8539 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:41.934156  8515 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 13:02:42.128417  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.098168  8574 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:41.978678  8407 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.111838  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.113008  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:42.093258  2203 leader_election.cc:336] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.503s (SENT)
I20260709 13:02:41.999922  8297 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:42.099299  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.127112  1508 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:42.122676  8560 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:42.146395  1538 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:42.130410  1657 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:42.131280  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:42.153153  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.131937  1615 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:42.115893  8483 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.242139  8483 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:42.244750  8358 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.148030  8582 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:42.245880  8582 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.245525  8358 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:42.246315  8582 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:42.248414  8358 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:42.138598  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.143172  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:42.251070  8025 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:02:42.251250  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 158010us
W20260709 13:02:42.254907  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 124540us
I20260709 13:02:42.255896  1657 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:42.256281  7743 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:42.256739  8401 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:42.257529  8461 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:42.099665  8563 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:42.257098  1657 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:42.258385  8563 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.259884  8527 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.260799  8563 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.157212  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.147423  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:42.147802  1508 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:42.266227  1537 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:42.266443  1611 raft_consensus.cc:2410] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
W20260709 13:02:42.266695  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 118989us
I20260709 13:02:42.268458  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.131852  7980 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:42.124725  8576 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:42.270974  8576 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.272678  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.197026  8187 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.171645  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3319) took 2217 ms. Trace:
I20260709 13:02:42.144583  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.173120  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1442) took 1106 ms. Trace:
I20260709 13:02:42.275249  1684 rpcz_store.cc:276] 0709 13:02:39.954540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.784159 (+829619us) service_pool.cc:224] Handling call
0709 13:02:40.784399 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:40.867490 (+ 83091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.171383 (+1303893us) spinlock_profiling.cc:243] Waited 75.7 ms on lock 0x7b5400028854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:42.171580 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174180224}
I20260709 13:02:42.105607  8084 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:02:42.177937  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2346) took 2132 ms. Trace:
W20260709 13:02:42.179045  8572 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:02:42.100308  8577 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.292091  1685 rpcz_store.cc:276] 0709 13:02:40.045884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.876335 (+830451us) service_pool.cc:224] Handling call
0709 13:02:40.876780 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.691428 (+814648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.177708 (+486280us) spinlock_profiling.cc:243] Waited 813 ms on lock 0x7b540038aed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:42.177876 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1868995456}
I20260709 13:02:42.296733  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:42.190116  8098 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.191679  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1237) took 3046 ms. Trace:
I20260709 13:02:42.133283  8574 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:42.169148  8430 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:42.193383  8547 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.193647  8560 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:42.196470  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:42.340747  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.199373  8176 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.199509  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2794) took 1048 ms. Trace:
I20260709 13:02:42.200335  8287 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:42.201892  8237 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.211598  8464 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.212212  8525 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:42.216023  8375 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.216572  8403 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.219053  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:42.151290  8297 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:42.220723  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:42.221046  8575 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 13:02:42.133208  8329 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 13:02:42.222043  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:42.197325  8449 raft_consensus.cc:3055] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:02:42.225741  1428 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab5b971190d342d2b2568c30f3c0c81f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8a6a5ae1b7f240408b3ddd30d8a18262 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:42.156040  8587 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:42.234764  2201 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:42.235829  8580 log.cc:927] Time spent T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 13:02:42.236384  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:42.238395  8556 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.257647  8009 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.265587  6976 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:42.408623  6976 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:42.137804  8564 log.cc:927] Time spent T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 13:02:42.281198  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 288161us
I20260709 13:02:42.382359  8449 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:42.296784  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.298172  1684 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:42.300045  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:42.301939  1609 rpcz_store.cc:276] 0709 13:02:39.145414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:39.987482 (+842068us) service_pool.cc:224] Handling call
0709 13:02:40.689861 (+702379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.866632 (+1176771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.190771 (+324139us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000875d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:42.191421 (+   650us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b54000875d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:42.191585 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4289647616}
I20260709 13:02:42.305585  1685 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:42.416430  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.416754  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:42.417603  1685 raft_consensus.cc:2410] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 13:02:42.309593  8577 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.277367  8576 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.415768  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:42.315891  1509 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:42.318041  8355 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.382346  8403 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:42.295910  8563 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:42.322081  8524 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:42.324169  8187 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.154594  8492 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.174s
I20260709 13:02:42.333621  1614 raft_consensus.cc:2376] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18. Re-sending same reply.
I20260709 13:02:42.337477  1582 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:02:42.435681  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 98346.3us
I20260709 13:02:42.230655  8409 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:42.340939  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 83908.6us
I20260709 13:02:42.341114  1690 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:42.341677  1542 rpcz_store.cc:276] 0709 13:02:41.151581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.170822 (+ 19241us) service_pool.cc:224] Handling call
0709 13:02:42.199292 (+1028470us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b540012e3d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:42.199432 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":504827776}
I20260709 13:02:42.437701  1657 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:42.438433  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.341718  8385 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.014s
I20260709 13:02:42.361184  8483 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.374650  8560 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.418427  8527 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:42.390437  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:42.348066  8084 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:42.383492  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.462984  1618 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:42.429207  8563 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.406075  1683 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
W20260709 13:02:42.407085  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 172405us
I20260709 13:02:42.276042  1541 rpcz_store.cc:276] 0709 13:02:41.067153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.069504 (+  2351us) service_pool.cc:224] Handling call
0709 13:02:41.097347 (+ 27843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:41.835225 (+737878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.172901 (+337676us) spinlock_profiling.cc:243] Waited 706 ms on lock 0x7b54001e5354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:42.173062 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1622983552}
I20260709 13:02:42.474699  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:42.475507  2233 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:42.417568  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:42.424517  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:42.476634  1609 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:42.477666  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:42.234642  8192 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.426102  6976 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.409228  8025 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:42.430042  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:42.431502  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:42.440130  1657 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:42.426892  8355 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:42.440186  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.490149  1542 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:42.411489  8587 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 21 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:42.500214  1685 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:42.501106  8587 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.502132  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.450681  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:42.503607  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:42.504101  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 53547.2us
I20260709 13:02:42.505667  8403 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:42.505651  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:42.479120  8084 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.506930  1582 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:42.451562  7743 consensus_queue.cc:1048] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:42.507172  8192 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.417809  8461 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:42.508643  8084 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.509025  8176 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:42.509588  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.510492  8287 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:42.511107  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.512648  1540 raft_consensus.cc:2410] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 13:02:42.515339  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.516392  1542 raft_consensus.cc:2393] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:42.495203  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.519905  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:42.523205  7743 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:42.396520  8567 raft_consensus.cc:697] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:42.378118  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:42.455868  8474 raft_consensus.cc:2764] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 13:02:42.463956  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:42.403302  1428 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 177625us
I20260709 13:02:42.432423  8588 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:42.463212  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:42.415802  8009 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:42.491853  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:42.492738  1610 raft_consensus.cc:2368] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 20. Current term is 21.
I20260709 13:02:42.492921  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3329) took 2373 ms. Trace:
W20260709 13:02:42.573211  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 81481.4us
I20260709 13:02:42.573588  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:42.573341  1692 rpcz_store.cc:276] 0709 13:02:40.119097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.774612 (+1655515us) service_pool.cc:224] Handling call
0709 13:02:41.774969 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.775306 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.775346 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.787604 (+ 12258us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.870002 (+ 82398us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:41.870068 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.870140 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.492627 (+622487us) raft_consensus.cc:1685] finished
0709 13:02:42.492657 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.492834 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4859,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":9114,"thread_start_us":70306,"threads_started":1,"wal-append.queue_time_us":138823}
W20260709 13:02:42.494275  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 54238.7us
I20260709 13:02:42.488689  8578 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.508010  1612 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:42.531497  2201 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:42.531104  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:42.533815  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:42.580065  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:42.411111  8556 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:42.535904  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.519619  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:42.583230  2229 raft_consensus.cc:2376] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18. Re-sending same reply.
I20260709 13:02:42.543231  1540 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:42.548434  7743 consensus_queue.cc:1059] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:42.552474  1542 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:42.558104  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:42.585917  2230 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:42.544266  8394 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:42.561831  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:42.563388  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3328) took 2444 ms. Trace:
I20260709 13:02:42.585180  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:42.453897  8574 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:42.587051  1688 rpcz_store.cc:276] 0709 13:02:40.118768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.426807 (+1308039us) service_pool.cc:224] Handling call
0709 13:02:41.427189 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.441702 (+ 14513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.441749 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.444219 (+  2470us) log.cc:844] Serialized 26 byte log entry
0709 13:02:41.724241 (+280022us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:41.724281 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:41.724343 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.563152 (+838809us) raft_consensus.cc:1685] finished
0709 13:02:42.563178 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.563326 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2114,"raft.queue_time_us":2220,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3383,"thread_start_us":219359,"threads_started":1,"wal-append.queue_time_us":387176}
I20260709 13:02:42.590200  1541 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:02:42.524066  8355 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:42.492169  8524 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:42.481659  8563 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:42.571040  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.571379  2233 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
W20260709 13:02:42.571695  2231 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.010s	sys 0.017s
I20260709 13:02:42.575649  1657 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:42.478942  8449 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:42.602015  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:42.580982  1535 raft_consensus.cc:2368] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 22. Current term is 23.
W20260709 13:02:42.581271  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.523114  8357 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:02:42.597203  8098 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.611660  8497 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:42.612825  8497 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:42.569559  1609 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:02:42.614634  1538 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:42.614997  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.577577  8375 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 13:02:42.596179  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:42.600437  2231 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.010s	sys 0.017s
I20260709 13:02:42.600513  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.588608  8192 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:42.626760  1618 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:42.627759  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.628314  1618 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
W20260709 13:02:42.608045  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:42.615438  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 496343us
I20260709 13:02:42.615646  1609 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:42.466063  8527 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:42.592651  8025 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:42.618505  2228 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:42.613849  8176 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:42.636472  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:42.636408  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:42.637171  1618 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:42.639725  8483 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.622386  8409 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:42.630484  8192 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.641775  8192 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:42.643018  8527 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:42.613915  8084 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:42.638857  1612 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:42.588881  8578 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.652419  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3330) took 2343 ms. Trace:
I20260709 13:02:42.653007  1687 rpcz_store.cc:276] 0709 13:02:40.309108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.778522 (+1469414us) service_pool.cc:224] Handling call
0709 13:02:41.778910 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.781848 (+  2938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:41.781887 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:41.791118 (+  9231us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.257377 (+466259us) spinlock_profiling.cc:243] Waited 2.79 ms on lock 0x7b48000cd158. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:02:42.271452 (+ 14075us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b18003fbc20. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:02:42.524161 (+252709us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:42.524211 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.524292 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:42.652186 (+127894us) raft_consensus.cc:1685] finished
0709 13:02:42.652206 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:42.652359 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":910,"raft.run_cpu_time_us":1859,"raft.run_wall_time_us":476176,"spinlock_wait_cycles":32685696,"thread_start_us":252469,"threads_started":1,"wal-append.queue_time_us":845651}
I20260709 13:02:42.634495  8449 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:42.660511  1618 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:42.663992  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.666128  1618 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:42.669502  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.672081  1618 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:42.676709  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.679901  1618 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:42.692899  2232 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:42.645730  8588 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:42.697053  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.637171  8025 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:42.700942  2228 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 24.
I20260709 13:02:42.694718  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:42.702241  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 49478.9us
I20260709 13:02:42.649070  8176 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:42.715772  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.709609  8025 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:42.739295  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.740075  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:42.741624  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.742461  1540 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:42.743626  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.744349  1540 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
W20260709 13:02:42.747229  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.745366  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.693713  8590 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:42.754943  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:42.634838  8567 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.774264  1542 raft_consensus.cc:2410] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 13:02:42.775717  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:42.773193  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 395167us
I20260709 13:02:42.776937  1540 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:42.782045  1540 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:42.783520  8545 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:42.784294  8545 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:42.787889  8375 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.788589  8409 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:42.788664  8375 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.789283  8409 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.790884  8464 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.791389  8409 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:42.793382  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:42.791430  8464 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.796835  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:42.796826  8589 log.cc:927] Time spent T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:02:42.797505  8556 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.798189  8464 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:42.795882  8578 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:42.798269  2228 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:42.777815  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:42.800284  8187 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:42.800755  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:42.800804  8187 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:42.800925  8556 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:42.802233  8187 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:42.802356  8556 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.802772  8556 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:42.760684  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 444886us
I20260709 13:02:42.819344  1509 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:42.822285  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:42.823032  2237 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:42.823974  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:42.824569  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:42.729858  8098 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:42.831518  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 300093us
I20260709 13:02:42.856021  2201 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:42.832206  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:42.857537  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:42.856729  2228 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:42.860074  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.771410  8084 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:42.870599  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:42.872758  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:42.870811  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:42.873380  1616 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:42.876773  1616 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:02:42.716022  8588 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.899688  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:02:42.896013  8582 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.003s
I20260709 13:02:42.890851  8409 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:42.891268  8009 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:42.806420  8590 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:42.871923  8098 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:42.903741  8355 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:42.901948  8574 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:42.878556  8578 consensus_queue.cc:1059] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:02:42.879810  8084 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.936303  8409 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:42.950137  8587 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:42.912009  8306 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:42.975198  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 2418) took 1017 ms. Trace:
I20260709 13:02:42.998133  1612 rpcz_store.cc:276] 0709 13:02:41.958024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.958152 (+   128us) service_pool.cc:224] Handling call
0709 13:02:42.975022 (+1016870us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b54002dc3d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:42.975134 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1257344768}
W20260709 13:02:42.980156  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:42.987578  1609 raft_consensus.cc:2410] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 20 because replica has last-logged OpId of term: 19 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:02:42.988111  1614 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:42.976707  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:42.943856  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:42.934710  8600 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:02:42.994972  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 255015us
I20260709 13:02:42.989760  8579 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:42.948927  8582 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:42.970036  8176 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:42.986009  8098 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:43.019861  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 76154.4us
I20260709 13:02:43.058295  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:43.023835  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 218461us
I20260709 13:02:42.999331  8409 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:42.973906  8588 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.061492  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:42.997918  8306 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:42.971972  8025 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.063709  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:43.006345  8587 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:43.080776  8306 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:43.084323  8098 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:43.032969  8579 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:43.062510  8604 raft_consensus.cc:3055] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:43.071484  1618 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:43.066561  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:42.969398  8192 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.098062  8192 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.021235  8527 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:43.057431  8578 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:43.101665  8527 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.102121  8527 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:43.102017  8578 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.104220  8578 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:43.072508  8600 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 13:02:43.107659  8600 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:43.108204  8600 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:43.109791  8587 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.116811  8524 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:43.117376  8524 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:43.122731  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.086364  8237 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.095366  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:43.127915  8375 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.129567  8584 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:43.138448  8584 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:43.138896  8584 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:43.143518  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 312254us
I20260709 13:02:43.096138  8176 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:02:43.148193  8610 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:02:43.094141  8599 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:43.126493  8604 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:43.081837  8556 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.093091  8187 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:43.149979  8596 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:02:43.157768  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:43.151455  8176 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:43.187698  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:43.191134  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46579.8us
W20260709 13:02:43.192564  8297 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.803s	user 0.009s	sys 0.005s
I20260709 13:02:43.193061  8297 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.195401  8297 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.196067  8176 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:43.198372  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:43.162757  8599 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.201514  2203 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:43.159162  8306 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:43.219245  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.127256  8600 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.221364  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:43.200589  8497 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.587s	user 0.010s	sys 0.007s
I20260709 13:02:43.234097  1689 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:43.197723  8578 raft_consensus.cc:2764] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:02:43.237214  1535 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:02:43.238263  1535 pending_rounds.cc:85] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 11
I20260709 13:02:43.238576  1535 pending_rounds.cc:107] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:02:43.238860  1535 raft_consensus.cc:2889] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:43.252812  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.156961  8579 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.234773  8613 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:43.282945  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:43.290417  1689 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:43.207129  8599 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.264879  8547 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.319998  8556 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:43.275125  8578 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:43.283587  8579 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:43.308400  8582 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.307651  8098 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:43.332867  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 476910us
I20260709 13:02:43.236433  8297 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:43.299837  8084 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.308756  8025 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:43.319334  1686 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.013s	sys 0.019s
I20260709 13:02:43.237812  8604 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:43.338459  1615 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 2421) took 1379 ms (client timeout 1601 ms). Trace:
I20260709 13:02:43.327764  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3338) took 1013 ms. Trace:
I20260709 13:02:43.237126  8602 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:43.299458  8584 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:43.339025  1615 rpcz_store.cc:269] 0709 13:02:41.958785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.964628 (+  5843us) service_pool.cc:224] Handling call
0709 13:02:43.338163 (+1373535us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54003c87d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:43.338375 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360191232}
I20260709 13:02:43.237746  8590 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.340003  1690 rpcz_store.cc:276] 0709 13:02:42.313957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.440906 (+126949us) service_pool.cc:224] Handling call
0709 13:02:42.441294 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.441605 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:02:42.592158 (+150553us) log.cc:844] Serialized 14 byte log entry
0709 13:02:43.327395 (+735237us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:43.327459 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:02:43.327508 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.327575 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.327707 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":436508,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":106975,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":5195,"spinlock_wait_cycles":19931520,"thread_start_us":867289,"threads_started":3,"wal-append.queue_time_us":484945}
W20260709 13:02:43.334764  1686 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.013s	sys 0.020s
I20260709 13:02:43.328122  8578 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:43.346139  8607 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:02:43.346200  8187 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.347414  8306 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:43.325701  8547 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.336082  8098 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:43.348842  8608 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:02:43.335469  8497 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.340550  8620 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:43.325192  8409 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.334044  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:43.379475  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 45520.2us
I20260709 13:02:43.364194  8578 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:43.361968  1536 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
W20260709 13:02:43.348248  8615 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:02:43.318106  8613 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.360680  8628 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.344954  8025 consensus_queue.cc:1059] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 13:02:43.381872  8497 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.395812  8497 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:43.396519  8563 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.783s	user 0.006s	sys 0.009s
I20260709 13:02:43.397533  8547 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.378309  8620 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.399180  8620 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:43.399712  8592 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.490s	user 0.001s	sys 0.002s
I20260709 13:02:43.401365  8545 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.403190  8098 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:43.385262  8565 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:43.404284  8565 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.372926  8084 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.405781  8084 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.407298  8565 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.407792  8084 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:43.408219  1584 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:43.409154  8584 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.409498  8409 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:43.410182  8409 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:43.410689  8409 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:43.411654  8409 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.361584  8527 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.414356  8385 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:43.415624  8527 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.416831  8599 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:43.417829  8591 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.456s	user 0.000s	sys 0.003s
I20260709 13:02:43.419040  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2825) took 1155 ms. Trace:
I20260709 13:02:43.419584  1609 rpcz_store.cc:276] 0709 13:02:42.263596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.636111 (+372515us) service_pool.cc:224] Handling call
0709 13:02:43.418768 (+782657us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b540021ff54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:43.418963 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":804085120}
I20260709 13:02:43.422195  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2817) took 2532 ms. Trace:
I20260709 13:02:43.423890  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2425) took 1429 ms. Trace:
I20260709 13:02:43.422766  1617 rpcz_store.cc:276] 0709 13:02:40.890202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:40.890794 (+   592us) service_pool.cc:224] Handling call
0709 13:02:40.891120 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:40.901681 (+ 10561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:40.901710 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:41.799339 (+897629us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b48001a01d8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:02:42.196895 (+397556us) spinlock_profiling.cc:243] Waited 28.2 ms on lock 0x7b2c0005b3b8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:02:42.197693 (+   798us) log.cc:844] Serialized 52 byte log entry
0709 13:02:42.581033 (+383340us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:42.581091 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.581183 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.399076 (+817893us) spinlock_profiling.cc:243] Waited 84.6 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:43.421899 (+ 22823us) raft_consensus.cc:1685] finished
0709 13:02:43.421931 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.422121 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10174,"raft.queue_time_us":51038,"raft.run_cpu_time_us":7066,"raft.run_wall_time_us":850877,"spinlock_wait_cycles":311274496,"thread_start_us":382815,"threads_started":1,"wal-append.queue_time_us":657159}
I20260709 13:02:43.424291  1616 rpcz_store.cc:276] 0709 13:02:41.994782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.877794 (+883012us) service_pool.cc:224] Handling call
0709 13:02:42.878204 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.887492 (+  9288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.887538 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.895020 (+  7482us) log.cc:844] Serialized 26 byte log entry
0709 13:02:43.249064 (+354044us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:02:43.266343 (+ 17279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.266429 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.423689 (+157260us) raft_consensus.cc:1685] finished
0709 13:02:43.423709 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.423828 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8813,"raft.queue_time_us":12124,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":9702,"thread_start_us":312960,"threads_started":1,"wal-append.queue_time_us":462420}
W20260709 13:02:43.425549  8611 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.157s	user 0.002s	sys 0.000s
I20260709 13:02:43.426467  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1682) took 1465 ms. Trace:
I20260709 13:02:43.426713  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3347) took 1065 ms. Trace:
I20260709 13:02:43.427227  1685 rpcz_store.cc:276] 0709 13:02:42.361770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.501353 (+139583us) service_pool.cc:224] Handling call
0709 13:02:42.508843 (+  7490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.599912 (+ 91069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:42.599957 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.600512 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.992573 (+392061us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:42.992814 (+   241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.103744 (+110930us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b54000f6254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:43.103815 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.424980 (+321165us) spinlock_profiling.cc:243] Waited 6.1 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:43.426487 (+  1507us) raft_consensus.cc:1685] finished
0709 13:02:43.426509 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.426650 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2956,"raft.queue_time_us":4876,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3404,"spinlock_wait_cycles":222608000,"thread_start_us":359270,"threads_started":1,"wal-append.queue_time_us":581018}
I20260709 13:02:43.430106  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 1252) took 1275 ms. Trace:
I20260709 13:02:43.369735  8604 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:43.430670  1618 rpcz_store.cc:276] 0709 13:02:42.155157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.715431 (+560274us) service_pool.cc:224] Handling call
0709 13:02:43.429835 (+714404us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b5400389854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:43.430023 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":811469184}
I20260709 13:02:43.426882  2231 rpcz_store.cc:276] 0709 13:02:41.961258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:41.961417 (+   159us) service_pool.cc:224] Handling call
0709 13:02:41.987102 (+ 25685us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:41.987567 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:02:41.987899 (+   332us) log.cc:844] Serialized 14 byte log entry
0709 13:02:42.629055 (+641156us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:42.633827 (+  4772us) log.cc:844] Serialized 26 byte log entry
0709 13:02:42.633974 (+   147us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:42.634006 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:42.634082 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:43.382587 (+748505us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:43.426164 (+ 43577us) raft_consensus.cc:1685] finished
0709 13:02:43.426258 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.426385 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6006,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":13938,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":14882,"spinlock_wait_cycles":71204224,"thread_start_us":640724,"threads_started":1,"wal-append.queue_time_us":921495}
I20260709 13:02:43.432957  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3405) took 1132 ms. Trace:
I20260709 13:02:43.341804  8297 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.463433  8297 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.467545  8613 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.381371  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3346) took 1031 ms. Trace:
I20260709 13:02:43.377738  8306 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:43.391682  8338 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.392071  8628 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:43.401613  8617 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.136s	user 0.004s	sys 0.000s
I20260709 13:02:43.433385  1541 rpcz_store.cc:276] 0709 13:02:42.300245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.535056 (+234811us) service_pool.cc:224] Handling call
0709 13:02:42.535506 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:42.590735 (+ 55229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.432687 (+841952us) spinlock_profiling.cc:243] Waited 54.4 ms on lock 0x7b54000ee2d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:43.432897 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":125008768}
I20260709 13:02:43.433144  8461 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:43.477625  1686 rpcz_store.cc:276] 0709 13:02:42.350053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.514841 (+164788us) service_pool.cc:224] Handling call
0709 13:02:42.525613 (+ 10772us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:42.525930 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:42.813548 (+287618us) log.cc:844] Serialized 14 byte log entry
0709 13:02:43.380956 (+567408us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:43.381007 (+    51us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:43.381050 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.381128 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:43.381316 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":383297,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":2229,"raft.queue_time_us":1574,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":1441,"thread_start_us":800033,"threads_started":2,"wal-append.queue_time_us":605370}
I20260709 13:02:43.386518  8176 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.482748  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1448) took 1479 ms. Trace:
W20260709 13:02:43.484514  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 50572.6us
I20260709 13:02:43.491083  1584 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:43.486316  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 406702us
I20260709 13:02:43.396965  8563 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:40.750850   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:43.489187  1536 rpcz_store.cc:276] 0709 13:02:42.003801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:42.004042 (+   241us) service_pool.cc:224] Handling call
0709 13:02:42.004362 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:43.381285 (+1376923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:43.482372 (+101087us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400343ad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:43.482658 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3121747200}
I20260709 13:02:43.497342  1584 leader_election.cc:400] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 22. Current term is 23.
I20260709 13:02:43.497897  1584 leader_election.cc:403] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:02:43.492625  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:43.504281  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:43.512586  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:43.488487  8577 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.962s	user 0.013s	sys 0.039s
I20260709 13:02:43.512900  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:43.518371  1543 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:43.513401  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:43.515282  1535 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 22: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:43.519800  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:43.521317  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:43.522101  1691 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:43.514744  8623 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:43.494577  8461 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.508199  8385 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:02:43.503647  8418 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:43.581184  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 61438.6us
I20260709 13:02:43.573669  1543 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:43.596552  8623 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.618062  8623 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:43.619700  8619 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:43.622745  8619 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:43.623306  8619 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:43.549825  8625 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.551580  8628 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.631366  8623 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.632755  8636 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.633473  8636 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.634080  8623 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:43.636204  8636 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.514941  8547 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 22. Current term is 23.
I20260709 13:02:43.544546  8563 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.529486  8192 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:43.510627  8556 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:02:43.517877  8577 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.014s	sys 0.040s
I20260709 13:02:43.535323  8176 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:43.646661  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 363795us
I20260709 13:02:43.628836  8625 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.640422  8525 consensus_queue.cc:1059] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:02:43.706267  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:43.708767  1657 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:43.693805  8385 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:02:43.510802  8464 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.173s
I20260709 13:02:43.695647  8461 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:43.510910  8084 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.173s
I20260709 13:02:43.524256  8541 consensus_queue.cc:1059] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.175s
W20260709 13:02:43.697461  8635 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:02:43.706856  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:43.646803  8560 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:43.648161  8582 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.714526  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:43.684841  8524 consensus_meta.cc:287] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.567s	user 0.010s	sys 0.002s
I20260709 13:02:43.720387  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:43.721036  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:43.721526  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:43.736701  1685 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:43.726435  1690 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:43.685189  8556 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.734723  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:43.740031  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:43.740584  1686 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:43.739136  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:43.740607  1534 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:43.742421  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.743213  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:43.744251  1618 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:43.745949  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:43.746763  1609 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:43.748056  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:43.749629  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:43.751257  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:43.752084  1612 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 24.
I20260709 13:02:43.745065  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:43.758129  1617 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:43.759927  8025 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:43.760367  8641 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:02:43.761334  8641 consensus_queue.cc:1059] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:43.761839  8025 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.762029  8641 consensus_queue.cc:1243] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:02:43.762553  8641 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:43.762372  8025 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.762897  8641 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:43.763767  8641 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.764531  8025 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:43.765057  8025 raft_consensus.cc:3055] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:43.766964  1616 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:02:43.712203  8385 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:43.765105  8641 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:43.781018  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.788074  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:43.698307  8192 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.788735  1613 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:43.789949  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:43.790079  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:43.795501  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:43.795800  1614 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:43.794027  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:43.794684  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:43.797384  1610 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:43.767122  8556 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:43.776465  8418 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.776228  8623 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:43.805317  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 419893us
W20260709 13:02:43.806996  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 426040us
I20260709 13:02:43.779320  8600 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:43.810626  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:43.824273  1534 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:43.825594  1614 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:43.777004  8025 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:43.834036  1618 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:02:43.767964  8524 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:43.834069  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 216536us
I20260709 13:02:43.835536  1609 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:43.739133  8560 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.765923  8582 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:43.845568  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68751.7us
I20260709 13:02:43.846112  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:43.850630  1613 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
W20260709 13:02:43.853904  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 145207us
I20260709 13:02:43.855016  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:43.856783  1657 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:43.861248  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:43.861898  2233 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:43.862105  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:43.810931  8418 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.864917  1657 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:43.866837  2201 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:43.868291  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:43.858647  8590 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.835788  8578 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.869120  2233 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:43.856407  8632 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:43.870303  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:43.870326  8628 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:43.877867  8574 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:43.878539  8574 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.837286  8600 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:43.881945  8638 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:02:43.883152  8643 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:43.883384  8574 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.883736  8600 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.867545  8648 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:43.884613  8648 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.884965  8648 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:43.885124  8620 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.886750  8524 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.887447  8620 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.887331  8590 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.883734  8643 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.888581  1657 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:43.889503  8590 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.889750  1657 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:43.892856  2231 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:02:43.893164  8643 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.894646  8176 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:43.894914  8643 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:43.900255  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:43.866932  8238 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:02:43.913712  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 46942.5us
I20260709 13:02:43.917408  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:43.861379  8297 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:02:43.920296  2201 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:43.895481  8176 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:43.906236  8524 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:43.928754  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:43.931133  1688 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:43.897747  8084 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:02:43.919986  8632 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:43.922554  8620 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:43.944741  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:43.942816  1508 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:43.943387  8625 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.945705  8620 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:43.947220  8620 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:43.916019  8464 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 13:02:43.914894  8025 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:43.948923  8643 consensus_queue.cc:1175] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "984c9089d9804652846f3caeb609d56e" responder_term: 16 status { last_received { term: 15 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:43.951120  8418 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.952353  8628 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:43.952595  8306 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:43.943558  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:43.955078  1688 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:43.923200  8176 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:43.958065  8176 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:43.961021  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:43.963053  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:43.964191  8461 raft_consensus.cc:3055] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:43.964670  8461 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:43.966710  8461 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.970631  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:43.971648  2229 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:02:43.972425  2235 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:43.973218  1508 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:43.974858  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:43.975266  1688 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:43.975673  2235 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:43.932384  8639 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:43.979228  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:43.980254  2232 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:43.982052  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:43.982834  2236 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 21
W20260709 13:02:43.985504  8649 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:02:43.985029  8620 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:43.986140  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:43.987104  2230 raft_consensus.cc:2368] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21.
I20260709 13:02:43.987615  8620 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.970189  8632 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:43.991992  8632 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:43.993242  8560 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:43.994011  8192 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.941695  8574 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:43.995607  8578 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.997941  8579 raft_consensus.cc:3055] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:43.998492  8579 raft_consensus.cc:740] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:43.999192  8582 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:43.999308  8579 consensus_queue.cc:260] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:44.002437  8556 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:44.002813  8648 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.004797  8648 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:44.005548  8579 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:44.007362  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:44.009074  8560 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:43.945019  8623 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:44.009788  8639 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:44.010341  8639 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:44.009824  8560 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:44.011658  8560 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:44.012936  8604 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.575s	user 0.010s	sys 0.000s
W20260709 13:02:44.018194  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 45051.2us
I20260709 13:02:44.021842  8574 raft_consensus.cc:2764] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:02:44.037158  2236 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
W20260709 13:02:44.025287  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 115391us
I20260709 13:02:44.020465  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:44.014400  8297 raft_consensus.cc:2764] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:02:44.004815  8556 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:44.052047  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:44.053112  1537 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 16: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:44.033581  8623 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:44.049212  8497 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:44.057502  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:44.066113  8619 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:44.047353  8590 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:44.076234  8385 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.077699  8338 raft_consensus.cc:3055] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:44.078441  8590 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.081447  8338 raft_consensus.cc:740] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:44.083132  8590 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:44.082545  8338 consensus_queue.cc:260] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.084607  8338 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:44.081041  8385 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:44.095546  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:44.080124  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:44.098565  2231 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:44.099526  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:44.099393  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:44.105957  8658 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:44.106521  8658 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.106935  8658 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:44.109813  2231 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:44.090026  8025 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:44.114476  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:44.122128  2235 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:44.115535  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:44.122689  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:44.116534  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:44.117491  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:44.123951  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:44.124433  1613 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:44.124553  1543 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:44.124794  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:44.125607  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.125942  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:44.118081  2230 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:44.127414  1541 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:44.127674  1609 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:44.128468  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.129140  1614 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:44.129395  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.126705  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:44.129966  1615 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:44.130843  1536 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:44.069779  8623 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:44.133980  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:44.134552  8623 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:44.137351  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:44.138798  8619 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:44.139350  1611 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:44.137683  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.137915  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:44.090473  8545 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:44.141520  1534 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:44.141727  8545 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:44.142189  8545 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:44.145013  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.145857  1614 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:44.146008  1612 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:02:44.130635  8403 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:44.147246  8403 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.151420  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:44.073729  8497 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 24 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:44.166419  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:44.171344  1534 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:44.178468  1618 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:44.182823  2237 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:44.174909  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.184532  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:44.185170  1536 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:44.186348  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:44.186978  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.187044  1612 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:44.193938  1617 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:44.100664  8574 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.199318  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.199739  2230 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:44.200004  1610 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:44.149330  8403 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:44.131922  8577 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:44.200759  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.210999  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:44.204002  2228 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:44.209203  2235 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:44.214169  1536 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:44.214254  2231 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
W20260709 13:02:44.216049  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 345797us
I20260709 13:02:44.216626  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:44.218408  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:44.222244  1612 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:44.223773  1617 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:02:44.228283  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:44.236526  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:44.237617  1541 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:02:44.239560  1535 raft_consensus.cc:2368] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:44.239657  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.240417  1610 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:44.240949  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.243005  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.243865  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:44.244206  1617 raft_consensus.cc:2368] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:44.242295  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.246126  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:44.141333  8176 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.257478  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.258810  1613 raft_consensus.cc:2410] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 24 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:02:44.247196  2235 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:44.260098  2228 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:02:44.260588  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:44.258831  1618 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:44.261158  1609 raft_consensus.cc:2376] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21. Re-sending same reply.
I20260709 13:02:44.261835  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.244551  2231 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:44.263837  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.263803  1618 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:02:44.266049  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.266925  1609 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:44.267792  1691 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:02:44.267668  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.268652  1692 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:44.268867  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:44.269546  1536 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:44.269945  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:44.271047  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:44.271852  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:44.272475  1541 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:44.274567  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:44.277606  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:44.204941  8574 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.248027  8654 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:44.279780  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:44.280392  2235 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:44.275393  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 384518us
I20260709 13:02:44.278841  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.284205  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:44.284669  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.285745  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.286113  1609 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:44.286444  1690 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:44.287397  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:44.289572  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:44.291489  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:44.291651  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.292212  1539 raft_consensus.cc:2368] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 20. Current term is 21.
I20260709 13:02:44.293020  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:44.290730  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:44.294384  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:44.295583  1539 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:02:44.296595  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.296767  1657 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:44.300312  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.303915  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:44.314827  1611 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:44.314886  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:44.316295  1691 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:44.308393  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:44.324718  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:44.326920  1685 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:44.260524  8577 consensus_queue.cc:1059] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
W20260709 13:02:44.350132  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.338121  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:44.351147  1534 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:44.329371  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:44.275925  7980 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:44.345364  2228 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:02:44.353711  2235 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:44.359422  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:44.359869  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.360144  1617 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:44.290647  8654 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.359215  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:44.300963  8636 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:44.380586  1536 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:44.298678  8574 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:44.388636  1611 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:44.280637  8563 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.083s
W20260709 13:02:44.400926  1438 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 343) took 57974 ms (client timeout 15000 ms). Trace:
I20260709 13:02:44.337997  8238 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:44.402166  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 213342us
W20260709 13:02:44.364249  8622 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:02:44.409842  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:44.365224  8403 raft_consensus.cc:2764] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:02:44.417707  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:44.412496  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.371706  8461 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:44.420248  1691 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:44.276391  8497 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.371951  8654 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:44.298971  8613 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.087s
I20260709 13:02:44.370934  8375 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.371297  8098 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:44.423892  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:44.371304  8648 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:44.371321  8632 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:44.431171  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:44.431730  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.444209  1508 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:44.435600  1617 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:44.458112  1683 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:44.458640  1508 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:44.365535  7980 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:44.477293  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:44.405726  1438 rpcz_store.cc:269] 0709 13:01:46.404942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:46.405200 (+   258us) service_pool.cc:224] Handling call
0709 13:02:44.378818 (+57973618us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:38.210548 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:38.223915 (+ 13367us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:38.223967 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:02:38.223990 (+    23us) tablet.cc:662] Decoding operations
0709 13:02:38.225523 (+  1533us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:38.225621 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:38.225665 (+    44us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:02:38.227593 (+  1928us) tablet.cc:701] Row locks acquired
0709 13:02:38.227669 (+    76us) write_op.cc:260] PREPARE: finished
0709 13:02:38.227968 (+   299us) write_op.cc:270] Start()
0709 13:02:38.228167 (+   199us) write_op.cc:276] Timestamp: P: 1783602158227877 usec, L: 0
0709 13:02:38.228216 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 13:02:38.334080 (+105864us) log.cc:844] Serialized 6347 byte log entry
0709 13:02:38.852274 (+518194us) op_driver.cc:464] REPLICATION: finished
0709 13:02:39.119006 (+266732us) write_op.cc:301] APPLY: starting
0709 13:02:39.136599 (+ 17593us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:39.873253 (+736654us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:39.873370 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:40.418657 (+545287us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:40.419187 (+   530us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 13:02:40.419271 (+    84us) write_op.cc:312] APPLY: finished
0709 13:02:40.419933 (+   662us) log.cc:844] Serialized 148 byte log entry
0709 13:02:40.546050 (+126117us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:40.586087 (+ 40037us) write_op.cc:454] Released schema lock
0709 13:02:40.592856 (+  6769us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:40.593387 (+   531us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:40.593465 (+    78us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":273635,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":215431,"apply.run_cpu_time_us":48417,"apply.run_wall_time_us":1490483,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71103,"num_ops":16,"prepare.queue_time_us":419242,"prepare.run_cpu_time_us":11138,"prepare.run_wall_time_us":277093,"raft.queue_time_us":294443,"raft.run_cpu_time_us":17564,"raft.run_wall_time_us":89828,"replication_time_us":623856,"thread_start_us":566632,"threads_started":5,"wal-append.queue_time_us":329953,"wal-append.run_cpu_time_us":76965,"wal-append.run_wall_time_us":2817992}]]}
I20260709 13:02:44.378459  8623 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 13:02:44.497939  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.500941  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:44.501946  1683 raft_consensus.cc:2368] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19.
I20260709 13:02:44.502702  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.382846  8658 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.506434  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:44.507440  1683 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:44.511781  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:44.513028  1683 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:44.420481  8574 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
W20260709 13:02:44.515642  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 220307us
W20260709 13:02:44.517580  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 368160us
I20260709 13:02:44.438333  8525 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:44.521684  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:44.533660  8576 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:44.543365  1539 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 25.
W20260709 13:02:44.555830  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 196263us
I20260709 13:02:44.437381  8375 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:44.574985  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 59458.4us
I20260709 13:02:44.575054  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:44.576292  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:44.581503  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 59913.1us
I20260709 13:02:44.512482  8297 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:44.582443  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:44.516263  1441 catalog_manager.cc:5697] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 11 to 12, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 12 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:44.520843  8600 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:44.472270  8648 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:44.472211  8098 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.630410  1441 catalog_manager.cc:5697] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 7 to 11, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 11 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
W20260709 13:02:44.590914  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 283692us
W20260709 13:02:44.595083  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:44.595813  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.597074  1685 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:44.599120  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.634436  1509 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:44.634949  1687 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:44.635849  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:44.636180  1683 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:44.636458  1687 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:02:44.596206  1583 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:44.586723  8665 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
I20260709 13:02:44.474977  8632 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:44.587203  8620 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:44.640143  2231 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:44.590097  8541 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:02:44.609560  8639 consensus_meta.cc:287] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.599s	user 0.009s	sys 0.002s
I20260709 13:02:44.610185  8556 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:44.623023  8604 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:44.649392  1441 catalog_manager.cc:5697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 8 to 9, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 9 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:02:44.664357  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.664359  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 89424.1us
I20260709 13:02:44.571550  8525 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.681285  1582 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:44.637387  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.609781  8176 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:44.683132  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.623404  8668 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.688161  1683 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:44.626248  8680 raft_consensus.cc:3055] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:44.625957  8600 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:44.648231  8590 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:44.595566  8659 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.103s
I20260709 13:02:44.724717  2233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:44.642120  8025 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:44.609404  8579 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 13:02:44.712550  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.713408  1441 catalog_manager.cc:5697] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:02:44.651850  8541 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:44.690644  8638 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:02:44.589926  8238 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.709648  8577 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.642645  8409 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.084s
I20260709 13:02:44.749300  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.750174  1683 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:44.729110  1684 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:44.709870  8448 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:44.691776  8680 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:44.705757  1687 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:44.745934  8375 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:44.724547  8646 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:44.762130  8646 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:44.762744  8646 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:44.680310  8639 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:44.741922  8338 consensus_meta.cc:287] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.651s	user 0.008s	sys 0.005s
I20260709 13:02:44.712599  8604 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:44.758491  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.765844  8604 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.766438  1687 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:02:44.764773  2229 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:44.767690  1687 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: []
W20260709 13:02:44.770107  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 187727us
I20260709 13:02:44.771837  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:44.751296  8664 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:02:44.731880  8682 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:02:44.780160  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:44.784646  1441 catalog_manager.cc:5697] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 12 to 14, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 14 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:02:44.789162  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:44.791186  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 295511us
I20260709 13:02:44.790772  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:44.792564  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:44.801010  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.802217  1687 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
W20260709 13:02:44.806594  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 215690us
I20260709 13:02:44.813714  2236 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:44.806730  1508 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:44.715066  8668 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.792904  8025 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:44.796399  8620 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:44.759050  8448 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 13:02:44.796200  8602 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:44.870870  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:44.830390  8670 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.325s	user 0.001s	sys 0.000s
I20260709 13:02:44.817876  8682 raft_consensus.cc:697] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:44.826102  7980 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:44.755393  8577 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.868779  8620 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:44.880235  1687 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:02:44.880837  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.831073  8547 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:44.882530  8699 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:44.818194  8600 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:44.871289  8602 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:44.797361  8541 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:44.885236  8385 consensus_meta.cc:287] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.768s	user 0.008s	sys 0.002s
I20260709 13:02:44.803619  8563 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.077s
I20260709 13:02:44.890821  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:44.905160  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 212498us
I20260709 13:02:44.932215  1616 raft_consensus.cc:2410] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 13:02:44.949617  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.950562  1688 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:44.952016  1688 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:44.955132  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:44.976133  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:44.977404  1658 leader_election.cc:400] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
I20260709 13:02:44.977753  1658 leader_election.cc:403] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:44.887635  8338 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:44.997924  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:45.035892  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 317779us
I20260709 13:02:44.895296  8620 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:45.049701  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:45.058125  8563 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:02:45.058774  8671 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.255s	user 0.000s	sys 0.000s
I20260709 13:02:45.058949  8563 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:02:45.059903  8525 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:45.059942  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1607) took 1005 ms. Trace:
W20260709 13:02:45.060704  8678 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
I20260709 13:02:45.061693  8590 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:45.060675  2232 rpcz_store.cc:276] 0709 13:02:44.054528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.054684 (+   156us) service_pool.cc:224] Handling call
0709 13:02:44.057637 (+  2953us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.071413 (+ 13776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.071472 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.072046 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 13:02:44.554130 (+482084us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:02:44.554289 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.554480 (+   191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.059609 (+505129us) raft_consensus.cc:1685] finished
0709 13:02:45.059640 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.059866 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13337,"raft.queue_time_us":13484,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":5580,"thread_start_us":481263,"threads_started":1,"wal-append.queue_time_us":679915}
I20260709 13:02:45.062170  8590 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:45.063030  8545 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.920s	user 0.010s	sys 0.000s
I20260709 13:02:45.063463  8545 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:45.065579  1614 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:45.070108  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1599) took 1096 ms. Trace:
I20260709 13:02:45.070562  2229 rpcz_store.cc:276] 0709 13:02:43.973676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.973956 (+   280us) service_pool.cc:224] Handling call
0709 13:02:43.974319 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.767573 (+793254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.069764 (+302191us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b5400630e54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:45.070027 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1817150336}
I20260709 13:02:45.080099  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:45.043697  8673 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.363s	user 0.001s	sys 0.001s
I20260709 13:02:44.956712  8176 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:44.956993  8659 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:44.927423  8602 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:45.127911  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:45.128937  1611 raft_consensus.cc:2393] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:45.130324  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:45.133286  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:45.133567  1610 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19. Ops: []
W20260709 13:02:45.060186  8687 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
I20260709 13:02:45.035284  8619 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:45.136607  1615 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 22: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:44.996870  8658 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:44.957350  8699 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:45.123131  1441 catalog_manager.cc:5697] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 8 to 9, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 9 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:02:45.003535  7980 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.145s
I20260709 13:02:45.154815  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:44.872428  8680 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:45.158362  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 365832us
W20260709 13:02:45.159302  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.111178  8403 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
W20260709 13:02:45.183823  1583 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.012s	sys 0.011s
I20260709 13:02:45.119268  8663 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.113829  8525 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:45.113510  8632 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:45.188082  1612 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:02:45.145524  1691 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.000s	sys 0.021s
I20260709 13:02:45.121400  8661 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:45.120527  8697 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:45.152475  8675 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.421s	user 0.000s	sys 0.001s
I20260709 13:02:45.189671  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:45.228127  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 2430) took 1104 ms. Trace:
I20260709 13:02:45.228740  1616 rpcz_store.cc:276] 0709 13:02:44.123633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.124585 (+   952us) service_pool.cc:224] Handling call
0709 13:02:45.227852 (+1103267us) spinlock_profiling.cc:243] Waited 806 ms on lock 0x7b5400104d54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:45.228046 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1854632192}
I20260709 13:02:45.114125  8338 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:45.200506  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 378016us
I20260709 13:02:45.186115  8545 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:45.134791  8666 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:45.210855  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:45.248837  1609 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:45.148783  8659 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.215905  8699 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.094534  8375 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.094s
I20260709 13:02:45.056665  8690 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:45.184231  8619 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 20 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:45.234905  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:45.238001  1508 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:45.201442  1583 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.012s	sys 0.011s
I20260709 13:02:45.243119  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:45.271941  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 675762us
I20260709 13:02:45.272380  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:45.273623  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:45.246317  8694 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 13:02:45.200878  8683 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
W20260709 13:02:45.214285  1691 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.001s	sys 0.021s
I20260709 13:02:45.213964  8525 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.261356  1612 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
W20260709 13:02:45.240533  8461 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.805s	user 0.011s	sys 0.001s
W20260709 13:02:45.248941  8582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.019s	sys 0.075s
W20260709 13:02:45.247651  8681 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 13:02:45.296221  1689 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:45.278724  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:45.281157  1583 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:45.281569  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:45.281751  1609 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 25.
I20260709 13:02:45.294644  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:45.304335  1690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.018s	sys 0.044s
W20260709 13:02:45.304872  1690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.707s	user 0.018s	sys 0.044s
I20260709 13:02:45.268507  8666 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:45.293910  8582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.019s	sys 0.076s
I20260709 13:02:45.219888  8663 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.290441  8525 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:45.314203  1583 leader_election.cc:400] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21.
I20260709 13:02:45.314939  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3365) took 1014 ms. Trace:
I20260709 13:02:45.314754  1441 catalog_manager.cc:5697] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 11 to 14, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } }
I20260709 13:02:45.272965  8639 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:45.278498  8625 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.354233  8625 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.356593  8625 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:45.291992  2201 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:45.335925  1690 rpcz_store.cc:276] 0709 13:02:44.300275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.339065 (+ 38790us) service_pool.cc:224] Handling call
0709 13:02:44.339403 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.339626 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:44.597449 (+257823us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.314022 (+716573us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.314081 (+    59us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:45.314136 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.314735 (+   599us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.314868 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1950,"prepare.queue_time_us":441382,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":17373,"thread_start_us":929848,"threads_started":2}
I20260709 13:02:45.335855  1583 leader_election.cc:403] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:02:45.358069  1537 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1470) took 1236 ms. Trace:
W20260709 13:02:45.358558  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 44574.2us
I20260709 13:02:45.358909  8663 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:45.229493  8661 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:45.358523  1537 rpcz_store.cc:276] 0709 13:02:44.121916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.123513 (+  1597us) service_pool.cc:224] Handling call
0709 13:02:44.123825 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.518767 (+394942us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.518836 (+    69us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.533305 (+ 14469us) spinlock_profiling.cc:243] Waited 8.18 ms on lock 0x7b480030c858. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:02:44.535843 (+  2538us) log.cc:844] Serialized 26 byte log entry
0709 13:02:44.790658 (+254815us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:44.790699 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.178787 (+388088us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54004187d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:45.178877 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.357840 (+178963us) raft_consensus.cc:1685] finished
0709 13:02:45.357866 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.358010 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":56525,"raft.run_cpu_time_us":2493,"raft.run_wall_time_us":168446,"spinlock_wait_cycles":436433024,"thread_start_us":465678,"threads_started":2,"wal-append.queue_time_us":340541}
I20260709 13:02:45.359493  8663 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21.
I20260709 13:02:45.236500  8697 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.359831  8418 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.372499  8418 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.373131  1692 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:45.375324  8418 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:45.376227  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 84345.5us
I20260709 13:02:45.376384  8710 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:45.376925  8710 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:45.377326  8710 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:45.378943  8602 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.381203  8602 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:45.382777  1684 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:45.383775  8547 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:45.384284  8547 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:45.386121  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:45.389138  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:45.390040  1684 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
W20260709 13:02:45.394609  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.396795  1692 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:45.398167  1539 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:45.402787  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:45.403235  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2823) took 1115 ms. Trace:
W20260709 13:02:45.419531  8646 consensus_meta.cc:287] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.656s	user 0.013s	sys 0.001s
I20260709 13:02:45.331771  8636 raft_consensus.cc:3055] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:45.325906  8668 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:45.223960  8682 consensus_queue.cc:237] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:45.293581  8545 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:02:45.415228  2201 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:45.411372  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 44535.8us
I20260709 13:02:45.410184  1537 consensus_queue.cc:237] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:02:45.361292  8703 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:45.426398  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:45.448329  1538 raft_consensus.cc:2410] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:02:45.322543  8693 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:45.428773  1542 rpcz_store.cc:276] 0709 13:02:44.287869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.288021 (+   152us) service_pool.cc:224] Handling call
0709 13:02:44.288397 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.294777 (+  6380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.294812 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.304606 (+  9794us) log.cc:844] Serialized 26 byte log entry
0709 13:02:44.635545 (+330939us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:44.635587 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.635655 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.402973 (+767318us) raft_consensus.cc:1685] finished
0709 13:02:45.403000 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.403160 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5951,"raft.queue_time_us":6758,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":38941,"thread_start_us":292085,"threads_started":1,"wal-append.queue_time_us":625749}
W20260709 13:02:45.430011  1509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.795s	user 0.000s	sys 0.011s
W20260709 13:02:45.450533  1509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.000s	sys 0.012s
W20260709 13:02:45.450874  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 816488us
I20260709 13:02:45.430351  1541 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 22: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:45.451829  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:45.410120  8409 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:45.436784  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:45.443241  8692 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
W20260709 13:02:45.443542  8700 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:02:45.474853  1582 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:45.344666  8338 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.431082  1687 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 21. Current term is 22. Ops: [21.15-21.15]
W20260709 13:02:45.449465  8654 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.975s	user 0.004s	sys 0.009s
I20260709 13:02:45.454629  8409 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.463197  8648 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:45.463850  8590 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:45.455094  8683 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:45.467221  8541 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:45.454663  8693 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:02:45.467767  8695 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 13:02:45.469319  8556 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:45.469743  8661 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:45.470955  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.421169  8625 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:02:45.449040  1507 leader_election.cc:400] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19.
I20260709 13:02:45.555559  1507 leader_election.cc:403] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:02:45.490303  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:45.445349  8704 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.450855  8602 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:02:45.501637  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:45.433355  8636 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:45.537035  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 2445) took 1374 ms. Trace:
I20260709 13:02:45.539005  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2373) took 1107 ms. Trace:
I20260709 13:02:45.569839  1689 rpcz_store.cc:276] 0709 13:02:44.431388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.431502 (+   114us) service_pool.cc:224] Handling call
0709 13:02:45.538767 (+1107265us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b54000aa5d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:45.538921 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1958020992}
I20260709 13:02:45.571393  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:45.572343  1689 raft_consensus.cc:2410] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:02:45.426070  8619 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.447641  8703 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:45.472860  8620 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.457875  8709 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.122s
I20260709 13:02:45.557992  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:45.574025  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:45.448079  8646 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:45.550175  8297 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.961s	user 0.008s	sys 0.004s
I20260709 13:02:45.592783  8098 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:45.594929  8702 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.232s	user 0.000s	sys 0.000s
I20260709 13:02:45.551192  8654 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:45.582032  8579 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:45.601410  8237 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:45.603210  8409 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:45.446025  8666 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:45.611189  8680 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:45.588002  8577 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:45.566418  8574 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19.
I20260709 13:02:45.562222  8713 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:45.612808  8693 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.569676  8683 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 21 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:45.570240  8602 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:45.616166  8697 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:45.570758  8661 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.618008  8636 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.571122  8704 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:45.604836  8707 raft_consensus.cc:3055] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:45.599383  8659 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:45.588955  8703 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:45.584923  8716 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:45.551887  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2374) took 1104 ms. Trace:
W20260709 13:02:45.563048  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 114213us
I20260709 13:02:45.611507  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:45.563695  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 109585us
I20260709 13:02:45.569331  1614 rpcz_store.cc:276] 0709 13:02:44.162707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.165769 (+  3062us) service_pool.cc:224] Handling call
0709 13:02:45.536766 (+1370997us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b5400001c54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:45.536936 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1966699392}
I20260709 13:02:45.576735  1684 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:45.543450  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:45.585976  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2369) took 1156 ms. Trace:
W20260709 13:02:45.628562  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 436719us
I20260709 13:02:45.562144  8556 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:45.611763  8590 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:45.846418  1506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.208s	user 0.000s	sys 0.030s
W20260709 13:02:45.908236  1688 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.010s	sys 0.005s
W20260709 13:02:45.914717  1690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.004s	sys 0.014s
W20260709 13:02:45.957242  2235 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.001s	sys 0.009s
W20260709 13:02:45.853716  1617 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.015s	sys 0.021s
W20260709 13:02:45.870368  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:45.626256  8604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.042s	sys 0.058s
I20260709 13:02:45.936661  2231 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:45.958637  2232 raft_consensus.cc:1240] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:45.854674  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2457) took 1589 ms. Trace:
I20260709 13:02:45.889739  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3366) took 1589 ms. Trace:
I20260709 13:02:45.846894  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2824) took 1558 ms. Trace:
I20260709 13:02:45.626065  8297 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:46.020432  1506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.383s	user 0.003s	sys 0.037s
W20260709 13:02:46.023437  2235 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.005s	sys 0.011s
W20260709 13:02:46.023207  1690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.005s	sys 0.023s
W20260709 13:02:46.022717  1688 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.017s	sys 0.008s
I20260709 13:02:46.028302  8699 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:46.028831  8625 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 13:02:46.029825  8098 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.029495  8699 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.031011  8098 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.032290  8708 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:46.032918  8625 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:46.033321  8708 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.033921  8448 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:46.034674  8604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.042s	sys 0.065s
I20260709 13:02:45.648371  8237 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:46.035873  8636 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:46.036077  1683 rpcz_store.cc:276] 0709 13:02:44.447405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.751962 (+304557us) service_pool.cc:224] Handling call
0709 13:02:44.752284 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.775072 (+ 22788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.775110 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.775622 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 13:02:45.126917 (+351295us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b1800421960. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:02:45.457031 (+330114us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:45.457070 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.457139 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.457186 (+    47us) raft_consensus.cc:1685] finished
0709 13:02:45.457199 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.551801 (+ 94602us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22071,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":11740,"spinlock_wait_cycles":14446848,"thread_start_us":329835,"threads_started":1,"wal-append.queue_time_us":676976}
I20260709 13:02:46.037981  8661 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.040165  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:45.849573  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 394330us
I20260709 13:02:46.041790  8602 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:46.042110  8716 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:45.780478  8646 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.014256  1691 rpcz_store.cc:276] 0709 13:02:44.429591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.429766 (+   175us) service_pool.cc:224] Handling call
0709 13:02:44.430088 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.438505 (+  8417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.438535 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.438984 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:02:44.608488 (+169504us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b1800457300. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:02:45.290081 (+681593us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:45.290129 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.290199 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.572472 (+282273us) raft_consensus.cc:1685] finished
0709 13:02:45.572501 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.585881 (+ 13380us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8002,"raft.queue_time_us":27746,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":36537,"spinlock_wait_cycles":5589888,"thread_start_us":656571,"threads_started":1,"wal-append.queue_time_us":778522}
I20260709 13:02:45.964370  8025 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:45.780014  8654 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:45.875284  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:46.046586  8704 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:46.046800  8707 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:46.049703  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.025893  1617 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.020s	sys 0.025s
I20260709 13:02:46.098742  1685 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:46.050711  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:46.051574  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.102495  8707 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.104169  8707 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:46.106170  8448 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.053203  8683 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.055013  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:46.055824  8560 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 13:02:46.057462  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 513949us
I20260709 13:02:46.057910  8409 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:46.118534  8409 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.068461  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3370) took 1110 ms. Trace:
I20260709 13:02:46.035899  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:46.121563  1611 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.009s	sys 0.017s
W20260709 13:02:46.122186  1611 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.009s	sys 0.017s
W20260709 13:02:46.092967  8741 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 13:02:46.120916  1688 rpcz_store.cc:276] 0709 13:02:44.958531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.963991 (+  5460us) service_pool.cc:224] Handling call
0709 13:02:44.964409 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.967082 (+  2673us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:45.035089 (+ 68007us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.390458 (+355369us) spinlock_profiling.cc:243] Waited 7.64 ms on lock 0x7b1800479620. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a 00007fe08faf6a2a
0709 13:02:46.067762 (+677304us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.067921 (+   159us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:46.068042 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.068164 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.068384 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37296,"prepare.queue_time_us":67958,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":67,"raft.queue_time_us":41946,"spinlock_wait_cycles":17563008,"thread_start_us":636239,"threads_started":1,"wal-append.queue_time_us":729480}
I20260709 13:02:46.098758  1509 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:02:46.024189  1583 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.015s	sys 0.026s
I20260709 13:02:45.645768  8579 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:46.046618  8620 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:45.766073  8710 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.132237  1683 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:46.087083  8697 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.133889  8697 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.087234  8613 consensus_queue.cc:1059] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:46.088100  8713 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:46.020836  1540 rpcz_store.cc:276] 0709 13:02:44.288492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.288673 (+   181us) service_pool.cc:224] Handling call
0709 13:02:44.288958 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.405531 (+116573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.405576 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.419637 (+ 14061us) log.cc:844] Serialized 26 byte log entry
0709 13:02:44.878834 (+459197us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:44.879336 (+   502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.881593 (+  2257us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.845694 (+964101us) raft_consensus.cc:1685] finished
0709 13:02:45.845804 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.846435 (+   631us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9171,"raft.queue_time_us":72426,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":14338,"thread_start_us":470070,"threads_started":2,"wal-append.queue_time_us":748930}
I20260709 13:02:46.019840  1613 rpcz_store.cc:276] 0709 13:02:44.264706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.265662 (+   956us) service_pool.cc:224] Handling call
0709 13:02:44.281486 (+ 15824us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:44.514772 (+233286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:44.514801 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:44.515268 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:02:44.818629 (+303361us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:02:44.818670 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:44.818751 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:45.853947 (+1035196us) raft_consensus.cc:1685] finished
0709 13:02:45.854006 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.854385 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":308407,"raft.queue_time_us":234236,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":40095,"thread_start_us":303055,"threads_started":1,"wal-append.queue_time_us":763791}
I20260709 13:02:46.072817  8625 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:46.098049  1657 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.739s (SENT) [suppressed 18 similar messages]
I20260709 13:02:46.019984  1686 rpcz_store.cc:276] 0709 13:02:44.300564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.300741 (+   177us) service_pool.cc:224] Handling call
0709 13:02:44.326658 (+ 25917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:44.518621 (+191963us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:02:45.382863 (+864242us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b180039eda0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08f6944bc 00007fe08c9e248b 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fc455 00007fe0934e43c7 00007fe08faf69a7 00007fe08fb060bf
0709 13:02:45.382999 (+   136us) log.cc:844] Serialized 14 byte log entry
0709 13:02:45.858373 (+475374us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:45.858506 (+   133us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:45.858628 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:45.889117 (+ 30489us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b540014a854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:45.889191 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:45.889507 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5687,"prepare.queue_time_us":284684,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":99537,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":378,"spinlock_wait_cycles":449841280,"thread_start_us":766597,"threads_started":2,"wal-append.queue_time_us":392522}
I20260709 13:02:46.113762  8560 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:46.092850  2233 raft_consensus.cc:606] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Received request to transfer leadership to TS 984c9089d9804652846f3caeb609d56e
I20260709 13:02:46.116993  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:46.069684  8577 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.139390  1583 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.016s	sys 0.026s
W20260709 13:02:46.149571  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 784579us
W20260709 13:02:46.097303  1441 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.017s	sys 0.018s
I20260709 13:02:46.102614  8237 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:46.152743  1441 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.018s	sys 0.018s
W20260709 13:02:46.156658  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 58275.2us
I20260709 13:02:46.172056  8579 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:46.173646  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 51740.1us
W20260709 13:02:46.174616  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 98885.4us
I20260709 13:02:46.176996  8577 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.177515  8710 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.153754  8237 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:46.210235  8639 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.176406  8654 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:02:46.235711  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 59860us
I20260709 13:02:46.122444  8716 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:46.211539  1692 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.590s	user 0.008s	sys 0.020s
I20260709 13:02:46.097781  8025 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.249902  1692 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:46.251135  7980 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:46.252031  8025 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.226153  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2848) took 1094 ms. Trace:
I20260709 13:02:46.229557  8297 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.082s
W20260709 13:02:46.249874  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 65914.8us
I20260709 13:02:46.252907  1617 rpcz_store.cc:276] 0709 13:02:45.131995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.132144 (+   149us) service_pool.cc:224] Handling call
0709 13:02:45.132436 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.132649 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:02:46.133665 (+1001016us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.225582 (+ 91917us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.225684 (+   102us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:46.225798 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.225903 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.226089 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64284,"prepare.queue_time_us":623517,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":67234,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"thread_start_us":1002471,"threads_started":2}
I20260709 13:02:46.255045  7980 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.254601  8625 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.255666  7980 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:46.256593  8625 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:46.259999  8697 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.235023  1685 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
W20260709 13:02:46.178433  8238 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.000s	sys 0.009s
W20260709 13:02:46.238706  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 203364us
I20260709 13:02:46.238437  8639 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.246038  1584 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:46.246459  8563 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:46.248663  8600 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:46.198072  8699 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.271438  8448 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:46.274820  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 51545.8us
I20260709 13:02:46.229583  8709 raft_consensus.cc:3055] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:46.229616  8704 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:46.274127  8238 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.000s	sys 0.010s
I20260709 13:02:46.261658  8098 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:46.272432  8600 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.286083  8563 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:46.294590  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:46.294651  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 48803.5us
I20260709 13:02:46.294727  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:46.284211  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:46.293743  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 195895us
I20260709 13:02:46.290306  8600 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:46.299852  8602 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.301977  1685 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:02:46.302275  8730 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:46.302799  1507 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:46.303759  8547 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.919s	user 0.007s	sys 0.010s
I20260709 13:02:46.303958  8574 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:46.303759  2236 raft_consensus.cc:1240] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:02:46.304358  8547 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.302093  8646 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 13:02:46.305528  8666 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:02:46.306743  8541 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.742s	user 0.012s	sys 0.007s
I20260709 13:02:46.307684  8409 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.307852  7980 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.308663  8713 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.294746  8661 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:46.310698  8708 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.310763  8602 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.311656  8560 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.313570  8560 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:46.313216  8563 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.305122  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:46.316910  8297 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:46.323025  1582 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:46.323447  8297 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.324122  8541 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:46.324700  8541 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:46.305588  8098 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:46.326057  8098 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.326102  8541 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.283758  8709 raft_consensus.cc:740] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:02:46.337263  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 45249.9us
I20260709 13:02:46.322501  8238 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:46.329159  8098 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:46.342609  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:46.340824  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 188740us
I20260709 13:02:46.321570  8704 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.380983  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:46.382318  1537 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:46.384251  8613 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:46.381338  7980 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:46.363632  8238 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:46.347332  8574 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:02:46.386840  1692 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:46.396027  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:46.396325  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:46.390208  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 273497us
I20260709 13:02:46.408912  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:46.391434  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:46.398260  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 68319.7us
W20260709 13:02:46.390838  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 158221us
W20260709 13:02:46.407757  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.420027  1688 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:46.397909  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:46.407625  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:46.388676  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:40336 (request call id 22) took 1664 ms. Trace:
I20260709 13:02:46.461591  2233 rpcz_store.cc:276] 0709 13:02:44.724382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.724551 (+   169us) service_pool.cc:224] Handling call
0709 13:02:46.388411 (+1663860us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54002a3fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fae6853 00007fe094ab460a 00007fe08e03a7d4 00007fe08e03a762 00007fe08e03a691 00007fe08e03a60c 00007fe08e0398c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:46.388543 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3144503040}
I20260709 13:02:46.396662  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:46.468511  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.415755  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:46.407104  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50372.3us
I20260709 13:02:46.395687  8579 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:46.393548  1618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2842) took 2126 ms (client timeout 2040 ms). Trace:
I20260709 13:02:46.419152  8613 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.385370  8625 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:46.393486  8025 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:46.502094  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:46.503248  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.455399  1687 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:46.390539  8577 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.505760  8574 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.505920  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:46.410710  8238 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.506392  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:46.507643  8547 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.510675  8703 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.511626  1691 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:46.511451  8720 consensus_queue.cc:1059] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:46.402140  8709 consensus_queue.cc:260] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.515882  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:46.497378  1618 rpcz_store.cc:269] 0709 13:02:44.267273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.269537 (+  2264us) service_pool.cc:224] Handling call
0709 13:02:46.393162 (+2123625us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b5400190854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:46.393320 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4667709824}
W20260709 13:02:46.505990  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.538920  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 83529.4us
I20260709 13:02:46.395071  8704 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:46.530359  1538 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:46.533855  1541 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:46.538177  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:46.544121  8574 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.546197  8237 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:46.546694  8237 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:46.547117  8237 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:46.549036  8600 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.550879  1688 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:46.552613  8600 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.539919  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:46.409332  8297 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.550256  8709 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:46.550897  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:46.399164  8639 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.416606  8721 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.523334  8613 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:02:46.401587  8680 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.697s	user 0.011s	sys 0.008s
I20260709 13:02:46.537951  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2852) took 1389 ms. Trace:
I20260709 13:02:46.556888  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:46.583213  2201 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:46.572955  1583 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:46.589283  1683 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:46.590216  1658 leader_election.cc:400] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17.
W20260709 13:02:46.595192  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.555263  8625 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.597458  1686 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:46.596171  1537 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 14.
I20260709 13:02:46.576853  8574 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:46.601272  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 58734.2us
I20260709 13:02:46.601464  1685 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
W20260709 13:02:46.602291  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 74219.9us
I20260709 13:02:46.597033  8590 raft_consensus.cc:3055] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:46.569180  1611 rpcz_store.cc:276] 0709 13:02:45.150640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.151218 (+   578us) service_pool.cc:224] Handling call
0709 13:02:45.198196 (+ 46978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:45.421826 (+223630us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:02:46.122806 (+700980us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.341358 (+218552us) log.cc:844] Serialized 14 byte log entry
0709 13:02:46.349990 (+  8632us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:46.353807 (+  3817us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:02:46.356800 (+  2993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.499690 (+142890us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54003c8cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:46.499782 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:46.537895 (+ 38113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":398858,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":4073,"raft.queue_time_us":5209,"raft.run_cpu_time_us":403,"raft.run_wall_time_us":15855,"spinlock_wait_cycles":513631488,"thread_start_us":849352,"threads_started":2,"wal-append.queue_time_us":256446}
I20260709 13:02:46.596572  8025 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:46.591164  8639 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:46.546242  2236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:46.615892  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:46.620298  8639 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.612972  1582 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:46.607263  8590 raft_consensus.cc:740] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:46.597621  8556 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.608577  8547 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:46.615177  8025 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:46.596446  8721 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:02:46.631389  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 66326.6us
W20260709 13:02:46.631750  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 240481us
I20260709 13:02:46.632046  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:46.626276  8620 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:02:46.634217  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.633852  2236 raft_consensus.cc:606] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
W20260709 13:02:46.633235  8693 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.008s	sys 0.015s
W20260709 13:02:46.635702  1508 consensus_peers.cc:597] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7ba0620abe844f5bb71ba9dd947c3b9c. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:02:46.641825  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 54552.8us
W20260709 13:02:46.642901  8752 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.089s	user 0.003s	sys 0.000s
I20260709 13:02:46.642937  1689 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:46.643587  8582 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:46.634845  8025 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:46.644073  8582 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.637579  8620 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.651319  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:46.652540  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3379) took 1095 ms. Trace:
I20260709 13:02:46.652839  8699 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:46.652951  1689 rpcz_store.cc:276] 0709 13:02:45.557478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.622314 (+ 64836us) service_pool.cc:224] Handling call
0709 13:02:45.622560 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:46.643669 (+1021109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:46.652282 (+  8613us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400028354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:46.652483 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2334049664}
W20260709 13:02:46.644677  8693 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.009s	sys 0.015s
W20260709 13:02:46.661749  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 62149.3us
I20260709 13:02:46.657886  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:46.654213  8620 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:46.659068  8590 consensus_queue.cc:260] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.683852  8625 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.684544  8625 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:46.685773  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:46.686712  1540 raft_consensus.cc:2368] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:46.648156  8582 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:46.709551  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:46.710789  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:46.716562  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:46.717692  1538 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:46.724623  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:46.725569  1689 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:46.685212  8625 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:46.729774  8525 consensus_meta.cc:287] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.360s	user 0.011s	sys 0.011s
I20260709 13:02:46.730315  8525 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.731534  1690 raft_consensus.cc:2368] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:46.731729  8625 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.732479  8525 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:46.733284  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 107294us
W20260709 13:02:46.736485  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:46.737174  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.738579  8683 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:46.754493  1541 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:46.754312  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:46.756119  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:46.715284  8703 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:46.770354  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:46.767957  8590 raft_consensus.cc:3060] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:46.775466  8721 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:46.776433  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 150563us
I20260709 13:02:46.778375  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:02:46.779165  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 124566us
I20260709 13:02:46.781548  8579 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:46.782176  2229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:46.764649  8704 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 13:02:46.786371  2229 raft_consensus.cc:606] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:46.786042  8639 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.790179  1582 leader_election.cc:400] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
I20260709 13:02:46.790529  1582 leader_election.cc:403] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 13:02:46.790570  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.789606  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:46.767531  8541 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:46.795156  1689 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:46.792320  8238 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:46.799628  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:46.800971  1541 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:02:46.793562  8577 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:46.802744  8577 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:46.803426  8238 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.803723  8577 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.805071  8577 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:46.810168  1683 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:46.805711  8238 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:46.814325  1688 raft_consensus.cc:2468] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
W20260709 13:02:46.814904  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 47763.7us
I20260709 13:02:46.818002  8625 raft_consensus.cc:993] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: : Instructing follower 984c9089d9804652846f3caeb609d56e to start an election
I20260709 13:02:46.818571  8625 raft_consensus.cc:1081] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Signalling peer 984c9089d9804652846f3caeb609d56e to start an election
I20260709 13:02:46.794476  8703 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:46.820643  8703 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.800860  8697 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:46.801048  8556 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:46.825114  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:46.828713  1538 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:46.826650  2201 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:46.829703  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 171128us
W20260709 13:02:46.834070  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 55851.5us
I20260709 13:02:46.816071  8525 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:46.827044  8713 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:46.823217  8541 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:46.829401  8697 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:46.849795  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:46.850996  8721 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:02:46.852031  8721 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:46.852415  8721 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:46.855129  8541 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.857578  8704 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:46.858213  8704 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:46.858543  1658 leader_election.cc:400] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 20. Current term is 21.
I20260709 13:02:46.858925  1658 leader_election.cc:403] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:02:46.859236  8704 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.861151  8648 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 20. Current term is 21.
I20260709 13:02:46.863679  2229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:46.864451  2229 raft_consensus.cc:606] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:46.864825  2229 raft_consensus.cc:614] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:46.842522  8708 raft_consensus.cc:2804] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:46.866310  8708 raft_consensus.cc:697] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:46.867429  8708 consensus_queue.cc:237] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.868893  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:46.822741  8579 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.873924  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:46.870888  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:46.874599  1690 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
W20260709 13:02:46.870617  8707 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.766s	user 0.009s	sys 0.005s
I20260709 13:02:46.869944  8697 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:46.884688  8556 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:02:46.885288  8556 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 25 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:46.845321  8238 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:46.889165  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 48797.2us
I20260709 13:02:46.891407  1507 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:46.893344  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:46.898576  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:46.881209  8025 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.901549  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:46.906893  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:46.908502  1543 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:02:46.910017  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:46.910228  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:46.912129  8648 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.912737  8648 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.891742  1582 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:46.918310  2229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:46.880527  1683 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb"
dest_uuid: "984c9089d9804652846f3caeb609d56e"
 from {username='slave'} at 127.0.0.1:40014
I20260709 13:02:46.919484  1683 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:46.919822  1683 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:46.914506  8646 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:46.882411  8703 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:46.926756  2229 raft_consensus.cc:606] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:46.927181  2229 raft_consensus.cc:614] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:46.925652  8693 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:46.931631  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 57390.4us
I20260709 13:02:46.853302  8721 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.929333  1539 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
I20260709 13:02:46.939883  1584 leader_election.cc:400] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Vote denied by peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459) with higher term. Message: Invalid argument: T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
W20260709 13:02:46.922551  8636 consensus_meta.cc:287] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.799s	user 0.010s	sys 0.004s
I20260709 13:02:46.949527  8613 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:46.927731  8547 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:46.891076  8237 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.954723  8648 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:46.928889  8639 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:46.926311  8646 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:46.961347  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 79524.5us
I20260709 13:02:46.931463  8693 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:46.968852   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:02:46.969672  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 77625.1us
I20260709 13:02:46.973177  1683 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.976467  2203 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:46.957789  8547 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:46.966878  8646 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:46.972813  8693 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:46.978612  8639 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.941540  8238 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.990949  8525 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:46.991849  8238 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:46.991604  8525 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:46.992372   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:46.987603  1683 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:46.995054  1507 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:46.980712  8547 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:47.010577  1584 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:47.020705  8699 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:47.021924  8560 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:47.025830  1542 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:47.030109  8590 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:47.030740  8590 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.032866  8590 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:47.014423  8683 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:02:47.062469  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 96052us
W20260709 13:02:47.063400  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 76157.7us
W20260709 13:02:47.068034  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 103419us
I20260709 13:02:46.993870  8556 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:47.062803  8579 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:47.076977  8769 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:02:47.077596  8639 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:47.087965  1613 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 14. (term mismatch)
I20260709 13:02:47.089036  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:47.092592  1618 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:47.096197  8707 raft_consensus.cc:993] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: : Instructing follower 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:02:47.096939  8707 raft_consensus.cc:1081] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Signalling peer 83f6f5810950417195f707ff308a8125 to start an election
I20260709 13:02:47.088443  1613 pending_rounds.cc:85] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 13
W20260709 13:02:47.090152  8770 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:02:47.089282  8525 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:47.103899  1613 pending_rounds.cc:107] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 23.14
I20260709 13:02:47.114946  1613 raft_consensus.cc:2889] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:47.122018  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 109507us
I20260709 13:02:47.123493  8600 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:47.127311  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:47.124204  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:47.118263  8613 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:47.130657  8600 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.131526  8025 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:47.135159  1618 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:47.128679  8697 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.139706  8576 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.140182  8590 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.141867  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.142170  1617 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:02:47.142922  2235 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:47.143146  8721 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:47.144027  1584 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:47.141870  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:47.145939  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:47.150123  2229 raft_consensus.cc:1240] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:47.118808  8237 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:47.170131  8782 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:47.170799  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:47.172355  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:47.172546  1683 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:02:47.173947  1688 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:02:47.183337  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.183686  8448 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:47.176448  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.184988  1685 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.185215  1541 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:47.186369  2233 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 22: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:47.187191  8590 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.187788  1584 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:47.128914  8680 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:47.182543  1684 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 15.
I20260709 13:02:47.196798  1685 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:47.190287  8703 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:47.201386  2201 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:47.205960  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 60146.5us
W20260709 13:02:47.206918  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 137882us
W20260709 13:02:47.207584  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.214427  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.190085  8776 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:02:47.216732  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.221840  1538 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:47.206638  8680 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:47.206099  8703 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:47.201890  8782 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:47.218608  8238 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.191102  8577 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 13:02:47.219262  8590 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:47.210258  8707 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:47.239982  1541 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:47.260082  1614 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:02:47.260890  8707 raft_consensus.cc:697] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:47.261354  8781 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.262029  8781 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:47.262833  8577 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:47.263293  8577 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:47.263626  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.264516  1538 raft_consensus.cc:2410] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:02:47.264192  8577 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.266469  8577 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.260281  8693 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:47.270865  8693 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:47.268074  8707 consensus_queue.cc:237] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.273595  8579 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:47.274422  8697 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:47.237597  8620 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.003s
I20260709 13:02:47.274672  8661 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:47.275199  8661 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:47.276391  8788 ts_tablet_manager.cc:933] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
I20260709 13:02:47.276237  8661 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:47.264309  8781 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:47.280866  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 44278.1us
I20260709 13:02:47.290693  1618 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:47.226691  8783 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:47.239379  8680 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:47.216984  8582 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:47.241087  8703 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:47.325815  8547 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.328111  8547 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:47.330258  8600 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:47.331228  1685 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:47.332420  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:47.277817  8541 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.335565  8541 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:47.335970  2233 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:47.340437  8582 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:47.341990  1611 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:47.343492  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:47.321801  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.344726  2236 raft_consensus.cc:2410] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 13:02:47.322893  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:47.343560  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.346482  2231 raft_consensus.cc:2376] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19. Re-sending same reply.
I20260709 13:02:47.318218  8783 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:47.324445  8590 consensus_queue.cc:1059] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:02:47.349726  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:47.349967  8783 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:47.350553  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.350821  8709 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:02:47.349251  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.351337  2231 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:47.351856  2235 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 24.
I20260709 13:02:47.350320  8590 raft_consensus.cc:3055] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:47.352530  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.352820  8590 raft_consensus.cc:740] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:47.353461  2232 raft_consensus.cc:2410] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:02:47.353837  8590 consensus_queue.cc:260] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.354358  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.355053  1618 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 25.
I20260709 13:02:47.355172  8590 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:47.355669  1610 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:47.356012  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.356631  1614 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:47.356871  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:47.354513  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.357725  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.358424  1616 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
W20260709 13:02:47.359241  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.223827  8576 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.359486  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:47.359957  8789 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:47.361295  8576 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:47.361600  8789 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:02:47.365314  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:47.373782  8582 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.374497  8582 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.306684  8620 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.376643  8582 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:47.377101  8620 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:47.349752  8777 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:02:47.255447  8782 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:47.320453  8646 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:47.320089  8693 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.059s
W20260709 13:02:47.379065  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.386147  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:47.390395  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 119839us
I20260709 13:02:47.392155  1509 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:47.398211  1684 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:47.399526  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.400629  1542 raft_consensus.cc:2410] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:02:47.401860  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.396795  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:47.374501  8713 consensus_meta.cc:287] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.001s
I20260709 13:02:47.402729  1539 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:47.408087  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:47.410871  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.409294  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:47.406750  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:47.419811  1539 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 23: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:47.419677  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.420385  1690 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:47.388983  8777 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:47.427623  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.436945  1541 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:02:47.429111  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.320901  8238 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:47.425307  1658 connection.cc:256] RPC call timeout handler was delayed by -1.755966849999993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
W20260709 13:02:47.442229  1658 connection.cc:256] RPC call timeout handler was delayed by -1.6470668499999874s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1647s for a response
I20260709 13:02:47.412351  8713 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17.
W20260709 13:02:47.442605  1658 connection.cc:256] RPC call timeout handler was delayed by -1.5939668500000153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
I20260709 13:02:47.437526  1541 pending_rounds.cc:85] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 12
W20260709 13:02:47.449988  1658 connection.cc:256] RPC call timeout handler was delayed by -1.4499668500000098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18660000000000002s for a response
I20260709 13:02:47.450104  1541 pending_rounds.cc:107] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 16.13
W20260709 13:02:47.450567  1658 connection.cc:256] RPC call timeout handler was delayed by -1.3950668499999779s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1927s for a response
I20260709 13:02:47.450654  1541 raft_consensus.cc:2889] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:47.451143  1658 connection.cc:256] RPC call timeout handler was delayed by -1.3824668500000143s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19410000000000002s for a response
W20260709 13:02:47.451586  1658 connection.cc:256] RPC call timeout handler was delayed by -1.3779668500000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1946s for a response
W20260709 13:02:47.453759  1658 connection.cc:256] RPC call timeout handler was delayed by -1.3257668499999795s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20040000000000002s for a response
W20260709 13:02:47.454303  1658 connection.cc:256] RPC call timeout handler was delayed by -1.293366849999984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20400000000000001s for a response
W20260709 13:02:47.454844  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2177668499999754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21240000000000003s for a response
I20260709 13:02:47.456658  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.457667  1539 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:47.406394  8781 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 13:02:47.458086  8448 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:47.432345  8636 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:02:47.429149  8774 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:47.460472  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:47.404024  8704 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 13:02:47.431093  8777 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:47.461642  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:47.469226  1539 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:47.470184  2229 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 13:02:47.470690  1612 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22. Ops: [19.11-21.12]
I20260709 13:02:47.442979  8590 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:47.474264  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.474958  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.475574  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:47.478266  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:47.478873  8788 tablet_copy_client.cc:323] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.193:43459
I20260709 13:02:47.493515  1609 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:47.494266  8541 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:47.494442  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.478785  8661 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:47.484179  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.466568  8636 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.505700  8636 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:02:47.468392  8774 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:47.486814  8693 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.487439  8646 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:47.488024  8703 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:47.488387  8683 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:47.490167  1616 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:47.497223  1617 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:47.500408  1690 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:47.502630  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:47.503206  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:47.504541  8590 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:47.464335  8620 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:47.510562  1610 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:47.511406  1684 raft_consensus.cc:2468] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:47.518785  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:47.513049  2203 connection.cc:256] RPC call timeout handler was delayed by -1.684397247999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1746s for a response
I20260709 13:02:47.517158  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:47.520475  2203 connection.cc:256] RPC call timeout handler was delayed by -1.6654972479999515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1767s for a response
W20260709 13:02:47.522243  2203 connection.cc:256] RPC call timeout handler was delayed by -1.4332972479999739s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2025s for a response
I20260709 13:02:47.524834  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:47.525398  1685 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:47.511713  8774 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:47.528273  1584 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:47.524047  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: true dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:47.529716  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.529826  1539 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:47.530975  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:47.514822  8683 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:47.546518  8683 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.547848  8683 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:47.493125  8783 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:47.551079  8783 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:47.535989  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.552175  1617 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:47.553689  1617 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:47.554445  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.554876  1617 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:02:47.555955  1611 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:47.540230  8703 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:47.556396  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.540802  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.557397  1612 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:47.557540  2233 raft_consensus.cc:2410] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 13:02:47.541301  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:47.519212  8025 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.558792  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.558919  8025 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.559777  2232 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:47.561069  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.557544  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:47.563934  2235 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 20: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:47.542855  1616 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:47.558419  1614 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:47.543114  8238 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:47.549718  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.561228  8025 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:47.566085  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:47.571949  8590 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:47.532932  8620 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.515506  8693 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.588574  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:47.588219  1609 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:47.590130  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:47.593852  8620 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:47.594174  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:47.594762  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.595391  1685 raft_consensus.cc:2376] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17. Re-sending same reply.
I20260709 13:02:47.595013  2228 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:47.568338  8773 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:02:47.602785  1507 leader_election.cc:336] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.675s (SENT)
I20260709 13:02:47.598979  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.604420  2235 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:47.620689  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:47.623061  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:47.623874  1542 raft_consensus.cc:2393] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:47.619556  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.630754  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.633641  2234 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 22: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:47.631749  2233 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:47.635715  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.636636  2235 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:47.639879  2236 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:02:47.643533  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:47.644224  1685 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:47.645270  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.646422  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.646483  2232 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:47.647140  2228 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:02:47.655584  1609 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:47.657451  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.659467  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.666079  8797 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:47.667155  2230 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 22: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:47.669572  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.670552  1542 raft_consensus.cc:2410] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:47.670805  8777 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.671452  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:47.674012  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:47.683802  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 82874.6us
I20260709 13:02:47.684881  1616 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 20: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:47.689786  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.692989  1614 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:02:47.694802  1539 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:02:47.695578  1541 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:47.696271  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.696992  2234 raft_consensus.cc:1240] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 13:02:47.697065  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.657532  8773 raft_consensus.cc:697] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:47.653573  8680 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.704113  1539 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:47.705123  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.709514  2228 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 13:02:47.684649  8777 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:47.712014  1691 raft_consensus.cc:1240] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:47.716681  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.724254  1612 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:47.724884  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.726768  1616 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:47.729720  1614 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:47.698215  8783 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.737602  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:47.743752  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:47.744385  2228 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:47.745574  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.752163  1690 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:47.756577  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:47.690563  8782 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:47.767992  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.768590  1685 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:02:47.729192  8448 consensus_queue.cc:1059] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:02:47.774115  1542 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 13:02:47.777309  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.782408  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:47.783540  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:47.779122  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:47.785504  1538 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:47.823220  1538 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:47.725986  8636 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:47.790258  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.790380  1540 raft_consensus.cc:2410] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 22 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:02:47.826653  1612 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:47.820832  2230 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:47.742972  8783 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.838155  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.839150  1542 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:47.842803  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:47.764948  8579 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 13:02:47.851449  1685 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:02:47.852674  1684 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:47.854631  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.855605  1537 raft_consensus.cc:1240] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22. Ops: []
W20260709 13:02:47.856161  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 92728.2us
I20260709 13:02:47.856997  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:47.861363  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:47.863783  1543 raft_consensus.cc:2393] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 26: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:47.861644  1685 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:47.866392  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.866279  1536 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:47.885262  2228 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:47.885298  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.872540  2237 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:02:47.874534  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.837261  8789 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:47.876850  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:47.878170  1688 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 14. Current term is 15. Ops: []
I20260709 13:02:47.765645  8774 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:47.881762  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:47.889650  1683 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:47.869886  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:47.886101  1541 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:47.887089  1534 raft_consensus.cc:2410] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:02:47.840790  8683 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 13:02:47.887727  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.895466  2233 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:02:47.892201  2230 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:02:47.892429  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 57788.2us
I20260709 13:02:47.892369  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:47.869844  8776 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:47.898444  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:47.899298  8025 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:47.838441  8625 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 21.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 13:02:47.903371  8782 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:47.903389  8025 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:47.904002  8625 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:47.900772  1691 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:02:47.904579  8805 log.cc:927] Time spent T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 13:02:47.899976  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.904706  8025 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.903026  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:47.903667  8693 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:47.904533  8625 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:47.899994  1538 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:47.908641  1538 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:47.902786  8620 consensus_queue.cc:1059] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:02:47.902045  2201 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:47.910255  8620 raft_consensus.cc:3055] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:02:47.910744  8620 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:47.911501  1691 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:02:47.905654  1540 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 15.
I20260709 13:02:47.912667  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.897210  1683 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:47.917586  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:47.916287  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.916059  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.914986  1685 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:02:47.857982  8720 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:02:47.923372  1686 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:47.923906  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:47.888392  8789 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.911639  8620 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.917313  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:47.916432  8699 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:47.914011  8813 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:02:47.926816  8807 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:02:47.929410  1582 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:47.929737  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.898595  8776 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:47.935302  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.960114  1685 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:02:47.929869  1614 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
W20260709 13:02:47.939754  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 70008.9us
I20260709 13:02:47.961673  1534 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:47.962065  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:47.932473  1657 leader_election.cc:336] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.509s (SENT)
I20260709 13:02:47.939011  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:47.937374  8803 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.236s	user 0.004s	sys 0.000s
I20260709 13:02:47.958931  1582 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:47.964998  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:47.946390  1683 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:47.948042  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:47.949587  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.951475  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:47.934892  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 68878us
I20260709 13:02:47.967599  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:47.908735  8625 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.968606  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:47.970718  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:47.971415  1683 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:02:47.972097  1686 raft_consensus.cc:2410] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 18 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:02:47.972906  1683 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:47.973621  8693 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:47.973903  8625 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:47.974102  8693 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:47.974680  8693 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:47.978758  1690 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:02:47.981648  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 64430.4us
I20260709 13:02:47.863106  8773 consensus_queue.cc:237] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:47.938511  8789 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:47.969491  1691 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:47.934535  8636 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:47.993662  1534 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976"
dest_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:33722
I20260709 13:02:47.994669  1534 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:02:47.995420  1534 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:47.995497  1688 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:47.950754  8699 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:47.950428  8813 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:47.994307  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 61923us
I20260709 13:02:47.967501  1509 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:47.964011  8776 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:47.963249  8772 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:48.006728  1582 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:47.983647  8541 raft_consensus.cc:2804] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:02:48.029692  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 61909.7us
I20260709 13:02:48.021739  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:48.032788  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:48.007764  8699 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:47.999696  8613 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.042068  8025 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:47.993206  8797 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:48.001159  8703 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:02:48.013300  8813 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:48.062680  8025 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.054167  8814 consensus_queue.cc:1059] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:02:48.040503  8238 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:02:47.912451  8600 consensus_queue.cc:1175] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" responder_term: 17 status { last_received { term: 17 index: 12 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:48.055388  1583 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:48.073879  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:48.074937  2236 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:48.056562  8579 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:02:48.097615  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 86730.1us
I20260709 13:02:48.099048  1658 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:48.101691  8772 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:48.059069  1534 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.072041  8025 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:48.105881  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:48.107059  2231 raft_consensus.cc:2368] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 20. Current term is 22.
I20260709 13:02:48.098603  8699 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.108455  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:48.108793  2230 raft_consensus.cc:2368] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 20. Current term is 21.
I20260709 13:02:48.109704  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:48.110606  2237 raft_consensus.cc:2376] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18. Re-sending same reply.
I20260709 13:02:48.111994  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:48.114760  8699 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:48.106112  1534 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:48.123164  8783 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:48.037053  8636 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:48.133486  2231 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:48.101181  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:48.129777  2237 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:48.043133  8541 raft_consensus.cc:697] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:48.058158  8525 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:48.016533  8782 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 13:02:48.142670  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:48.152225  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:48.141809  8783 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:48.069963  8613 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.139801  8797 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:48.165997  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 159331us
I20260709 13:02:48.146114  8636 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:48.170339  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.144826  8238 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:48.171530  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 70508.2us
I20260709 13:02:48.172971  1582 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:48.175140  8613 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:48.176476  8797 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.177441  8541 consensus_queue.cc:237] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:48.183867  1657 leader_election.cc:336] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.739s (SENT)
W20260709 13:02:48.185655  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 130410us
I20260709 13:02:48.186535  1583 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:48.188678  1583 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.192437  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:48.193615  2234 raft_consensus.cc:2393] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:48.194473  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:48.194693  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:48.195858  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:48.196318  8547 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:48.196491  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:48.197480  8547 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:48.197723  2236 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:48.198150  2201 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:48.199378  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:48.201009  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.201834  8699 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:48.202003  8774 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:48.202291  8699 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:48.202744  8699 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:48.202615  8707 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:02:48.203876  8661 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.697s	user 0.011s	sys 0.001s
I20260709 13:02:48.204872  8636 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:48.209784  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.212033  2236 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:48.191032  8781 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:48.188472  8703 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:02:48.214332  8707 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.215093  8707 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.217954  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:48.218706  2234 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:48.219314  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:48.225520  8693 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.227972  8693 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:48.225534  2237 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:02:48.229734  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:48.230643  8783 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:48.231222  8707 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:48.231988  1658 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:48.233656  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 60768.1us
I20260709 13:02:48.232208  1538 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:48.235785  1615 raft_consensus.cc:2410] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 13:02:48.196506  2232 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:48.194473  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:48.240476  1582 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.241201  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:48.241904  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 53293us
I20260709 13:02:48.245255  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:48.249382  8713 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:48.249815  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 55437.2us
I20260709 13:02:48.249347  8773 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 13:02:48.249276  8781 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:48.250560  8025 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:48.252396  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.250065  8713 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.253299  8713 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:48.255653  8579 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:48.256182  8579 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:48.256690  8025 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:48.197890  8547 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:48.259387  8025 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:48.260479  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:48.244731  1583 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.262871  1685 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:48.264556  1508 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:48.228818  8789 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:48.267287  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 299876us
I20260709 13:02:48.238147  8448 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:48.268707  8448 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.269086  8448 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:48.234946  8577 raft_consensus.cc:3055] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:48.269157  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:48.271870  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.272487  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:48.274821  2234 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:48.280635  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:48.285074  2203 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.282936  8025 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.285200  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:48.260991  8783 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:48.294835  2236 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:48.273531  8577 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:48.285187  8576 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:48.285713  8774 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.274073  8823 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:02:48.273906  8721 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
W20260709 13:02:48.347492  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 84564.7us
I20260709 13:02:48.294636  8525 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:02:48.344436  1657 leader_election.cc:336] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.927s (SENT)
I20260709 13:02:48.306166  8797 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:48.258038  8813 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:48.339439  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 65114.5us
I20260709 13:02:48.305207  8707 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:48.337656  8600 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.301852  8683 consensus_queue.cc:1059] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:02:48.287315  8776 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.338564  8576 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:48.338343  8693 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:48.373855  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 129220us
W20260709 13:02:48.369781  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 91777.3us
W20260709 13:02:48.366462  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.352715  8797 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:48.346167  8774 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:48.379720  1583 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:48.392055  2203 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:02:48.392985  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.393515  2203 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.378451  8823 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:48.394114  8707 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:48.392910  8721 consensus_queue.cc:1243] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 23.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:02:48.395313  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:02:48.395685  1657 leader_election.cc:336] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.750s (SENT)
I20260709 13:02:48.376647  8576 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:48.399448  8707 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:48.396724  8636 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.400476  8590 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:48.401957  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:48.402562  8525 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:48.404071  8720 raft_consensus.cc:3055] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:48.398567  1657 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:48.369410  8600 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.382665  8797 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:48.357052  8547 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.381578  8773 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:02:48.382201  8693 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:48.396742  8823 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:48.394150  8813 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:48.398314  8832 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:48.406613  8625 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:48.406284  8703 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 13:02:48.412636  8782 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:48.445830  8721 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:48.462929  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:48.412312  8590 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:48.413707  8720 raft_consensus.cc:740] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:48.449595  8783 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:48.474915  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 79247.1us
I20260709 13:02:48.398125  8577 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:48.446167  8833 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.034s
W20260709 13:02:48.483876  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 90454us
I20260709 13:02:48.416416  8636 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.477699  8824 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.477511  8560 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.491614  2203 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.492909  2235 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:02:48.483527  8789 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:48.485903  8833 consensus_queue.cc:1243] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:02:48.474043  8721 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:48.476815  8831 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 13:02:48.492378  8025 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.003s
W20260709 13:02:48.547559  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 56070.1us
I20260709 13:02:48.486491  8782 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:48.555622  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:48.557821  8025 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:48.557987  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.494179  8525 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:48.558321  8025 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.558910  8025 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:48.547214  8646 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.565881  8720 consensus_queue.cc:260] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:48.567628  8836 log.cc:927] Time spent T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:02:48.490091  8823 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:48.569041  8713 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.570125  8774 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:48.570614  8680 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:48.557099  8830 log.cc:927] Time spent T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:48.497153  8560 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.577448  8772 raft_consensus.cc:3055] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:48.494122  8840 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.075s
W20260709 13:02:48.579085  8828 log.cc:927] Time spent T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:02:48.579589  8693 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:48.632253  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 74349.5us
I20260709 13:02:48.580482  8620 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:02:48.552345  8776 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:48.588598  1508 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:48.635025  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 46506.6us
I20260709 13:02:48.592100  8025 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:48.587035  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 134226us
I20260709 13:02:48.547993  8625 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:48.604882  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:48.606173  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:48.639564  1691 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:02:48.488032  8707 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.620225  8849 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:48.620755  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 271963us
I20260709 13:02:48.623260  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:48.623818  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:48.625105  1609 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:02:48.659929  1616 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:02:48.625492  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:48.623313  1614 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:48.625797  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:48.626190  8713 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:48.496145  8824 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:48.633257  2203 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.638047  1610 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:48.601562  8646 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.551064  8590 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.643639  1611 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
W20260709 13:02:48.645901  8829 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 13:02:48.646668  8600 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:48.648425  8577 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:48.648097  8448 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:48.650082  8636 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:48.688408  8448 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:48.654983  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 2470) took 1300 ms. Trace:
I20260709 13:02:48.675652  1611 pending_rounds.cc:85] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 13
I20260709 13:02:48.691890  1611 pending_rounds.cc:107] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 13:02:48.655548  8834 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:48.656432  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:48.656865  8560 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:48.657127  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:48.033537   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:48.655028  1685 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:48.661484  1618 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:48.629948  8772 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:48.671111  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.615401  8699 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.671996  1584 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.673048  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:48.640362  8693 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:48.682881  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:48.692245  1611 raft_consensus.cc:2889] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:48.691198  1615 rpcz_store.cc:276] 0709 13:02:47.354547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.356521 (+  1974us) service_pool.cc:224] Handling call
0709 13:02:48.654753 (+1298232us) spinlock_profiling.cc:243] Waited 876 ms on lock 0x7b5400001c54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:48.654900 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2014856960}
I20260709 13:02:48.731139  1609 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:48.695564  8025 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:48.692893  8835 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 13:02:48.702461  8590 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:48.697567  8525 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:48.709693  1691 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:48.712069  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:48.656920  8720 raft_consensus.cc:3060] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:48.721498  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 49630us
I20260709 13:02:48.721736  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:48.737758  8025 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:48.722649  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:48.738344  8025 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:48.729609  1616 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 26.
W20260709 13:02:48.730415  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.731731  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:48.744822  1618 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:48.746613  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:48.747416  2235 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:48.747288  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:48.732409  1617 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:48.673383  8823 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:48.749204  1616 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:48.736645  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:48.722332  2237 pending_rounds.cc:85] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 14
I20260709 13:02:48.750638  2237 pending_rounds.cc:107] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:02:48.747272  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.750955  2237 raft_consensus.cc:2889] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:02:48.750560  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 55208.5us
I20260709 13:02:48.753862  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:48.754194  2229 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:48.754575  1688 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:48.757767  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:48.636528  8579 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.750208  2234 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:48.713378  8547 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:48.724009  8849 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:48.778410  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:48.779170  2233 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 22: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:48.730882  8693 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.776041  8847 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:02:48.776170  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:48.776610  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:48.776854  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:48.720862  8699 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:48.716602  8824 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:48.841542  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:48.770896  8025 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.842883  1613 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 24. Current term is 25. Ops: []
I20260709 13:02:48.848793  1609 raft_consensus.cc:2393] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:48.849756  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:48.856309  1537 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:48.851426  1543 raft_consensus.cc:2410] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 20 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 13:02:48.851867  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 71005.4us
I20260709 13:02:48.776618  1434 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:48.854806  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 181821us
I20260709 13:02:48.859551  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:48.860337  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.782459  8541 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:48.864445  8707 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:48.864764  8721 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:48.864863  8707 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:48.865793  8707 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:48.719033  8772 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:48.842926  8661 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.868350  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.847558  8693 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:48.878079  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.874465  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.882871  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:48.881018  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 100792us
W20260709 13:02:48.896742  8821 log.cc:927] Time spent T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:02:48.897043  1535 raft_consensus.cc:1275] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:48.897292  1541 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:48.881991  8661 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:48.902635  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:48.903928  8850 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:48.904562  8850 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.904951  8850 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:48.905031  1689 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:02:48.905757  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:48.904016  8661 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:48.902690  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:48.908692  8646 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:48.908857  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:48.909996  8646 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:48.912740  1537 raft_consensus.cc:2410] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:48.912669  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:48.913142  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:48.913806  1683 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
I20260709 13:02:48.913620  8834 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:48.915282  8834 consensus_queue.cc:1059] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:02:48.889364  1507 leader_election.cc:336] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.517s (SENT)
I20260709 13:02:48.901960  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:48.920827  1688 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:48.920737  1617 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:48.916471  8721 raft_consensus.cc:3055] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:48.916985  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:48.926133  8721 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:48.929399  8579 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:48.930126  8721 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:48.931363  8721 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:48.928670  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:48.916992  8782 consensus_queue.cc:1059] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 13:02:48.934237  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 57960.4us
I20260709 13:02:48.934491  8782 raft_consensus.cc:3055] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:48.934923  8782 raft_consensus.cc:740] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:48.883632  8699 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:48.938133  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.886309  8840 raft_consensus.cc:2764] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:02:48.936180  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:48.953234  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 60892.4us
I20260709 13:02:48.959910  1537 raft_consensus.cc:1240] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 15. Current term is 16. Ops: []
I20260709 13:02:48.961413  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:48.961263  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:48.962725  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:48.953683  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.916404  8834 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:48.956246  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:48.958793  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:48.967707  1540 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
I20260709 13:02:48.972465  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:48.974352  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:48.968273  1509 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:48.975639  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:02:48.965924  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 217193us
I20260709 13:02:48.890223  8774 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.081s
I20260709 13:02:48.913689  8636 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:02:48.957078  8772 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:48.987170  1610 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: [17.13-19.14]
I20260709 13:02:48.988255  1609 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:48.965575  8789 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:02:48.995213  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:49.010260  1614 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:48.967960  8646 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:02:49.002291  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 67310.1us
I20260709 13:02:49.005228  8858 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:48.970990  8844 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:48.969267  8834 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.990547  2201 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:49.046454  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55992.1us
I20260709 13:02:49.051326  1583 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:49.026870  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 52844.2us
I20260709 13:02:49.052682  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:48.981113  8824 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:48.981590  8448 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:48.978399  8814 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:48.953096  8541 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:02:48.983676  8683 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:49.055361  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:49.057222  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:48.965801  8579 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 13:02:49.060734  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:49.058059  8818 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260709 13:02:49.069172  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 62688.2us
I20260709 13:02:49.042315  8646 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.073279  1686 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:49.045687  8844 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:48.988692  8600 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 13:02:48.994766  8840 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 13:02:49.085826  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:49.075239  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:49.086630  2235 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:49.057667  8834 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:49.076107  1583 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:49.078367  8843 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:02:49.080631  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.064281  8683 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.099520  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:49.108186  2228 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
W20260709 13:02:49.109607  8854 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.269s	user 0.002s	sys 0.000s
W20260709 13:02:49.111207  8846 log.cc:927] Time spent T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:02:49.111984  1614 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
W20260709 13:02:49.109475  1507 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.703s (SENT)
I20260709 13:02:49.113721  1507 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:49.010686  8782 consensus_queue.cc:260] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:49.080364  8646 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:02:49.144906  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 68865.5us
I20260709 13:02:49.148729  8636 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:49.149595  8636 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:49.152650  8636 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:49.153828  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 101247us
W20260709 13:02:49.154824  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 171931us
W20260709 13:02:49.156545  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 183747us
I20260709 13:02:49.159874  1609 raft_consensus.cc:1240] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 13:02:49.158591  8636 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:49.082727  8844 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:49.113215  8683 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:49.162058  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:02:49.168073  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 92906.1us
I20260709 13:02:49.106034  8237 raft_consensus.cc:3055] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:49.055668  8858 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:49.081550  8704 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:02:49.190557  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:49.198776  1609 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 24: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:49.086686  8448 consensus_queue.cc:1048] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.088s
I20260709 13:02:49.165572  8703 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:49.203572  2203 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:49.112936  8857 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:49.169966  8824 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:49.145679  8620 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
W20260709 13:02:49.205540  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 58768.4us
I20260709 13:02:49.216521  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:49.189632  8237 raft_consensus.cc:740] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:49.175570  8789 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 26 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:49.220916  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:49.200918  8782 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:49.203047  8832 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:49.255885  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:49.206779  8703 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.202327  8814 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:49.219079  8840 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:49.189786  8859 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.255s
W20260709 13:02:49.267431  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 63946us
I20260709 13:02:49.225346  8541 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:02:49.253577  8842 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:49.257215  8832 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:49.262137  8703 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:49.275070  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:49.289605  8704 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.290220  8704 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:49.278620  8832 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:49.278188  8620 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:02:49.334558  8865 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 13:02:49.337477  8862 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
W20260709 13:02:49.338238  8863 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 13:02:49.338493  8867 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:02:49.334575  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 59598.9us
I20260709 13:02:49.364777  8874 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.365833  8874 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:49.377846  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 272135us
I20260709 13:02:49.391271  8858 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:49.392014  8858 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:49.337502  8824 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:49.328649  8620 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:49.314101  8840 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:49.394269  8620 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:49.339061  8855 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:49.435894  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 95133.7us
W20260709 13:02:49.444171  1507 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.707s (SENT)
I20260709 13:02:49.395567  8814 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:49.438864  8831 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.463267  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:49.455543  8836 log.cc:927] Time spent T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:02:49.299068  8237 consensus_queue.cc:260] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
W20260709 13:02:49.481752  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 47038us
I20260709 13:02:49.454097  8620 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:49.478161  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:49.443702  8857 raft_consensus.cc:697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:49.476130  8704 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:49.486199  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 338979us
W20260709 13:02:49.547282  1507 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.741s (SENT)
I20260709 13:02:49.511713  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:49.396296  8789 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:49.552343  1507 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:49.551021  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.554033  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.555646  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.523550  8590 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.762s	user 0.012s	sys 0.004s
W20260709 13:02:49.556417  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:49.561084  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.509812  8773 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.487759  8855 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:49.524501  8867 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:49.573872  8709 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:49.574740  8871 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.246s	user 0.000s	sys 0.000s
W20260709 13:02:49.589339  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 111205us
I20260709 13:02:49.577136  8886 consensus_queue.cc:1059] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:49.577842  8873 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:49.557497  8876 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:49.615542  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 51665.7us
I20260709 13:02:49.573876  1541 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
W20260709 13:02:49.578855  8576 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.164s	user 0.015s	sys 0.001s
I20260709 13:02:49.552807  8237 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:49.590526  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:49.481165  8831 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:49.649361  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 138871us
W20260709 13:02:49.650871  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 139239us
W20260709 13:02:49.624980  8826 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.002s	user 0.004s	sys 0.005s
W20260709 13:02:49.624123  8693 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.733s	user 0.007s	sys 0.008s
I20260709 13:02:49.621955  8873 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:49.660411  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 113166us
W20260709 13:02:49.663693  1507 leader_election.cc:336] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.874s (SENT)
I20260709 13:02:49.664234  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:49.653754  8860 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
W20260709 13:02:49.668744  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 111819us
I20260709 13:02:49.625409  8874 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:49.610946  8773 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:49.674670  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:49.677047  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:49.679248  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 57031.6us
I20260709 13:02:49.682811  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:49.668183  8823 consensus_meta.cc:287] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.906s	user 0.006s	sys 0.011s
I20260709 13:02:49.626645  8576 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:49.662045  8886 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:02:49.685101  8850 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.780s	user 0.010s	sys 0.008s
I20260709 13:02:49.679450  8590 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:49.727550  1612 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
W20260709 13:02:49.732169  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68515.2us
I20260709 13:02:49.626165  8876 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:49.666604  8693 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:49.725152  8613 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.014s	sys 0.052s
W20260709 13:02:49.739709  1507 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.909s (SENT)
I20260709 13:02:49.740152  1539 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:49.676559  8867 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:49.715809  8703 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:49.749811  8703 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:49.741585  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:49.752974  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:49.754333  1612 raft_consensus.cc:2410] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:02:49.685520  8525 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 13:02:49.756156  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:49.763996  1609 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:49.766681  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:49.769357  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:49.773574  8680 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:49.774022  8886 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:49.774571  8886 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:49.778674  8576 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:49.780169  8773 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:49.784650  8646 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.615s	user 0.002s	sys 0.010s
I20260709 13:02:49.787719  1617 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 23. Current term is 24. Ops: []
W20260709 13:02:49.788446  8699 consensus_meta.cc:287] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.829s	user 0.009s	sys 0.000s
I20260709 13:02:49.746727  8895 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.050s
W20260709 13:02:49.790194  8797 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 1.339s	user 0.007s	sys 0.006s
I20260709 13:02:49.790654  8797 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:49.796429  2228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1714) took 1826 ms (client timeout 2348 ms). Trace:
W20260709 13:02:49.797174  8887 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:02:49.798605  8693 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:49.799401  8699 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:49.799808  8882 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:49.800402  8889 raft_consensus.cc:2764] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:02:49.768738  8874 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:02:49.742211  8870 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:49.757839  8590 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:49.796283  8896 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:02:49.816076  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:49.816903  1612 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:49.819747  8866 raft_consensus.cc:3055] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:49.820295  8866 raft_consensus.cc:740] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:49.821164  8866 consensus_queue.cc:260] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:49.821990  8895 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:49.822420  8895 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:49.822489  8866 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:49.822741  8683 consensus_meta.cc:287] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.625s	user 0.004s	sys 0.011s
I20260709 13:02:49.823308  8683 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:49.860944  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 136116us
I20260709 13:02:49.785136  8646 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:49.811299  8707 consensus_meta.cc:287] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.945s	user 0.010s	sys 0.001s
W20260709 13:02:49.858639  8721 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.897s	user 0.013s	sys 0.000s
I20260709 13:02:49.647810  8857 consensus_queue.cc:237] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:49.797107  2228 rpcz_store.cc:269] 0709 13:02:47.970135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.970348 (+   213us) service_pool.cc:224] Handling call
0709 13:02:49.796121 (+1825773us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54002e95d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:49.796300 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2613937536}
W20260709 13:02:49.813773  8720 consensus_meta.cc:287] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.058s	user 0.008s	sys 0.004s
I20260709 13:02:49.804266  8797 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:49.737455  8850 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:49.747536  8613 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.014s	sys 0.052s
I20260709 13:02:49.804327  8824 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:49.893046  1538 raft_consensus.cc:1275] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:02:49.893561  1538 pending_rounds.cc:85] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 9
I20260709 13:02:49.893867  1538 pending_rounds.cc:107] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:02:49.894194  1538 raft_consensus.cc:2889] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:49.761826  8709 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
I20260709 13:02:49.794171  8876 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:49.885736  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:49.805235  1507 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:49.890501  1686 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:49.897773  2203 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:49.881237  1657 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:49.901678  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 96545.2us
I20260709 13:02:49.891721  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:49.908851  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:49.881697  8886 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:49.909600  2234 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:49.926245  1612 raft_consensus.cc:1240] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:49.905273  8721 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:49.879045  8707 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:49.932257  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:49.934449  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:49.935230  2228 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:49.827267  8870 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:49.911372  8683 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:49.954856  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 73685.3us
I20260709 13:02:49.967206  1657 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:49.967175  1583 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:49.912242  8876 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 13:02:50.030575  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 63588us
I20260709 13:02:50.030607  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.037789  1612 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:02:49.938423  8886 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:50.039022  2232 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:49.899066  8831 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:50.033226  2228 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:50.048081  1616 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 13:02:50.048235  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 81090.6us
I20260709 13:02:50.053236  1657 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:50.054530  1657 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:50.033278  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 318870us
I20260709 13:02:50.042002  8547 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:50.068804  1658 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:50.059087  8883 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
I20260709 13:02:49.919521  8895 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.031534  8867 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.063395  8879 log.cc:927] Time spent T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 13:02:50.031777  8904 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.033902  8721 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:50.096091  8904 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.099321  8904 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.096926  8721 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:50.059480  8525 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:50.100510  8886 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:50.067312  8884 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 13:02:50.101320  8886 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:50.101872  8886 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 22 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:50.103142  8891 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 13:02:50.102748  8886 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.078488  8547 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.112349  2203 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:50.112766  8855 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:50.113914  8855 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:50.114023  8861 raft_consensus.cc:3055] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:50.114566  8861 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:50.114861  8882 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:50.114183  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:50.090735  8560 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.122764  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 68295.9us
I20260709 13:02:50.076493  8831 consensus_queue.cc:1059] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:02:50.146464  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.119042  8547 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:50.156303  8906 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:02:50.138887  8797 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:50.157526  8797 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.159664  8797 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.142957  8911 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 13:02:50.163362  1686 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:02:50.144624  8560 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.164475  1657 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:50.115594  8861 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.167860  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.168676  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:50.182587  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.183502  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1639) took 1459 ms. Trace:
I20260709 13:02:50.123997  8646 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:50.183931  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.182086  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1292) took 1208 ms. Trace:
I20260709 13:02:50.192140  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2483) took 1567 ms. Trace:
I20260709 13:02:50.106123  8867 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:50.178539  8834 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.057s	user 0.010s	sys 0.003s
W20260709 13:02:50.190476  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 113810us
I20260709 13:02:50.123500  8909 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:50.192767  1611 rpcz_store.cc:276] 0709 13:02:48.624548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.624658 (+   110us) service_pool.cc:224] Handling call
0709 13:02:48.624948 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:48.768586 (+143638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.191805 (+1423219us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b54003c91d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:50.192057 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35130240}
I20260709 13:02:50.197933  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.183890  2237 rpcz_store.cc:276] 0709 13:02:48.720448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.721807 (+  1359us) service_pool.cc:224] Handling call
0709 13:02:48.722137 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:49.942954 (+1220817us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b1800244120. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08f6944bc 00007fe08c9e248b 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fc455 00007fe08fae4060 00007fe08fae4bfb 00007fe08faef64b
0709 13:02:49.943106 (+   152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.943134 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:49.943718 (+   584us) log.cc:844] Serialized 52 byte log entry
0709 13:02:50.111787 (+168069us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:02:50.112116 (+   329us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.112498 (+   382us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.114670 (+  2172us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.114763 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.179261 (+ 64498us) raft_consensus.cc:1685] finished
0709 13:02:50.179286 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.179447 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5005,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":34412,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":5139,"spinlock_wait_cycles":4797568,"thread_start_us":150818,"threads_started":2,"wal-append.queue_time_us":121417}
I20260709 13:02:50.203864  2232 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:50.123116  8699 consensus_queue.cc:1059] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 13:02:50.176473  8889 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:50.206248  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2486) took 1580 ms. Trace:
I20260709 13:02:50.140995  8855 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
W20260709 13:02:50.197305  8772 consensus_meta.cc:287] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 1.171s	user 0.012s	sys 0.021s
I20260709 13:02:50.179699  8866 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:50.209687  1613 rpcz_store.cc:276] 0709 13:02:48.626137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.626880 (+   743us) service_pool.cc:224] Handling call
0709 13:02:48.627304 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:48.854650 (+227346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.206001 (+1351351us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400104d54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:50.206184 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353918720}
W20260709 13:02:50.216632  8913 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:02:50.218290  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.219036  1692 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:50.219233  8870 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.220103  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.220878  1692 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 17: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:50.221256  8867 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:50.206936  8834 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:50.226313  8560 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.192600  1615 rpcz_store.cc:276] 0709 13:02:48.973173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.076302 (+103129us) service_pool.cc:224] Handling call
0709 13:02:49.091287 (+ 14985us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.117338 (+ 26051us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.117386 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:49.163752 (+ 46366us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.512128 (+348376us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:49.512186 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.512279 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.181792 (+669513us) raft_consensus.cc:1685] finished
0709 13:02:50.181821 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.182006 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25490,"raft.queue_time_us":49731,"spinlock_wait_cycles":39248768,"thread_start_us":304743,"threads_started":1,"wal-append.queue_time_us":451433}
I20260709 13:02:50.210675  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.211534  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.212191  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.212684  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.207350  1583 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:50.208403  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:50.197242  8844 consensus_meta.cc:287] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.996s	user 0.009s	sys 0.000s
I20260709 13:02:50.164907  8915 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 13:02:50.228655  1686 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:50.232851  2237 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:50.233628  1691 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:50.241839  1689 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:50.234133  1688 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:02:50.228287  1685 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:02:50.240077  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.254385  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.251777  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.253665  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.254031  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.256038  1691 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:50.257133  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.258301  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.249495  2232 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:02:50.250664  8832 consensus_meta.cc:287] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.909s	user 0.017s	sys 0.006s
I20260709 13:02:50.262406  2237 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:50.255645  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:50.238853  8782 consensus_meta.cc:287] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.976s	user 0.012s	sys 0.000s
I20260709 13:02:50.216687  8866 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.269809  1507 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:50.241803  8560 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:50.273819  1687 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 13:02:50.268306  8909 raft_consensus.cc:697] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:50.253052  8867 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:50.280329  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.286276  1691 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:50.260995  8832 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.247144  8881 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 13:02:50.245990  8844 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:50.273600  8866 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:50.295143  8903 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.276345  8560 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:50.303512  8911 raft_consensus.cc:3055] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:50.305812  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3499) took 1443 ms. Trace:
I20260709 13:02:50.278658  8870 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:50.316059  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.314232  1539 rpcz_store.cc:276] 0709 13:02:48.862854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.863075 (+   221us) service_pool.cc:224] Handling call
0709 13:02:48.863355 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.747681 (+884326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.305510 (+557829us) spinlock_profiling.cc:243] Waited 877 ms on lock 0x7b5400343fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:50.305744 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2015985408}
I20260709 13:02:50.316816  1691 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:50.315464  8560 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:50.319878  8844 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:50.321805  8832 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:50.322906  8636 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.145s	user 0.006s	sys 0.011s
I20260709 13:02:50.283933  8867 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:50.323361  8636 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:50.324841  8867 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:50.314076  8911 raft_consensus.cc:740] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:50.346295  8911 consensus_queue.cc:260] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.347633  8911 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:50.349014  8867 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:50.354749  1691 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
W20260709 13:02:50.329298  8620 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.806s	user 0.007s	sys 0.002s
I20260709 13:02:50.356209  2232 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:50.309582  8834 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:50.349666  8867 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:50.361730  2229 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:50.363008  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.363826  1691 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:50.350430  8921 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.366576  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: true dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.367182  1684 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 15: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:50.364205  8920 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:50.376370  8867 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:50.377209  8857 consensus_queue.cc:1059] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:50.378409  8857 raft_consensus.cc:3055] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:02:50.378726  8857 raft_consensus.cc:740] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:50.379518  8857 consensus_queue.cc:260] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.380863  8857 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:50.388437  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.389345  1691 raft_consensus.cc:2376] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22. Re-sending same reply.
I20260709 13:02:50.391737  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.392812  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:50.394368  1534 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:50.393586  1542 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:50.396355  8870 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.396656  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:50.397282  1542 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:50.398104  1536 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: [19.11-19.11]
W20260709 13:02:50.399065  8872 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.137s	user 0.002s	sys 0.000s
I20260709 13:02:50.400628  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1636) took 1681 ms. Trace:
I20260709 13:02:50.400874  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:50.400647  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:50.402563  1537 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 15: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:50.402473  8834 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:50.401741  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.403681  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:50.404371  1683 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:50.404565  1534 raft_consensus.cc:2410] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 7.
I20260709 13:02:50.403229  8834 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.396939  8870 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.401041  2231 rpcz_store.cc:276] 0709 13:02:48.719586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.721570 (+  1984us) service_pool.cc:224] Handling call
0709 13:02:48.721911 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:48.867611 (+145700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.867648 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:48.879379 (+ 11731us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.148634 (+269255us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:49.214826 (+ 66192us) log.cc:844] Serialized 14 byte log entry
0709 13:02:49.447973 (+233147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:49.448095 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.398285 (+950190us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:50.400314 (+  2029us) raft_consensus.cc:1685] finished
0709 13:02:50.400447 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.400573 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66710,"prepare.queue_time_us":177617,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":90819,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":202,"spinlock_wait_cycles":4506880,"thread_start_us":604652,"threads_started":4,"wal-append.queue_time_us":306780}
I20260709 13:02:50.387112  8560 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.408811  8870 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.409241  8720 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.409991  8720 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.412310  8720 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.413002  8797 raft_consensus.cc:3055] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:50.407433  8866 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.413523  8797 raft_consensus.cc:740] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:50.415198  1683 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:02:50.414741  8797 consensus_queue.cc:260] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.415851  8833 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.416466  8797 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:50.418011  8867 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:50.419452  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.428576  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:50.429502  1541 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:50.432274  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:50.433986  1543 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 26 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 25. Current term is 26. Ops: []
I20260709 13:02:50.438079  1540 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 18: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:50.438855  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.439451  1684 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:02:50.441798  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.442416  1684 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
W20260709 13:02:50.442467  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:50.442986  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.446923  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.448171  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.448719  1684 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:50.449735  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.450313  1691 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:50.418587  8867 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:50.434756  8844 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:50.431844  8560 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.437408  8789 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.416419  8833 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:50.368858  8921 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.330710  8903 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.457345  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:50.467501  1543 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:50.458886  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:50.467680  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.468220  1537 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 18: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:50.460076  1539 raft_consensus.cc:1275] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:02:50.460803  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:50.461902  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:50.469715  1535 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:50.462252  1553 tablet_copy_service.cc:140] P 83f6f5810950417195f707ff308a8125: Received BeginTabletCopySession request for tablet 7ba0620abe844f5bb71ba9dd947c3b9c from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:33722)
W20260709 13:02:50.456609  8908 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.333s	user 0.000s	sys 0.003s
I20260709 13:02:50.468722  1539 pending_rounds.cc:85] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 12
I20260709 13:02:50.484477  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:50.518150  1691 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:02:50.518460  1539 pending_rounds.cc:107] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:02:50.455484  8844 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.488536  8866 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.459478  8789 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:50.492063  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.526782  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:50.471704  1542 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 17. (term mismatch)
I20260709 13:02:50.530861  1542 pending_rounds.cc:85] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 16
I20260709 13:02:50.531129  1542 pending_rounds.cc:107] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 13:02:50.531368  1542 raft_consensus.cc:2889] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.518811  1539 raft_consensus.cc:2889] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.518823  1540 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:50.520722  1541 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:50.521512  1535 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:50.523910  1684 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 24.
W20260709 13:02:50.519804  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 81904.6us
I20260709 13:02:50.527704  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:50.533393  1537 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:50.528466  1536 raft_consensus.cc:1240] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:02:50.370035  8909 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.535454  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:50.544014  1537 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
W20260709 13:02:50.530602  8930 log.cc:927] Time spent T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
I20260709 13:02:50.545526  1543 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:02:50.519140  8843 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:50.551373  1534 raft_consensus.cc:1275] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:02:50.553798  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1637) took 1833 ms. Trace:
I20260709 13:02:50.532094  8869 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:50.558602  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:50.562726  8921 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.564545  8844 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.536590  8547 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.570533  8867 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:50.545776  8859 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:50.571142  8833 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:50.537902  8870 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:50.525090  1553 tablet_copy_service.cc:161] P 83f6f5810950417195f707ff308a8125: Beginning new tablet copy session on tablet 7ba0620abe844f5bb71ba9dd947c3b9c from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:33722: session id = cecad4f86a114a7696e398b3e8cc511e-7ba0620abe844f5bb71ba9dd947c3b9c
I20260709 13:02:50.583312  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:50.573729  8824 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.673s	user 0.006s	sys 0.006s
I20260709 13:02:50.587442  8824 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:50.573841  8832 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:50.532721  8915 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:02:50.588402  8832 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.588567  8915 consensus_queue.cc:1243] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:50.588860  8903 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.589205  8915 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:50.589670  8915 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:50.560220  8900 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:02:50.590548  8900 consensus_queue.cc:1243] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 26.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:02:50.591112  8900 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 25 LEADER]: Stepping down as leader of term 25
I20260709 13:02:50.591463  8900 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:50.590721  8915 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:50.575517  8869 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.542174  8783 consensus_queue.cc:1059] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:02:50.559523  8843 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.584438  8844 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.592733  8844 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:50.593058  8915 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:50.536955  8782 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:50.578406  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.580775  1582 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:50.580873  8789 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:50.583680  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 64758.9us
I20260709 13:02:50.596414  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:50.623574  1540 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:50.606873  1507 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:50.620923  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:50.622898  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.554309  2230 rpcz_store.cc:276] 0709 13:02:48.719887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.734777 (+ 14890us) service_pool.cc:224] Handling call
0709 13:02:48.735133 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:48.761230 (+ 26097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:48.761267 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:48.779932 (+ 18665us) log.cc:844] Serialized 26 byte log entry
0709 13:02:49.792370 (+1012438us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b18003703c0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:02:49.906097 (+113727us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:49.942615 (+ 36518us) log.cc:844] Serialized 14 byte log entry
0709 13:02:50.028148 (+ 85533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.028245 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:50.330108 (+301863us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:50.549509 (+219401us) raft_consensus.cc:1685] finished
0709 13:02:50.549540 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:50.553710 (+  4170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39521,"prepare.queue_time_us":36001,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":108014,"raft.run_cpu_time_us":335,"raft.run_wall_time_us":85385,"spinlock_wait_cycles":8783360,"thread_start_us":113525,"threads_started":1,"wal-append.queue_time_us":1303710}
I20260709 13:02:50.583367  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.619668  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:50.663228  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:50.630605  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.627552  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.667229  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:50.667229  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:50.670550  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:50.627516  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:50.672941  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 53401.4us
I20260709 13:02:50.632299  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:50.627249  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.669579  1692 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:50.674893  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.671136  2230 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 15.
I20260709 13:02:50.676424  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.676774  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:50.677390  1540 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:50.677837  2234 raft_consensus.cc:2410] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:02:50.677515  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.678858  2230 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:50.677300  2231 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:50.630908  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:50.598846  8843 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.588855  8832 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:50.627364  8646 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:50.777523  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1645) took 1694 ms. Trace:
I20260709 13:02:50.778769  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.619410  8782 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:50.782342  8782 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:50.783283  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:50.783927  1553 tablet_copy_source_session.cc:215] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:02:50.784261  1542 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:02:50.785701  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:50.782344  8707 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:50.595608  8834 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.600788  8783 raft_consensus.cc:3055] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:50.601449  8859 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:50.763693  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:50.789799  2231 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:50.789721  8859 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.790866  8850 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:50.792049  8859 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:50.793025  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2875) took 1588 ms. Trace:
I20260709 13:02:50.793407  1609 rpcz_store.cc:276] 0709 13:02:49.204951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.205334 (+   383us) service_pool.cc:224] Handling call
0709 13:02:50.792879 (+1587545us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b5400105754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:50.792971 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1103772032}
I20260709 13:02:50.794091  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1610) took 2065 ms. Trace:
I20260709 13:02:50.764901  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.594781  8844 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:50.766466  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:50.766407  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.795950  1657 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:50.796191  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:50.766191  8876 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:50.767608  2237 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:50.680652  8833 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:50.626027  8870 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.802657  2231 raft_consensus.cc:2410] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
W20260709 13:02:50.769135  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 95283.6us
I20260709 13:02:50.771018  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.667366  8929 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:50.672437  8237 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:50.676028  8849 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:50.777920  8646 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.780714  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:50.600144  8721 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:50.794483  1685 rpcz_store.cc:276] 0709 13:02:48.729023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.729167 (+   144us) service_pool.cc:224] Handling call
0709 13:02:48.729497 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.243098 (+1513601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.793855 (+550757us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540028ebd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:50.794028 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3445813120}
I20260709 13:02:50.778120  2229 rpcz_store.cc:276] 0709 13:02:49.083247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.083356 (+   109us) service_pool.cc:224] Handling call
0709 13:02:49.820339 (+736983us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b5800010598. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c9c7b71 00007fe08c9c7eee 0000000000375939 00007fe094a3b1e0 00007fe094b040c4 00007fe094b04131 00007fe094ac5fc0 00007fe094ab0929 00007fe094ab03d2 00007fe08e027bd4
0709 13:02:49.820526 (+   187us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:50.362241 (+541715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.771656 (+409415us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b540065a9d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:50.777406 (+  5750us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1245104128}
I20260709 13:02:50.810935  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:50.813225  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:50.771672  8620 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.815407  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3498) took 1970 ms. Trace:
I20260709 13:02:50.680810  8881 consensus_queue.cc:1059] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.123s
I20260709 13:02:50.762809  8613 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:50.820978  8636 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:50.839886  2234 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:02:50.823594  1541 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 17. (term mismatch)
I20260709 13:02:50.842340  1541 pending_rounds.cc:85] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 16
I20260709 13:02:50.842700  1541 pending_rounds.cc:107] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 13:02:50.843000  1541 raft_consensus.cc:2889] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:50.826017  1534 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:50.796031  8834 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:50.828263  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:50.766005  8789 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:50.831478  1538 rpcz_store.cc:276] 0709 13:02:48.845596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:48.846030 (+   434us) service_pool.cc:224] Handling call
0709 13:02:48.846359 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:49.903348 (+1056989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.815173 (+911825us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540012e3d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:50.815350 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2397538048}
I20260709 13:02:50.834533  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:50.806492  8884 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:02:50.805759  8893 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:50.800150  8783 raft_consensus.cc:740] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:50.849004  2237 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:50.852118  1539 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 22: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:50.854496  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:50.855383  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 44586.8us
I20260709 13:02:50.861049  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:50.866555  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:50.813212  8833 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:50.829062  8646 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:50.994434  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:50.867939  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:50.869442  2229 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:50.869638  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:50.919535  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.992223  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:50.834244  8831 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:50.997102  1617 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:50.998613  2237 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
W20260709 13:02:51.001010  8912 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.934s	user 0.002s	sys 0.000s
I20260709 13:02:51.003875  8920 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:51.004333  8920 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:51.004662  8844 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:51.005084  8844 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:51.005465  8844 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:51.005916  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:51.006515  1615 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:50.858947  8909 raft_consensus.cc:3055] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:51.011389  8909 raft_consensus.cc:740] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:51.012300  8909 consensus_queue.cc:260] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.811870  8859 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:02:50.854005  8547 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:51.016677  8859 raft_consensus.cc:3055] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:50.808917  8889 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
W20260709 13:02:51.012225  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.839754  8636 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:51.018281  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 146573us
I20260709 13:02:50.723690  8900 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:50.850669  8867 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:51.030238  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:51.017150  8859 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:50.863567  8893 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:50.870577  8560 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:50.873272  8525 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:50.993157  8907 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.958s	user 0.000s	sys 0.003s
I20260709 13:02:51.047250  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:50.860771  8789 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.058652  8934 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:51.050180  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:51.060274  8789 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:51.060948  2229 raft_consensus.cc:1240] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:51.004705  8920 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:51.054387  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:50.851336  8620 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:50.851706  8613 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.036116  8909 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:51.061110  1686 raft_consensus.cc:2410] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 13:02:51.062826  2237 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:50.838608  8237 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:50.829315  8929 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.041147  8636 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:02:51.084167  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:50.862789  8873 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:51.052645  8893 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:51.050318  8560 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.043784  8831 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:51.078028  8857 consensus_meta.cc:287] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.697s	user 0.011s	sys 0.003s
W20260709 13:02:51.111478  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 504638us
I20260709 13:02:51.135532  1540 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:51.139405  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:51.125584  1617 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:51.081091  8850 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.038681  8776 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:02:51.138897  2230 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 13:02:51.140735  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:51.150012  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:51.111384  8925 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 13:02:51.156188  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:51.085996  8867 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:51.096478  8869 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:51.172425  1507 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:51.132777  2237 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:51.139719  8560 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:51.178622  1583 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:51.078563  8525 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.145483  8824 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:02:51.169025  8789 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:51.164031  8941 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:02:51.197278  8933 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.425s	user 0.001s	sys 0.001s
I20260709 13:02:51.097482  8783 consensus_queue.cc:260] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.181944  8869 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:02:51.196090  8547 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:51.164057  8850 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.169992  8832 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:51.261647  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 223704us
I20260709 13:02:51.110615  8833 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.288499  8895 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:51.138357  8859 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.268471  8579 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.272954  8713 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.289912  8579 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.290057  8713 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.292201  8579 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:51.292771  8579 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.273025  8237 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:51.261667  8934 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.297549  1615 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:51.275429  8941 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:51.293215  8904 consensus_queue.cc:1059] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:51.282009  8773 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.340349  8773 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.277477  8789 consensus_queue.cc:1059] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:02:51.298692  8547 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:02:51.324824  1690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.009s	sys 0.024s
W20260709 13:02:51.357963  1690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.009s	sys 0.024s
I20260709 13:02:51.332849  1689 raft_consensus.cc:1275] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:51.295588  8870 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:51.302201  8783 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:51.280393  8824 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:51.346448  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:51.366957  1687 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:51.293210  8579 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.279263  8940 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 13:02:51.363029  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1648) took 1877 ms. Trace:
I20260709 13:02:51.370277  1689 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 13:02:51.390200  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.355751  8773 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:51.374190  8935 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.356s	user 0.000s	sys 0.001s
I20260709 13:02:51.407133  1541 raft_consensus.cc:1240] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:02:51.407325  1538 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 13:02:51.407635  8579 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:51.454162  8904 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:51.459457  8904 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.373032  2233 rpcz_store.cc:276] 0709 13:02:49.485273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:49.485444 (+   171us) service_pool.cc:224] Handling call
0709 13:02:49.511532 (+ 26088us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:49.618564 (+107032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:49.618610 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:50.409530 (+790920us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b2c000cc068. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:02:50.410152 (+   622us) log.cc:844] Serialized 26 byte log entry
0709 13:02:50.470558 (+ 60406us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:50.470581 (+    23us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:50.470628 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:51.349119 (+878491us) raft_consensus.cc:1685] finished
0709 13:02:51.349149 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:51.362928 (+ 13779us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36365,"raft.queue_time_us":205497,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":9561,"spinlock_wait_cycles":5578624,"thread_start_us":145078,"threads_started":2,"wal-append.queue_time_us":216411}
I20260709 13:02:51.461361  8904 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:51.440869  8613 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:51.399158  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:51.462504  8897 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:51.397078  8882 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 13:02:51.463292  8897 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.440507  8929 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:51.443202  8547 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.451745  8636 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:51.473001  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.473855  8900 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:02:51.442703  8831 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:51.476063  8636 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:51.478183  8943 log.cc:927] Time spent T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:02:51.442003  8832 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:51.480896  8850 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:51.488629  8941 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:51.492944  8942 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 13:02:51.428752  8525 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:51.308954  8934 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.315626  8867 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:51.374616  8938 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.298s	user 0.000s	sys 0.002s
W20260709 13:02:51.513139  8782 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.730s	user 0.011s	sys 0.000s
I20260709 13:02:51.514091  8782 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.531354  8897 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:51.538838  8934 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:51.496760  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.398465  8620 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:51.548307  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:51.548306  8620 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.002s
W20260709 13:02:51.575841  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.578858  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:51.579888  1538 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:51.580096  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:51.580686  1541 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:51.580751  8525 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:02:51.585397  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 413019us
I20260709 13:02:51.583813  1692 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
I20260709 13:02:51.587754  1687 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:51.372159  8824 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:51.585028  8834 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.682s	user 0.007s	sys 0.003s
I20260709 13:02:51.605398  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:51.592304  8844 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.584s	user 0.010s	sys 0.000s
I20260709 13:02:51.616351  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:51.609035  1507 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:51.618887  1582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.029s	sys 0.065s
W20260709 13:02:51.604810  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 340212us
I20260709 13:02:51.617163  2232 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:51.597322  8824 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:02:51.625219  1508 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.817s	user 0.020s	sys 0.028s
W20260709 13:02:51.627429  1508 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.021s	sys 0.028s
I20260709 13:02:51.634106  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:51.608366  8834 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:02:51.621670  1582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.030s	sys 0.065s
I20260709 13:02:51.610657  8934 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:51.651396  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:51.673960  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.674640  1508 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:51.658160  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:51.678165  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:51.679018  1690 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:51.684461  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:51.685134  1688 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:51.686170  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:51.696071  1534 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:51.693756  1507 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:51.698390  2228 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
W20260709 13:02:51.696659  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:51.649895  8870 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:51.708599  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:51.648051  8940 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:51.651063  8833 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:51.701922  8893 consensus_meta.cc:287] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.545s	user 0.007s	sys 0.004s
W20260709 13:02:51.709795  1509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.912s	user 0.018s	sys 0.052s
I20260709 13:02:51.710299  8951 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:51.615855  8844 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:51.709228  8720 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.016s	sys 0.037s
I20260709 13:02:51.619410  8613 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 13:02:51.676108  8789 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.649796  8954 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.696108  8237 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:02:51.705622  8646 consensus_meta.cc:287] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.668s	user 0.012s	sys 0.000s
W20260709 13:02:51.723157  1509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.019s	sys 0.052s
I20260709 13:02:51.720527  8579 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.698894  8869 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:51.757262  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 959654us
I20260709 13:02:51.722267  8949 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:51.697865  1508 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:51.706485  8840 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:51.722554  8920 consensus_meta.cc:287] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.631s	user 0.012s	sys 0.000s
I20260709 13:02:51.677842  8636 consensus_queue.cc:1059] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 13:02:51.716120  8870 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:51.722697  8904 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:51.701900  8834 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.729578  1537 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
W20260709 13:02:51.730647  8720 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.016s	sys 0.037s
I20260709 13:02:51.703261  8934 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.836432  8934 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:51.766215  8921 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.011s	sys 0.008s
W20260709 13:02:51.837370  8921 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.012s	sys 0.008s
I20260709 13:02:51.726891  8951 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:51.838388  8951 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:51.725837  8833 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.842626  8833 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:51.756625  8237 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:51.843544  8237 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:51.779639  8949 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.726539  8893 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.853544  8893 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:51.830605  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:51.822566  8870 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:51.767004  8773 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.755474  8789 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.831681  8904 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.819886  8840 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.759963  8954 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.850601  8951 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.851311  8889 raft_consensus.cc:3055] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Stepping down as leader of term 23
W20260709 13:02:51.865902  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:51.765767  8560 consensus_meta.cc:287] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.513s	user 0.008s	sys 0.004s
I20260709 13:02:51.761910  8957 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:02:51.898171  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 221207us
I20260709 13:02:51.772503  8579 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:51.830799  8782 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:51.829510  8620 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:51.884888  8893 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:51.901206  1617 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:51.822067  8876 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 13:02:51.766448  8646 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:51.904757  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 211111us
I20260709 13:02:51.920130  1541 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19. Ops: []
I20260709 13:02:51.920361  1507 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:51.906731  1618 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:51.777016  8869 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:02:51.900372  1615 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 13:02:51.897110  8940 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:51.900215  8889 raft_consensus.cc:740] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:51.938619  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:51.924305  1618 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:51.822935  8920 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.856186  8834 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.954548  1618 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:51.930023  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:51.884150  8636 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 13:02:51.931051  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:51.923403  1683 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 13:02:51.903568  8957 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:51.947573  8025 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.947659  1617 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:51.885195  8773 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:51.952610  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:51.958745  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:51.969376  1615 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 13:02:51.969647  1614 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:51.882777  8958 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
W20260709 13:02:51.964524  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 201729us
I20260709 13:02:51.964862  8025 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.971542  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:51.974479  1691 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:51.974630  1612 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:51.926187  8893 raft_consensus.cc:697] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:51.909065  8560 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.979693  1613 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:51.965798  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:51.952793  8867 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.983646  1618 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:51.924085  8620 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:51.988229  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:51.997215  1616 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:52.000628  8025 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:51.980949  8974 raft_consensus.cc:3055] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:02:52.004033  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 53528.4us
I20260709 13:02:51.974086  8934 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.983053  8970 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:51.970362  8957 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:51.973583  8848 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:51.984524  8950 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:51.986243  8940 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.009528  1610 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:02:52.020371  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:51.990427  8941 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:02:52.028301  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44305.5us
I20260709 13:02:52.035092  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:52.035266  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.007256  8974 raft_consensus.cc:740] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:52.019457  8876 raft_consensus.cc:3055] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:52.039986  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:52.019256  8844 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:52.018591  8950 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:51.996407  8867 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:52.034945  8962 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.258s	user 0.001s	sys 0.001s
I20260709 13:02:52.062124  1509 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:52.042260  8967 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:02:52.063257  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:52.037075  8963 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
W20260709 13:02:52.061246  8961 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 13:02:52.043751  8876 raft_consensus.cc:740] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:52.028411  8848 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:52.018337  8934 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:52.079753  8789 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:52.081192  8867 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:52.026790  8970 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.081254  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3464) took 1215 ms. Trace:
I20260709 13:02:52.088148  1615 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:52.132321  1618 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 25. Current term is 26. Ops: []
W20260709 13:02:52.089525  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.062659  8950 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:52.131327  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 3467) took 1221 ms. Trace:
I20260709 13:02:52.144865  1692 rpcz_store.cc:276] 0709 13:02:50.866982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.867701 (+   719us) service_pool.cc:224] Handling call
0709 13:02:52.087998 (+1220297us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b54000f6c54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:52.088111 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1351296768}
I20260709 13:02:52.145689  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.146720  1616 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:52.147136  8848 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:52.148679  8981 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:02:52.030927  8889 consensus_queue.cc:260] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:52.149884  8954 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:52.150095  8889 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:52.132001  1691 rpcz_store.cc:276] 0709 13:02:50.866113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.870085 (+  3972us) service_pool.cc:224] Handling call
0709 13:02:51.013886 (+143801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.974877 (+960991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:52.081013 (+106136us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b540014a854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:52.081191 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2208637696}
W20260709 13:02:52.151350  8969 log.cc:927] Time spent T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:02:52.153628  8970 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:52.153942  8972 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 13:02:52.090528  8893 consensus_queue.cc:237] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:52.154963  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.155819  1615 raft_consensus.cc:2410] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 13:02:52.156652  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:52.089898  8968 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:02:52.142807  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.157256  1615 raft_consensus.cc:2376] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18. Re-sending same reply.
W20260709 13:02:52.162066  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 98103.7us
I20260709 13:02:52.166643  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.030248  8940 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.171980  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:52.171975  1616 raft_consensus.cc:2410] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 13:02:52.173079  1618 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:52.058804  8941 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:52.178390  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 480583us
I20260709 13:02:52.201064  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:52.204247  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:52.216151  1686 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2447) took 1633 ms (client timeout 1847 ms). Trace:
I20260709 13:02:52.216575  1612 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:52.217650  8858 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.218097  1508 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:52.218186  8858 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.219130  8861 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:52.220582  8858 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:52.221117  8858 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.222772  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:52.223591  1537 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 18: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:52.223767  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:52.224750  1536 raft_consensus.cc:2393] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:52.225612  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:52.226125  8966 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260709 13:02:52.227078  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:52.227339  1537 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:52.227722  1543 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:52.206861  1611 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:40914 (request call id 4) took 1144 ms. Trace:
I20260709 13:02:52.143615  8904 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:52.153975  8920 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:52.144277  8949 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:52.157263  8636 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:52.208283  8834 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:52.209002  1618 raft_consensus.cc:1240] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:52.210016  8940 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:52.211419  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.244683  1614 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:52.211892  8789 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:02:52.212836  8973 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 13:02:52.213683  1657 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:52.225744  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:52.168790  8962 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:02:52.206125  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:52.216598  1686 rpcz_store.cc:269] 0709 13:02:50.582406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.582843 (+   437us) service_pool.cc:224] Handling call
0709 13:02:52.215962 (+1633119us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b540041e6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:52.216094 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":906371840}
W20260709 13:02:52.258196  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 44580us
I20260709 13:02:52.228565  1611 rpcz_store.cc:276] 0709 13:02:51.062840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.063158 (+   318us) service_pool.cc:224] Handling call
0709 13:02:52.206459 (+1143301us) spinlock_profiling.cc:243] Waited 47.7 ms on lock 0x7b54000875d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf2735 00007fe08faea9fa 00007fe094ab24c9 00007fe08e02ec54 00007fe08e02ebe2 00007fe08e02eb11 00007fe08e02ea8c 00007fe08e02dd42 00007fe08de2748c 00007fe08de268c6
0709 13:02:52.206739 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109702400}
I20260709 13:02:52.234961  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.259164  1657 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:52.228209  8683 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:52.169376  8951 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:52.289300  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:52.239423  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 67601us
I20260709 13:02:52.256580  1538 raft_consensus.cc:2376] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21. Re-sending same reply.
I20260709 13:02:52.255662  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:52.291890  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.216202  1615 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:52.295143  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.234629  8869 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:52.241415  8855 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.221560  8858 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.263032  8025 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:52.246157  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:52.261065  1617 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:52.205215  8840 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:52.257908  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.313758  1613 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 25: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:52.315038  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.316427  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:52.316860  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.238000  8850 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:52.235994  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:52.319176  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 83280.4us
I20260709 13:02:52.289189  8904 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:02:52.286846  8900 consensus_meta.cc:287] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.812s	user 0.012s	sys 0.002s
I20260709 13:02:52.321003  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:52.320806  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.329624  1615 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:02:52.322396  8850 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.288518  8921 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:52.288499  8834 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:52.345638  1690 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 14. (term mismatch)
I20260709 13:02:52.352944  1690 pending_rounds.cc:85] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 13
I20260709 13:02:52.353229  1690 pending_rounds.cc:107] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 25.14
I20260709 13:02:52.353624  1690 raft_consensus.cc:2889] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:52.282497  8984 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.117s
I20260709 13:02:52.282981  8940 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:52.336038  1657 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 1.539s (SENT) [suppressed 24 similar messages]
I20260709 13:02:52.305585  8025 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.301522  8975 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.237262  8978 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.099s
I20260709 13:02:52.296562  8636 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:52.234448  8974 consensus_queue.cc:260] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.328677  2203 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:02:52.416703  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 88146.8us
I20260709 13:02:52.408265  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:52.312528  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.374312  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.335398  8840 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:02:52.370880  8824 consensus_meta.cc:287] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.716s	user 0.013s	sys 0.000s
W20260709 13:02:52.394476  1428 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab5b971190d342d2b2568c30f3c0c81f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 8a6a5ae1b7f240408b3ddd30d8a18262 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:52.298169  8896 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.081s
I20260709 13:02:52.380991  8970 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:02:52.397253  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.357664  8525 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.243593  8876 consensus_queue.cc:260] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:52.466718  8876 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:52.466531  8525 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:52.468325  8941 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:52.298350  8947 consensus_queue.cc:1048] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:02:52.470403  8893 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:52.470752  8947 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:02:52.470983  8893 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.296589  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.472261  1618 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:52.473115  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.473768  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.474504  1617 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:52.474602  1618 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:02:52.475788  8986 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 13:02:52.475960  1618 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:02:52.476104  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:52.390089  8237 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.546s	user 0.013s	sys 0.000s
I20260709 13:02:52.477394  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:52.477716  8237 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.320331  8949 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.478719  8858 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:52.479056  8949 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.479610  8834 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.427732  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.480006  8237 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:52.415606  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 174514us
W20260709 13:02:52.460634  1428 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 79780.4us
I20260709 13:02:52.480141  8834 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:52.369292  8620 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:52.482385  8834 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:52.484324  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:52.485028  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:52.485744  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:52.486734  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:52.487071  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:52.487926  1691 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:52.471130  8904 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:52.489938  8904 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.492863  8904 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:52.494272  8850 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:52.495947  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:52.497206  1687 raft_consensus.cc:2410] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:52.402995  8975 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.499056  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:52.499502  8975 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:52.500135  8975 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:52.500319  1688 raft_consensus.cc:2410] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:52.506296  1689 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:52.506875  1689 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:52.507424  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:52.508637  1539 raft_consensus.cc:2410] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 7.
I20260709 13:02:52.508097  1689 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.509872  8238 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:52.510358  8238 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:52.465605  8824 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.510780  8238 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:52.511333  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:52.512411  8824 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:52.513851  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:52.514999  1687 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:52.517134  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:52.518224  1688 raft_consensus.cc:2393] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:52.341210  8855 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.368213  8987 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.134s
I20260709 13:02:52.502561  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:52.540858  1691 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:52.542460  2229 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 13:02:52.542958  2229 pending_rounds.cc:85] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 13
I20260709 13:02:52.543422  2229 pending_rounds.cc:107] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:02:52.543733  2229 raft_consensus.cc:2889] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:52.486799  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:52.488006  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:52.523140  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:52.524263  1612 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:52.525514  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:52.500499  8975 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:52.552456  1685 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 24.
I20260709 13:02:52.458614  8625 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:52.527647  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:52.441792  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 105789us
I20260709 13:02:52.510468  1689 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:52.535979  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.538647  8025 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:52.545251  1542 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:52.546032  1543 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 16.
W20260709 13:02:52.467446  8989 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:02:52.377054  8940 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:52.558558  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 141157us
I20260709 13:02:52.520526  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:52.561329  1507 leader_election.cc:336] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.847s (SENT)
I20260709 13:02:52.526983  8893 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:52.576442  1688 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:52.576383  2203 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:52.577903  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:52.579847  1507 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:52.581465  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.584867  1687 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:52.550625  8707 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.535683  8971 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:52.601339  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:52.602495  2235 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 16.
I20260709 13:02:52.588141  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:52.601459  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:52.542963  8974 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:52.600901  8025 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:52.605892  2236 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:52.604890  8707 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.606376  1691 raft_consensus.cc:2468] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:52.619031  8940 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:52.586475  8889 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.620566  8889 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:52.607580  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 46320.6us
I20260709 13:02:52.622840  8889 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:52.625118  1542 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 24.
W20260709 13:02:52.587661  8870 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.733s	user 0.003s	sys 0.009s
W20260709 13:02:52.635506  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.636176  1687 raft_consensus.cc:2410] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 20 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:02:52.637387  1689 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 17. (term mismatch)
W20260709 13:02:52.637814  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.637975  1689 pending_rounds.cc:85] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 16
I20260709 13:02:52.638341  1689 pending_rounds.cc:107] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 13:02:52.535331  8970 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.644776  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:52.645629  2236 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
W20260709 13:02:52.638746  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 125035us
I20260709 13:02:52.638631  1689 raft_consensus.cc:2889] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:52.581995  8625 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:52.649320  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.650307  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:52.652398  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:52.653107  2232 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:52.657305  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:52.666965  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:52.667645  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.679596  2235 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:52.626708  8996 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:52.610562  8971 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.780890  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:52.799528  2231 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 15.
I20260709 13:02:52.643586  8870 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:52.713704  8840 raft_consensus.cc:697] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:52.649197  8950 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:52.818261  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.645826  8975 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:52.853430  1537 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 21. Current term is 22. Ops: []
W20260709 13:02:52.860627  2262 heartbeater.cc:646] Failed to heartbeat to 127.0.107.254:36375 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.0.107.254:36375 timed out after 15.000s (SENT)
I20260709 13:02:52.875212  1539 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:52.837126  8941 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:52.927572  8025 consensus_queue.cc:1048] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:52.928534  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:52.945137  8025 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:52.814523  8620 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.086s
I20260709 13:02:52.963297  2203 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:52.876268  8889 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:52.979590  8855 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:52.852335  9001 consensus_queue.cc:1048] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.114s
I20260709 13:02:52.947197  8861 consensus_queue.cc:1048] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 13:02:52.951159  8850 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 13:02:52.874743  8625 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.951635  8941 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:52.976301  8889 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:52.999812  9000 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:02:53.016968  1610 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1301) took 2145 ms. Trace:
I20260709 13:02:53.020637  1610 rpcz_store.cc:276] 0709 13:02:50.861352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.861498 (+   146us) service_pool.cc:224] Handling call
0709 13:02:50.992437 (+130939us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:52.025798 (+1033361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.004848 (+979050us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400389d54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:53.006853 (+  2005us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2338194176}
I20260709 13:02:52.999812  9005 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:02:53.038158  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:52.983781  8840 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.075229  8025 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:53.001184  8855 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.060s
W20260709 13:02:53.086261  9003 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:02:53.099267  8889 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:53.101379  8970 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:53.102499  8870 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:53.133558  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.134971  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.138424  2232 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:53.065712  8713 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:53.158614  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2889) took 1138 ms. Trace:
I20260709 13:02:53.160626  1534 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2874) took 1761 ms. Trace:
W20260709 13:02:53.161497  2228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1746) took 2362 ms (client timeout 1539 ms). Trace:
W20260709 13:02:53.162073  2228 rpcz_store.cc:269] 0709 13:02:50.805401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.832941 (+ 27540us) service_pool.cc:224] Handling call
0709 13:02:53.161184 (+2328243us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b54004bb254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:53.161341 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1928205056}
I20260709 13:02:53.164568  1617 raft_consensus.cc:1240] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:53.159263  1616 rpcz_store.cc:276] 0709 13:02:52.019932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.020101 (+   169us) service_pool.cc:224] Handling call
0709 13:02:53.158262 (+1138161us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54002bd254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:53.158526 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":277511040}
I20260709 13:02:53.138913  8950 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:53.161160  1534 rpcz_store.cc:276] 0709 13:02:51.399063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.400686 (+  1623us) service_pool.cc:224] Handling call
0709 13:02:51.401005 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.702045 (+301040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.160288 (+1458243us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54000ec9d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:53.160545 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":678128384}
W20260709 13:02:53.178393  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 213459us
I20260709 13:02:53.179283  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:53.181124  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:53.187463  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54764 (request call id 2880) took 1432 ms. Trace:
I20260709 13:02:53.188838  9001 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:53.187978  1541 rpcz_store.cc:276] 0709 13:02:51.754631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.754802 (+   171us) service_pool.cc:224] Handling call
0709 13:02:51.770520 (+ 15718us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:51.920730 (+150210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.187169 (+1266439us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400418cd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:53.187399 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":343618304}
W20260709 13:02:53.189332  8238 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.678s	user 0.013s	sys 0.001s
I20260709 13:02:53.191850  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:53.192301  1540 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54764 (request call id 2868) took 2580 ms (client timeout 2209 ms). Trace:
W20260709 13:02:53.193269  1540 rpcz_store.cc:269] 0709 13:02:50.608159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:50.725306 (+117147us) service_pool.cc:224] Handling call
0709 13:02:53.188307 (+2463001us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54000e83d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:53.188420 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":733831424}
I20260709 13:02:53.193698  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:53.193594  9006 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.236s	user 0.002s	sys 0.002s
I20260709 13:02:53.191403  8850 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:53.204965  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:53.189327  9001 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:53.191660  8788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ba0620abe844f5bb71ba9dd947c3b9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:02:53.208640  1540 raft_consensus.cc:1240] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 13:02:53.211357  2236 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:53.204986  8941 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:53.200389  8238 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:53.220410  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.209108  9001 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:53.222705  8625 consensus_queue.cc:1059] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:02:53.233804  8773 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:53.235921  8889 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:53.236725  8889 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.239246  8238 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:53.250941  8876 consensus_meta.cc:287] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.784s	user 0.010s	sys 0.001s
W20260709 13:02:53.253037  8636 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.834s	user 0.007s	sys 0.003s
I20260709 13:02:53.253566  8636 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:02:53.255475  8957 consensus_meta.cc:287] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.225s	user 0.011s	sys 0.002s
I20260709 13:02:53.255923  8957 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:53.256919  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 1305) took 1025 ms. Trace:
I20260709 13:02:53.257612  1538 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:53.257303  1612 rpcz_store.cc:276] 0709 13:02:52.231596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.288956 (+ 57360us) service_pool.cc:224] Handling call
0709 13:02:53.256705 (+967749us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54000c7e54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:53.256825 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538311552}
I20260709 13:02:53.260383  1538 raft_consensus.cc:1240] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 13:02:53.243265  8892 raft_consensus.cc:3055] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:53.262283  8625 consensus_queue.cc:1059] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:02:53.263480  9013 raft_consensus.cc:3055] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:02:53.263847  9013 raft_consensus.cc:740] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:53.264739  9013 consensus_queue.cc:260] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.266827  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:53.268292  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.268420  8831 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:53.268838  8831 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.269181  8831 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:02:53.270308  1582 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.024s	sys 0.046s
W20260709 13:02:53.270664  1582 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.024s	sys 0.046s
W20260709 13:02:53.270955  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 811320us
W20260709 13:02:53.272639  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.273123  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.273957  1657 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:53.276918  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:53.277601  8636 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:02:53.278447  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:53.279069  2235 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:53.277772  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:53.283010  2228 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
W20260709 13:02:53.283630  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.283794  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.283833  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:53.284165  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.284734  2203 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:53.285321  8025 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:53.289149  1535 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:53.239920  8889 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:53.280181  8957 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:53.296206  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.310498  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:53.301045  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 70670.4us
I20260709 13:02:53.303475  1611 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:02:53.304261  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.305413  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:53.306241  8958 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.307s	user 0.007s	sys 0.007s
I20260709 13:02:53.306180  9018 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:53.307381  1689 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:53.311136  1618 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:53.312042  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.316425  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:53.317406  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:53.313310  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:53.277187  8850 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:02:53.322830  1614 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 22: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:53.331499  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.331844  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.280591  9013 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:02:53.326545  8958 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.327s	user 0.007s	sys 0.007s
I20260709 13:02:53.324656  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.341657  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:53.342142  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:53.342465  1688 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:02:53.289834  8947 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.325073  1613 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:53.345727  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:53.346243  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:53.325659  1691 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:53.308377  8870 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:53.328905  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 44246us
I20260709 13:02:53.325954  8840 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:53.351454  8840 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.351927  8840 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:53.353574  8850 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.354099  8850 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.356240  8850 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:53.359236  9001 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.308595  8636 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:02:53.361959  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.322629  9012 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.363173  9012 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.364176  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:53.365383  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:53.366011  1689 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:53.366053  8840 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:53.367002  8974 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.722s	user 0.010s	sys 0.002s
I20260709 13:02:53.262236  8892 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:53.367038  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:53.367964  1686 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:53.323979  9018 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:53.368238  8892 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.368948  9018 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:53.369510  8892 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:53.370677  1542 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:53.322435  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.326210  1537 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 22: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:53.332504  1618 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:53.326771  1683 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:02:53.332192  1610 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:53.332279  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:53.372107  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.372534  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.372865  1618 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:53.373973  1618 raft_consensus.cc:1240] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 21. Current term is 22. Ops: []
W20260709 13:02:53.376581  8990 log.cc:927] Time spent T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 1.031s	user 0.000s	sys 0.003s
I20260709 13:02:53.377414  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:53.321983  8889 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:53.324499  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:53.391691  1687 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:53.334484  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.392565  1692 raft_consensus.cc:2410] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:02:53.392997  1617 raft_consensus.cc:2410] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:02:53.290040  8861 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:53.307775  8941 consensus_queue.cc:1059] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:02:53.300323  8620 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:53.396176  8870 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:53.396450  1617 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:53.396432  8620 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.323457  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:53.349588  8947 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.395146  8889 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.396636  8870 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:53.402788  8870 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.348950  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:53.398159  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:53.404592  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:53.350229  1658 connection.cc:256] RPC call timeout handler was delayed by -1.300905336000028s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
W20260709 13:02:53.405212  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.405546  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2554667070000392s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1627s for a response
W20260709 13:02:53.405882  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.406134  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2239667070000451s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16620000000000001s for a response
W20260709 13:02:53.406394  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2172053360000064s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
W20260709 13:02:53.406641  1658 connection.cc:256] RPC call timeout handler was delayed by -1.2167667070000334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.167s for a response
W20260709 13:02:53.406881  1658 connection.cc:256] RPC call timeout handler was delayed by -1.1983053360000326s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2096s for a response
W20260709 13:02:53.407224  1658 connection.cc:256] RPC call timeout handler was delayed by -1.1281346750000125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1761s for a response
I20260709 13:02:53.362653  1613 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:53.306599  9022 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.407532  8831 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.408267  9022 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.409825  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:53.410422  9022 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:53.335242  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:53.411072  8975 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:53.393783  9001 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:53.329588  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 55700.6us
I20260709 13:02:53.348052  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:53.396636  8861 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:53.414057  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 66086.4us
I20260709 13:02:53.414783  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:53.415443  8947 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:53.415786  8861 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:53.416102  8947 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:53.404815  8870 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:53.416842  8947 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:53.410477  1539 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:02:53.417491  1612 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:53.417663  8613 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.418285  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:53.418169  8613 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:53.418951  1539 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:53.419360  8934 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:53.419834  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:53.420332  8613 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:53.420571  1539 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:53.420883  8613 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:53.407567  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.417344  8947 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.421403  8613 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:53.423163  1536 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:53.423230  9012 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:53.421689  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0232667070000048s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1885s for a response
I20260709 13:02:53.289804  8625 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:53.416901  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1424) took 2382 ms. Trace:
I20260709 13:02:53.424664  2229 rpcz_store.cc:276] 0709 13:02:51.034310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.108174 (+ 73864us) service_pool.cc:224] Handling call
0709 13:02:51.154541 (+ 46367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.544265 (+1389724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.416662 (+872397us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54005361d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:53.416819 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3190897920}
I20260709 13:02:53.403227  8889 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:53.424742  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:53.427552  1582 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:53.426234  1539 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:02:53.387705  1614 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 24. Current term is 25. Ops: []
W20260709 13:02:53.446549  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 160841us
I20260709 13:02:53.449229  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:53.450284  2235 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 25.
I20260709 13:02:53.451962  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:53.452770  2231 raft_consensus.cc:2410] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:02:53.454602  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:53.455950  2234 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:02:53.456045  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:53.457119  2233 raft_consensus.cc:2393] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 20: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:53.458562  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:53.422058  8947 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:53.422147  8840 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:53.480652  2235 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:53.483300  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:53.483495  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.496479  1614 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:53.497820  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.498636  1614 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:53.500083  1614 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 13:02:53.475986  1685 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:53.484647  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:53.485535  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.487360  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:53.489275  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:53.482788  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:53.475783  1509 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:53.484210  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:53.503090  1613 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:53.470762  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:53.493644  2232 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:53.505020  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:53.505759  1509 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:53.449286  9022 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:53.504242  1536 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:53.507934  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:53.511015  2203 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:53.512262  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:53.515918  2233 raft_consensus.cc:1240] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:53.455890  9012 raft_consensus.cc:2804] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:53.449200  8889 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:53.528923  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:53.534188  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:53.528190  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:53.449218  8625 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.535033  8950 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 13:02:53.593724  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:53.525483  9022 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.530194  8876 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:53.525286  8831 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:53.492719  9013 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.103s
W20260709 13:02:53.628410  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 215225us
I20260709 13:02:53.676352  8994 raft_consensus.cc:3055] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:53.676702  1688 raft_consensus.cc:2410] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 18 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:02:53.676846  8547 raft_consensus.cc:3055] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:53.678172  8646 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:53.679989  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.488832  9008 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.107s
I20260709 13:02:53.681236  8840 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:53.523061  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:53.681586  8971 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:53.507700  8941 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:53.470384  8960 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.084s
I20260709 13:02:53.683117  8894 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.682806  8889 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:53.684926  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:53.793678  8876 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.685364  1609 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43412 (request call id 2878) took 1786 ms. Trace:
W20260709 13:02:53.686266  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.686393  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:53.686463  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:53.689620  8850 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:02:53.834049  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 311137us
I20260709 13:02:53.690367  8832 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:53.692991  9012 raft_consensus.cc:697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:53.809945  8789 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 13:02:53.697294  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.698508  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 3511) took 1215 ms. Trace:
I20260709 13:02:53.701906  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:53.848418  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 146660us
I20260709 13:02:53.850541  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:53.702594  8858 raft_consensus.cc:3055] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:53.707216  8579 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:53.708932  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2452) took 2025 ms. Trace:
I20260709 13:02:53.712625  8560 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:53.843078  8850 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.522280  8974 consensus_queue.cc:1059] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.078s
I20260709 13:02:53.715019  8983 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 13:02:53.715480  8980 log.cc:927] Time spent T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 1.210s	user 0.004s	sys 0.009s
W20260709 13:02:53.722222  9007 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.744s	user 0.002s	sys 0.000s
I20260709 13:02:53.740780  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:53.741870  9005 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.742460  8957 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.843701  8894 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.530833  9001 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:53.746598  1582 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:53.882791  9035 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:53.733412  1434 catalog_manager.cc:5697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:53.773799  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:53.938985  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.940697  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:53.941679  1691 raft_consensus.cc:2410] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 27 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
W20260709 13:02:53.942767  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 166533us
I20260709 13:02:53.812002  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:53.799924  8547 raft_consensus.cc:740] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:53.792995  9022 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:53.839342  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.840157  2233 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:53.797916  8994 raft_consensus.cc:740] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:53.825635  1609 rpcz_store.cc:276] 0709 13:02:51.899047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.899196 (+   149us) service_pool.cc:224] Handling call
0709 13:02:51.921771 (+ 22575us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:51.922173 (+   402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:51.922203 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:51.925316 (+  3113us) log.cc:844] Serialized 52 byte log entry
0709 13:02:53.155107 (+1229791us) spinlock_profiling.cc:243] Waited 6.05 ms on lock 0x7b48001a01d8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe094a15aa1 00007fe094a0cb37 00007fe094a0c9ca 00007fe094b0426f 00007fe094b03f37 00007fe094b1855c 00007fe094b18508 00007fe094b18488 00007fe094b18443 00007fe094b176ec 00007fe093a41a96
0709 13:02:53.269842 (+114735us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b1800391de0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:02:53.305546 (+ 35704us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:53.305862 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 13:02:53.306072 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.306146 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:53.684986 (+378840us) raft_consensus.cc:1685] finished
0709 13:02:53.685007 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:53.685303 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2417,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":3922,"raft.run_cpu_time_us":13249,"raft.run_wall_time_us":1229632,"spinlock_wait_cycles":16717440,"thread_start_us":35467,"threads_started":1,"wal-append.queue_time_us":1513206}
I20260709 13:02:53.845377  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:53.848029  1535 rpcz_store.cc:276] 0709 13:02:52.482822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.484681 (+  1859us) service_pool.cc:224] Handling call
0709 13:02:53.698205 (+1213524us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b54003453d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:53.698385 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1767247744}
I20260709 13:02:53.813820  8613 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:53.819646  8941 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.529593  8636 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.090s
I20260709 13:02:53.834517  8876 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:53.851171  8789 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:53.853976  1690 rpcz_store.cc:276] 0709 13:02:51.683203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:51.687408 (+  4205us) service_pool.cc:224] Handling call
0709 13:02:51.687941 (+   533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:52.354102 (+666161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.708535 (+1354433us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54000f5354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:53.708831 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1511300352}
I20260709 13:02:53.849843  9025 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:53.852114  8773 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:02:53.903122  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:53.852272  8858 raft_consensus.cc:740] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:53.853684  8707 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:53.852500  8525 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:53.901507  1612 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 24.
I20260709 13:02:53.967224  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:53.524991  8855 raft_consensus.cc:3055] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:53.883540  8850 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:53.878374  9024 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:53.935567  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 189089us
I20260709 13:02:53.956281  8941 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:53.938118  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 143958us
I20260709 13:02:53.808542  1613 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:53.975425  1657 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:53.945729  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:53.854738  8832 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.897749  8983 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:53.948453  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:53.943960  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:53.982683  1541 raft_consensus.cc:2410] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:53.893987  9009 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:02:53.988230  9009 raft_consensus.cc:3055] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:02:53.988658  9009 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:53.911477  8949 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:53.882674  8579 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 13:02:53.989722  8994 consensus_queue.cc:260] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.991358  8859 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:53.991813  9032 log.cc:927] Time spent T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:02:53.992359  8858 consensus_queue.cc:260] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:53.994109  9034 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.087s	user 0.003s	sys 0.000s
I20260709 13:02:53.995085  8983 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.981031  8560 raft_consensus.cc:2764] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
W20260709 13:02:53.997203  9031 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 13:02:53.949055  9035 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:53.998881  8707 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:02:53.999874  8788 consensus_meta.cc:287] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.596s	user 0.013s	sys 0.000s
I20260709 13:02:53.936288  8870 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.972247  8525 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:53.961197  1542 raft_consensus.cc:3060] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:53.956558  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 121158us
I20260709 13:02:53.939072  8620 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:53.910197  8238 consensus_queue.cc:1059] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:02:53.878708  8934 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:02:53.954967  1441 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55368 (request call id 328) took 61139 ms (client timeout 15000 ms). Trace:
I20260709 13:02:53.934813  8957 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:53.954691  9025 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:53.953848  8861 consensus_queue.cc:1059] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 13:02:53.967926  9012 consensus_queue.cc:237] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:53.946708  1434 catalog_manager.cc:5697] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 15 to 19, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 19 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:53.976832  1687 raft_consensus.cc:3060] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 20
W20260709 13:02:53.749399  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:53.961820  8773 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:53.988521  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 85682.1us
I20260709 13:02:53.950160  1539 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:54.001384  8788 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:39466 (request call id 1378) took 6863 ms. Trace:
W20260709 13:02:53.984083  8892 consensus_meta.cc:287] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.612s	user 0.009s	sys 0.013s
I20260709 13:02:54.017665  8788 rpcz_store.cc:276] 0709 13:02:47.137717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:47.140868 (+  3151us) service_pool.cc:224] Handling call
0709 13:02:47.276769 (+135901us) ts_tablet_manager.cc:934] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459)
0709 13:02:52.942609 (+5665840us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:02:53.403417 (+460808us) tablet_metadata.cc:665] Metadata flushed
0709 13:02:53.403547 (+   130us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:02:54.000399 (+596852us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:02:54.000836 (+   437us) tablet_replica.cc:206] Creating consensus instance
0709 13:02:54.001219 (+   383us) ts_tablet_manager.cc:989] Replica registered.
0709 13:02:54.001294 (+    75us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":20254,"tablet-copy.queue_time_us":134912,"thread_start_us":248495,"threads_started":2}
I20260709 13:02:54.019299  8788 tablet_copy_client.cc:806] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 13:02:54.019961  8788 tablet_copy_client.cc:670] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
W20260709 13:02:54.032699  9030 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:02:53.969508  8855 raft_consensus.cc:740] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:54.034094  8855 consensus_queue.cc:260] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:54.038865  8789 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:54.040222  8789 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:54.043812  1539 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:54.045697  8625 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:54.045862  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.046994  9035 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:54.047129  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 98750us
I20260709 13:02:54.047725  8994 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:54.048676  2201 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:54.048974  8889 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:54.049759  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.050181  8894 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:54.051024  8947 consensus_meta.cc:287] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.002s
W20260709 13:02:54.001667  9018 consensus_meta.cc:287] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.632s	user 0.012s	sys 0.000s
W20260709 13:02:53.994258  9029 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 13:02:54.021605  1684 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:54.051393  8957 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:54.053866  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.054746  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.054936  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.055368  1534 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:54.056216  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 27 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:54.002229  8975 consensus_meta.cc:287] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.591s	user 0.013s	sys 0.000s
I20260709 13:02:54.050906  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.063609  8947 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:54.056272  8625 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:54.072643  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.002058  8525 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:54.075282  8525 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:54.001992  8870 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:54.060391  2201 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:54.002781  9033 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 13:02:54.061997  1534 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 13:02:54.004096  1441 rpcz_store.cc:269] 0709 13:01:52.795862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:01:52.796217 (+   355us) service_pool.cc:224] Handling call
0709 13:02:53.910239 (+61114022us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:46.169343 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:46.170353 (+  1010us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:46.170410 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:02:46.170452 (+    42us) tablet.cc:662] Decoding operations
0709 13:02:46.171756 (+  1304us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:46.171877 (+   121us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:46.171942 (+    65us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:02:46.173171 (+  1229us) tablet.cc:701] Row locks acquired
0709 13:02:46.173237 (+    66us) write_op.cc:260] PREPARE: finished
0709 13:02:46.210186 (+ 36949us) write_op.cc:270] Start()
0709 13:02:46.210409 (+   223us) write_op.cc:276] Timestamp: P: 1783602166210061 usec, L: 0
0709 13:02:46.210454 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 13:02:46.247227 (+ 36773us) log.cc:844] Serialized 2861 byte log entry
0709 13:02:46.839990 (+592763us) op_driver.cc:464] REPLICATION: finished
0709 13:02:47.025336 (+185346us) write_op.cc:301] APPLY: starting
0709 13:02:47.025603 (+   267us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:47.261157 (+235554us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:47.261286 (+   129us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:47.368418 (+107132us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:47.368769 (+   351us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 13:02:47.368834 (+    65us) write_op.cc:312] APPLY: finished
0709 13:02:47.369188 (+   354us) log.cc:844] Serialized 73 byte log entry
0709 13:02:47.369388 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:47.369529 (+   141us) write_op.cc:454] Released schema lock
0709 13:02:47.369599 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:47.370156 (+   557us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:47.370188 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":97677,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":177287,"apply.run_cpu_time_us":15534,"apply.run_wall_time_us":344800,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33429,"num_ops":7,"prepare.queue_time_us":770570,"prepare.run_cpu_time_us":28835,"prepare.run_wall_time_us":180103,"raft.queue_time_us":153880,"raft.run_cpu_time_us":45342,"raft.run_wall_time_us":176556,"replication_time_us":615406,"thread_start_us":988704,"threads_started":4,"wal-append.queue_time_us":306204,"wal-append.run_cpu_time_us":20174,"wal-append.run_wall_time_us":1819293}]]}
I20260709 13:02:54.066859  9044 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.013121  9036 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:54.090221  9042 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:54.090054  9044 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:54.067065  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:54.004105  9023 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 13:02:54.005092  9025 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:53.933864  9005 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:54.070570  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.108170  1539 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:54.071940  1543 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3517) took 1587 ms. Trace:
I20260709 13:02:54.066442  8850 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:53.996405  8983 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:02:53.970115  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 88074.1us
I20260709 13:02:54.015372  8858 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:54.051563  9018 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:54.010524  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.083043  1542 raft_consensus.cc:2410] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 7.
I20260709 13:02:54.084203  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:53.985335  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:54.130038  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
W20260709 13:02:54.130575  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 145301us
I20260709 13:02:54.130795  1534 raft_consensus.cc:2376] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20. Re-sending same reply.
I20260709 13:02:54.086297  2201 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:54.009996  1434 catalog_manager.cc:5697] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:54.131585  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:54.093528  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:54.093813  1611 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:54.090787  9042 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.099826  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:54.133044  1688 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:54.090260  9036 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.083935  8855 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:54.010601  8773 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:02:54.100883  8894 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:54.109346  1543 rpcz_store.cc:276] 0709 13:02:52.484361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:52.574783 (+ 90422us) service_pool.cc:224] Handling call
0709 13:02:52.595347 (+ 20564us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:53.699648 (+1104301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:53.699698 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:53.700327 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 13:02:53.906048 (+205721us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:53.906753 (+   705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.011466 (+104713us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54000e92d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:54.011517 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.071641 (+ 60124us) raft_consensus.cc:1685] finished
0709 13:02:54.071663 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.071880 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15293,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":129,"spinlock_wait_cycles":2538424064,"thread_start_us":205193,"threads_started":1,"wal-append.queue_time_us":254474}
I20260709 13:02:54.084872  8957 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:54.134656  1582 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:54.135720  8025 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:54.135936  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:54.078671  1687 raft_consensus.cc:2468] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:54.137012  1541 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 23: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:02:54.139150  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.139992  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 53834.4us
I20260709 13:02:54.127939  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.141731  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:54.127569  8832 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:54.175755  1539 raft_consensus.cc:2393] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:54.175488  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:54.172392  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 197033us
I20260709 13:02:54.178117  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:54.179728  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:54.179749  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:54.172190  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:54.183903  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 49342.6us
I20260709 13:02:54.145936  9036 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:54.133607  9042 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:54.177805  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:54.137521  8560 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:02:54.186308  1434 catalog_manager.cc:5697] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:54.190002  1509 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:54.143543  8960 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:54.005491  8547 consensus_queue.cc:260] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:54.188309  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.084637  9024 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:54.202157  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:54.202818  2229 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:54.189854  2233 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:53.989770  9009 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:54.200760  1684 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:02:54.149574  8831 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:02:54.201723  1657 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:54.200268  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:54.223215  1539 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:54.186018  8707 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:54.216259  1687 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:54.153295  8857 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:54.218375  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.185992  9012 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:54.199391  8560 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.220643  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:54.199630  8613 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.211315  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.225939  1434 catalog_manager.cc:5697] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 12 to 15, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 15 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:54.236260  2201 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:54.143384  8861 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:02:54.229007  1541 raft_consensus.cc:2410] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 18 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:02:54.216660  1540 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33722 (request call id 1526) took 1015 ms. Trace:
I20260709 13:02:54.240823  1540 rpcz_store.cc:276] 0709 13:02:53.201298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.219493 (+ 18195us) service_pool.cc:224] Handling call
0709 13:02:53.221067 (+  1574us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:53.221801 (+   734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:53.221831 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:53.242075 (+ 20244us) log.cc:844] Serialized 26 byte log entry
0709 13:02:53.488474 (+246399us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:02:53.488528 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:53.489847 (+  1319us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.216449 (+726602us) raft_consensus.cc:1685] finished
0709 13:02:54.216471 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:54.216599 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":350,"raft.queue_time_us":481,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":214701,"threads_started":1,"wal-append.queue_time_us":530137}
W20260709 13:02:54.246285  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.217351  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:54.155248  8525 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 13:02:54.255264  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.232574  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.239356  1434 catalog_manager.cc:5697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:54.184140  8957 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:54.257959  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.260159  1539 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
W20260709 13:02:54.230895  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 213770us
I20260709 13:02:54.262329  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.252774  1583 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:54.226891  8707 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:54.273986  1434 catalog_manager.cc:5697] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:54.232584  8560 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:54.277014  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.265074  2234 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
W20260709 13:02:54.283519  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 81903.7us
I20260709 13:02:54.278761  1542 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:54.285321  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:54.287058  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 69842.8us
I20260709 13:02:54.279847  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:54.291298  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:54.289343  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:54.294450  1542 raft_consensus.cc:2376] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 24. Re-sending same reply.
I20260709 13:02:54.295620  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.296633  1542 raft_consensus.cc:2410] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 7.
I20260709 13:02:54.273475  9009 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:54.293851  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.293352  1434 catalog_manager.cc:5697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 15 to 18, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:54.293442  1539 raft_consensus.cc:2410] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:02:54.282742  8707 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:54.302989  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.288892  8788 tablet_copy_client.cc:538] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:02:54.282317  8889 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:02:54.286741  9013 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.286238  9008 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:02:54.305476  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 102444us
I20260709 13:02:54.306916  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.315464  1539 raft_consensus.cc:2410] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:54.306232  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:54.288082  1582 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:54.291715  8832 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:54.333973  8832 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.334576  8832 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:54.252914  9035 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:02:54.338840  9035 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:02:54.339359  9035 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:54.340341  9035 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:54.241393  8613 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:54.343873  8613 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:54.311530  1434 catalog_manager.cc:5697] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 17 to 19, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 19 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:02:54.345490  1434 catalog_manager.cc:5697] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 15 to 19, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 19 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:54.347705  1434 catalog_manager.cc:5697] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:54.357972  9035 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:54.362339  1434 catalog_manager.cc:5697] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 17 to 19, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 19 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:54.364830  1434 catalog_manager.cc:5697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:54.367249  1434 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:54.370011  8934 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:54.370636  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 82663us
I20260709 13:02:54.378772  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:54.381364  1434 catalog_manager.cc:5697] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:02:54.399422  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.414346  1537 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:54.304622  9038 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:02:54.418990  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.304749  9048 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.304529  9046 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:54.383628  8840 consensus_meta.cc:287] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.564s	user 0.008s	sys 0.006s
I20260709 13:02:54.304307  9045 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:54.310298  8889 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.458657  9045 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.304152  9041 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:54.440723  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 156286us
I20260709 13:02:54.459333  9045 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:54.459477  9041 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:54.459777  1657 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:54.460144  9041 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:54.437744  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:54.460886  1657 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:54.388685  8934 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:54.417990  8947 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:54.435256  8579 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 13:02:54.452628  8844 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.304445  8941 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:54.377563  9040 raft_consensus.cc:2804] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:54.378415  9049 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.187s
I20260709 13:02:54.386507  9013 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:54.458103  1508 connection.cc:256] RPC call timeout handler was delayed by -1.0573348739999915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
W20260709 13:02:54.491142  1508 connection.cc:256] RPC call timeout handler was delayed by -1.0483348739999769s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.189s for a response
I20260709 13:02:54.458756  8889 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:54.461961  1584 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:54.442313  8831 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 13:02:54.430023  9038 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:54.475449  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.443521  8870 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:54.457799  8994 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:54.472442  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:54.471020  9026 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:02:54.477083  9018 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:54.491545  1508 connection.cc:256] RPC call timeout handler was delayed by -1.013234873999977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
I20260709 13:02:54.465296  8892 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:54.493463  9051 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 13:02:54.550032  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:54.461977  1434 catalog_manager.cc:5697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:02:54.474503  8934 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:54.473440  8985 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:54.470582  8840 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:54.452785  8859 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 13:02:54.443819  9008 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:02:54.498234  9038 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:54.494264  8625 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:54.563198  1536 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:54.446120  9048 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:54.572021  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 258044us
I20260709 13:02:54.497318  8857 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:54.466636  9046 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:54.564157  1508 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:54.501794  9040 raft_consensus.cc:697] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:54.491760  8844 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:54.601569  8950 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:54.605621  8831 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:02:54.569895  8892 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:54.562614  8238 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:02:54.566131  9024 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:54.599305  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:54.597359  9066 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:54.575593  8613 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:02:54.600358  1508 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:54.613487  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 112982us
W20260709 13:02:54.616712  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.615590  1434 catalog_manager.cc:5697] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 17 to 19, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 19 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:02:54.626820  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.627529  1584 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:54.595456  2229 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
I20260709 13:02:54.572695  8870 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:02:54.631918  8870 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:54.573143  9062 consensus_queue.cc:1059] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
W20260709 13:02:54.598627  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 308860us
I20260709 13:02:54.604611  9056 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:54.514544  8994 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:54.623633  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:54.624177  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 151851us
I20260709 13:02:54.643009  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:54.648196  8994 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:54.648113  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:02:54.620368  8876 consensus_meta.cc:287] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.653s	user 0.011s	sys 0.004s
W20260709 13:02:54.623499  9022 consensus_meta.cc:287] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.668s	user 0.012s	sys 0.000s
I20260709 13:02:54.600231  8857 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:54.624634  8832 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:54.756174  9050 log.cc:927] Time spent T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 13:02:54.663390  8876 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:54.792222  8850 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:54.875777  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.893738  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 250878us
W20260709 13:02:54.899382  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 241216us
W20260709 13:02:54.900777  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 277215us
I20260709 13:02:54.665982  8625 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.098s
I20260709 13:02:54.902786  1610 raft_consensus.cc:1240] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:02:54.765776  9040 consensus_queue.cc:237] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:54.893805  1536 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 3523) took 1049 ms. Trace:
I20260709 13:02:54.903784  1611 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:02:54.904116  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:54.621207  9049 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:54.956127  1612 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:54.906298  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:54.907789  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:54.908990  1617 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
W20260709 13:02:54.909143  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:54.894232  9013 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:54.933772  8025 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:54.941855  9048 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:54.978675  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:54.978948  2232 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 24. Current term is 25. Ops: []
I20260709 13:02:54.949308  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:54.979377  1689 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:02:54.979588  1688 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:54.891412  8960 raft_consensus.cc:3055] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:54.955585  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:54.956652  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:54.957918  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:54.948776  1584 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:54.958590  2231 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 22. Current term is 23. Ops: [21.14-22.15]
I20260709 13:02:54.956845  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:54.934331  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.912503  8560 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.618s	user 0.007s	sys 0.008s
W20260709 13:02:54.918623  8789 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.872s	user 0.009s	sys 0.009s
W20260709 13:02:54.968175  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.968257  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:54.968793  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 320974us
I20260709 13:02:54.955412  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:54.918025  9025 consensus_meta.cc:287] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.783s	user 0.011s	sys 0.007s
I20260709 13:02:54.955979  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:54.952385  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:54.968677  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.034345  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:54.970491  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:54.956588  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:55.035334  1611 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:55.035607  2233 raft_consensus.cc:2410] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:02:54.955592  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:54.957985  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:54.960829  1614 raft_consensus.cc:1240] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:54.910149  8579 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:54.905486  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:54.914323  8947 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:02:54.943408  9057 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.342s	user 0.003s	sys 0.006s
I20260709 13:02:54.974364  1618 raft_consensus.cc:2410] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 13:02:54.955377  1536 rpcz_store.cc:276] 0709 13:02:53.844872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.845111 (+   239us) service_pool.cc:224] Handling call
0709 13:02:54.775585 (+930474us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b540012cfd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:54.893746 (+118161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1291583360}
I20260709 13:02:54.973977  9062 raft_consensus.cc:3055] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:54.975669  8025 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:54.943042  9042 consensus_meta.cc:287] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.721s	user 0.007s	sys 0.006s
I20260709 13:02:54.953121  9065 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.048802  2229 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:55.048739  9065 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.049993  8025 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:54.972486  8859 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:55.050992  9065 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:55.051602  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 99300.8us
I20260709 13:02:55.052754  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:54.918654  8525 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:54.981401  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:54.951700  8858 consensus_meta.cc:287] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.822s	user 0.013s	sys 0.005s
W20260709 13:02:54.951951  9061 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.386s	user 0.005s	sys 0.000s
W20260709 13:02:55.061800  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 113107us
I20260709 13:02:54.948457  8857 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.030228  1692 raft_consensus.cc:3060] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:02:55.065301  2229 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:55.001348  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.031664  1686 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:02:55.036334  2236 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:02:54.952105  8625 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:55.032084  2235 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:54.982084  1535 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:55.037626  1543 raft_consensus.cc:2410] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:02:54.920394  9066 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:55.039106  1610 raft_consensus.cc:2393] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 21: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:02:55.039535  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 83718.8us
I20260709 13:02:55.039599  9013 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.078642  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:55.080664  2235 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:55.081833  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:55.088083  1688 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:02:55.074568  9008 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:55.092010  8832 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.091861  9008 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.094277  9066 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.095096  1692 raft_consensus.cc:2468] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:55.043146  8947 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:55.059718  9065 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.043895  1616 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:55.046725  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:55.102751  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 56087.7us
I20260709 13:02:55.042351  8560 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:54.971940  8974 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:55.053568  2234 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:55.082852  8870 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:54.948757  8238 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:54.973412  9049 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:54.980203  8941 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.030208  9048 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:55.036314  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:54.917834  9022 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.067137  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:55.139209  9041 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.679s	user 0.015s	sys 0.001s
I20260709 13:02:55.139456  8876 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.139732  9041 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:55.133283  8844 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.140753  8547 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:55.133271  8974 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:55.142134  8934 consensus_meta.cc:287] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.566s	user 0.005s	sys 0.007s
I20260709 13:02:55.111888  8831 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:55.142588  8934 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:55.143523  9045 consensus_meta.cc:287] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.683s	user 0.016s	sys 0.000s
I20260709 13:02:55.144024  9045 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:55.144732  8934 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.048316  9042 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.114784  9075 raft_consensus.cc:2764] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 13:02:55.066177  8579 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:55.041100  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:55.041987  8957 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:54.972779  9054 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.488s	user 0.001s	sys 0.005s
I20260709 13:02:55.060117  9062 raft_consensus.cc:740] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:54.972718  9060 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.469s	user 0.000s	sys 0.006s
I20260709 13:02:55.059381  1658 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:55.070870  9046 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:55.097033  9055 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 13:02:55.103803  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:55.181334  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 77609.5us
I20260709 13:02:55.150061  9076 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.039234  9025 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.122426  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:55.124133  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3503) took 1147 ms. Trace:
I20260709 13:02:55.057650  9067 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 13:02:55.083660  9069 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:02:55.125015  8855 consensus_meta.cc:287] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.970s	user 0.016s	sys 0.002s
I20260709 13:02:55.135062  1611 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:55.083940  9044 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.102180  8892 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.121381  8861 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:02:55.183898  9035 consensus_meta.cc:287] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.823s	user 0.020s	sys 0.000s
I20260709 13:02:55.031953  8960 raft_consensus.cc:740] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:55.109367  9077 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:02:55.154920  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 120698us
I20260709 13:02:55.097227  9013 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.133669  9049 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:55.162843  1615 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:55.163373  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:55.196826  8858 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:02:55.176347  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 117037us
I20260709 13:02:55.237330  8858 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.237898  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:55.136471  9048 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:55.240931  9048 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:55.028919  8773 consensus_meta.cc:287] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.856s	user 0.012s	sys 0.004s
I20260709 13:02:55.182088  2201 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:55.247712  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 65702.1us
I20260709 13:02:55.122962  9065 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.151629  8831 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:55.192878  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.197042  1685 rpcz_store.cc:276] 0709 13:02:53.976264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.977366 (+  1102us) service_pool.cc:224] Handling call
0709 13:02:53.977870 (+   504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:54.232970 (+255100us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.233003 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:54.242780 (+  9777us) log.cc:844] Serialized 26 byte log entry
0709 13:02:54.421371 (+178591us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:02:54.421455 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.421567 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.951642 (+530075us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b240025b098. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c8859c4 00007fe08c883acb 00007fe08ca3edc9 00007fe08ca3ea55 00007fe08ca3e09d 00007fe08f67010e 00007fe08f68788c 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221
0709 13:02:55.119701 (+168059us) raft_consensus.cc:1685] finished
0709 13:02:55.119724 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.124060 (+  4336us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40797,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":4458752,"thread_start_us":421971,"threads_started":2,"wal-append.queue_time_us":240951}
I20260709 13:02:55.198233  1584 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:02:55.182271  9038 consensus_meta.cc:287] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.582s	user 0.010s	sys 0.004s
I20260709 13:02:55.246714  8773 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.210888  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.294173  1534 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:55.178349  8844 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:55.080001  8625 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:55.183771  8889 consensus_meta.cc:287] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.687s	user 0.011s	sys 0.008s
W20260709 13:02:55.184084  9009 consensus_meta.cc:287] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.879s	user 0.012s	sys 0.002s
W20260709 13:02:55.180469  8985 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.602s	user 0.013s	sys 0.001s
W20260709 13:02:55.182966  9036 consensus_meta.cc:287] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.962s	user 0.004s	sys 0.011s
W20260709 13:02:55.187215  8707 consensus_meta.cc:287] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.775s	user 0.008s	sys 0.008s
W20260709 13:02:55.184393  8983 consensus_meta.cc:287] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.051s	user 0.013s	sys 0.007s
I20260709 13:02:55.224005  2236 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:55.225558  9008 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.233655  2228 raft_consensus.cc:2368] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 19. Current term is 20.
I20260709 13:02:55.235551  2232 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:55.337981  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:02:55.315377  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 93200.6us
W20260709 13:02:55.163920  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.254283  9035 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 13:02:55.340516  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:55.262545  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:55.341187  2236 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:55.273186  2235 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:55.209152  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.216097  8861 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.346604  1540 raft_consensus.cc:2393] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:55.291493  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.295336  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.347436  1537 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 24.
I20260709 13:02:55.237972  8858 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:02:55.291648  9055 log.cc:927] Time spent T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:02:55.255332  8855 raft_consensus.cc:2804] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:55.200260  9076 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.226229  8579 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.274471  8857 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.291862  8892 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:55.299815  9026 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:55.337500  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3502) took 1361 ms. Trace:
I20260709 13:02:55.341105  1541 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 25.
I20260709 13:02:55.317934  9041 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:55.219053  8974 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.203567  8947 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:55.353278  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:55.370402  2232 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:55.354848  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.350564  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:55.356052  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 101045us
I20260709 13:02:55.318614  9077 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:55.320176  8985 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:55.347496  1534 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
W20260709 13:02:55.369064  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 170869us
I20260709 13:02:55.349383  8861 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:55.309588  8889 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.377696  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.384486  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.385008  1685 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:02:55.378075  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.385943  1689 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:55.386521  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:55.387084  1616 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:55.387537  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.358855  8844 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:55.379516  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:55.373102  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:55.375927  1584 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:55.380632  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:55.298780  9038 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:55.382022  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.350229  9045 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:55.388520  1617 raft_consensus.cc:2410] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:02:55.383285  1538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3533) took 1334 ms. Trace:
I20260709 13:02:55.361654  1690 rpcz_store.cc:276] 0709 13:02:53.975921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:53.977256 (+  1335us) service_pool.cc:224] Handling call
0709 13:02:53.993740 (+ 16484us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:54.076257 (+ 82517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.076288 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:54.076899 (+   611us) log.cc:844] Serialized 52 byte log entry
0709 13:02:54.253459 (+176560us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:54.253500 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.271247 (+ 17747us) spinlock_profiling.cc:243] Waited 82.1 ms on lock 0x7b5400375854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:54.271327 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:55.309859 (+1038532us) raft_consensus.cc:1685] finished
0709 13:02:55.309883 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.337433 (+ 27550us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3497,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":7522,"spinlock_wait_cycles":188786944,"thread_start_us":176136,"threads_started":1,"wal-append.queue_time_us":227213}
I20260709 13:02:55.362368  8560 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.340922  8707 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.392002  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:55.393080  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:55.395082  1689 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:55.363940  8855 raft_consensus.cc:697] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:55.402333  1692 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 13:02:55.394466  1538 rpcz_store.cc:276] 0709 13:02:54.049001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.057233 (+  8232us) service_pool.cc:224] Handling call
0709 13:02:54.078032 (+ 20799us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:54.078490 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:54.078553 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:54.084311 (+  5758us) log.cc:844] Serialized 26 byte log entry
0709 13:02:54.233952 (+149641us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:02:54.284732 (+ 50780us) log.cc:844] Serialized 14 byte log entry
0709 13:02:54.284947 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:54.285014 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:54.760572 (+475558us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b1800400a80. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:02:55.375186 (+614614us) raft_consensus.cc:1685] finished
0709 13:02:55.383077 (+  7891us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:55.383217 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":43614,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":2598,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":9696,"spinlock_wait_cycles":7080192,"thread_start_us":106459,"threads_started":1,"wal-append.queue_time_us":210264}
I20260709 13:02:55.402966  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.417048  1688 raft_consensus.cc:2393] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:55.417207  1616 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:55.418300  1509 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:55.391448  8947 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:55.363834  8857 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.338939  9036 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.407811  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:55.383072  8934 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:55.446864   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:55.448187  2237 raft_consensus.cc:1240] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:55.404238  2232 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:55.389039  9077 raft_consensus.cc:697] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:55.392699  9067 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:55.421130  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.298007  9062 consensus_queue.cc:260] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.390094  9022 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:55.424535  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 48675.1us
I20260709 13:02:55.435817  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:55.430260  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.435710  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.469758  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:55.469863  1686 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:55.470245  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.470850  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.471115  1689 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
I20260709 13:02:55.471470  1686 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:55.438660  1583 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:55.435240  1541 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 3549) took 1204 ms (client timeout 1592 ms). Trace:
W20260709 13:02:55.472443  1541 rpcz_store.cc:269] 0709 13:02:54.230715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.254888 (+ 24173us) service_pool.cc:224] Handling call
0709 13:02:55.435075 (+1180187us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003444d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:55.435181 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2494901632}
I20260709 13:02:55.414990  1685 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:55.449839  9025 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:55.370687  9026 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.424801  9079 raft_consensus.cc:2764] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:02:55.406193  8983 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:55.452152  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:55.490031  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:55.462538  1690 raft_consensus.cc:3060] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:55.490726  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.411234  9042 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.417078  9041 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:02:55.478067  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.417960  9044 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:55.481302  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:55.466308  9082 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:55.432284  9013 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.492578  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:55.432906  9005 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.425567  9065 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:55.427330  9076 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:55.390103  9084 consensus_queue.cc:1175] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "984c9089d9804652846f3caeb609d56e" responder_term: 20 status { last_received { term: 20 index: 11 } last_committed_idx: 11 last_received_current_leader { term: 20 index: 11 } }
I20260709 13:02:55.372072  8831 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.491590  8983 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:55.499887  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:55.491392  1692 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:55.466389  9075 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:55.422245  9001 consensus_queue.cc:1059] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 13:02:55.420532  9087 consensus_queue.cc:1059] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:02:55.424714  9009 consensus_queue.cc:1059] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 13:02:55.479912  9035 raft_consensus.cc:2764] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260709 13:02:55.514422  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.514739  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:55.502432  9100 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:55.560314  9072 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:55.518733  9041 raft_consensus.cc:2764] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:02:55.527230  1690 raft_consensus.cc:2468] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 25.
I20260709 13:02:55.500382  8892 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:55.437505  8855 consensus_queue.cc:237] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.512868  9063 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:02:55.586366  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 116681us
I20260709 13:02:55.536959  9076 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:55.550072  9102 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 13:02:55.500255  9022 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:02:55.557521  9075 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:55.557863  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.594820  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.595393  1686 raft_consensus.cc:2410] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:02:55.558202  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.558887  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.494890  9088 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:02:55.550524  9098 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:02:55.574431  1535 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54758 (request call id 3532) took 1525 ms. Trace:
I20260709 13:02:55.599946  1535 rpcz_store.cc:276] 0709 13:02:54.048705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.054734 (+  6029us) service_pool.cc:224] Handling call
0709 13:02:55.574142 (+1519408us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b54001d79d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:55.574342 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2128145664}
I20260709 13:02:55.562747  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.531924  8983 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:02:55.536931  9095 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 13:02:55.517410  9082 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:55.504027  9081 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:55.585608  8625 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.618414  9026 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.589221  9044 raft_consensus.cc:3055] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:55.619174  9044 raft_consensus.cc:740] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:55.620023  9044 consensus_queue.cc:260] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:55.605036  9086 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:02:55.625829  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.631024  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.583384  1692 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
W20260709 13:02:55.588307  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 136918us
I20260709 13:02:55.584827  8934 raft_consensus.cc:697] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:55.590404  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:55.542057  8773 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.601141  1690 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:55.579387  8892 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:55.585459  9009 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:55.604069  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.605005  8788 tablet_bootstrap.cc:492] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 13:02:55.610077  9008 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.610123  8857 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:55.611064  9038 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.613788  8707 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:55.634114  1685 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:55.633718  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:55.643426  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:55.652654  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.654441  1610 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 25: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:55.652698  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:02:55.646131  8547 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.505s	user 0.011s	sys 0.000s
I20260709 13:02:55.647572  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.658322  1692 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:55.627960  9044 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:55.513206  9073 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.146s
I20260709 13:02:55.621811  9075 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:55.593864  9076 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.659963  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.661633  1690 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 24.
I20260709 13:02:55.661242  1612 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:55.648644  8889 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.635947  9048 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.652009  9008 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.669772  9048 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.671411  9008 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.671706  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.672387  1690 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:55.673262  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.674018  1690 raft_consensus.cc:2393] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 22: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:55.667585  1685 raft_consensus.cc:2376] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19. Re-sending same reply.
I20260709 13:02:55.675418  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.675969  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.676477  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.676761  1692 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 22: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:55.676857  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:55.677752  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 87406.2us
I20260709 13:02:55.678364  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.678499  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:55.684010  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:55.689482  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.689885  1507 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:55.691116  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.693578  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.661595  9038 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.695812  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.717197  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.718451  1685 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:55.694814  8957 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:55.720587  8957 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:55.721040  8957 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:55.721459  8957 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:55.725533  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.690791  9072 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:55.666455  9036 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:55.730464  9106 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:02:55.719676  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.731832  1685 raft_consensus.cc:2393] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:55.730733  9056 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:55.669382  9049 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.741022  9056 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:02:55.741479  9115 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:02:55.742197  1691 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:55.741930  9056 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.742563  8707 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.743022  9056 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:55.743180  8707 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.553736  9077 consensus_queue.cc:237] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:55.744657  9066 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:55.745054  9066 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:55.745543  9075 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:55.745934  9066 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:55.615373  9082 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:55.747078  9066 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:55.613461  9088 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.734817  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:55.690765  8773 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:55.720590  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.743803  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.716795  8934 consensus_queue.cc:237] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:55.732460  1538 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 24.
W20260709 13:02:55.753509  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.771242  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 92802.9us
I20260709 13:02:55.767755  1541 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:55.772322  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:55.731093  9009 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:55.773913  1691 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 25: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:55.776702  1535 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:02:55.777138  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.780591  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:55.752369  8238 consensus_meta.cc:287] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.618s	user 0.009s	sys 0.003s
I20260709 13:02:55.781316  1541 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:55.785597  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.788658  1691 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:55.767762  8857 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:55.784200  9113 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:02:55.794430  8857 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.790038  8238 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:55.801647  8857 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.802587  9049 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.803026  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:55.804884  8985 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.735541  9046 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:02:55.805819  8238 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:55.793277  8858 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.805588  9008 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:55.807673  2236 raft_consensus.cc:1240] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:55.809491  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.810575  1541 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 25: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:55.791584  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.811626  1691 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 16: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:55.809010  8985 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.815783  8858 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.816624  8985 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.816797  9008 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.819115  9088 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.819924  9008 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.822410  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.822938  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.827409  1691 raft_consensus.cc:2393] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:55.830919  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.831761  1691 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:55.832633  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.833308  1691 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 25: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:55.740679  9038 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.833884  9009 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.836475  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.805675  9112 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:02:55.828181  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.837414  1541 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:02:55.837965  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:55.830565  1584 connection.cc:256] RPC call timeout handler was delayed by -1.3331758339999737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
I20260709 13:02:55.838788  1683 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:55.839874  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.839881  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.840714  1683 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:55.791224  8707 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.841694  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.848748  1542 raft_consensus.cc:2393] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 24: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:55.842469  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.842952  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.843160  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.849687  1615 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:55.850373  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.851051  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:55.846907  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.851146  1613 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 25: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:55.843961  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.852083  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.852839  1614 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:55.855057  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:55.855875  1541 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:55.856341  1611 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:55.855259  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:55.857050  1618 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:55.858675  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:55.859812  1691 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:02:55.863400  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.865886  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:55.867648  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 193314us
I20260709 13:02:55.869247  1535 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:02:55.873714  1683 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:55.877256  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:55.878062  1539 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:55.878934  1691 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:02:55.873063  9038 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:55.881004  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:55.881579  1534 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:55.883834  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:55.884467  1542 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:55.847764  8983 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:55.896185  8983 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.896507  1584 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:55.897935  8547 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:55.898224  8957 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:55.900513  8957 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:55.906052  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:55.907013  1609 raft_consensus.cc:2410] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:02:55.906896  1582 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:55.908341  9112 raft_consensus.cc:2804] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:02:55.908385  2201 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:55.909026  9112 raft_consensus.cc:697] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:55.908736  8547 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:02:55.910336  8547 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:55.909993  9112 consensus_queue.cc:237] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:55.910724  8547 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:55.911188  2201 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:55.911612  8547 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:55.912271  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:55.913867  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:55.914634  1540 raft_consensus.cc:2393] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 22: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:55.914186  2201 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:55.900501  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:55.916016  2237 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:55.919560  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:02:55.924324  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 234486us
I20260709 13:02:55.925863  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:55.913024  8547 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:55.918049  8861 consensus_meta.cc:287] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.528s	user 0.007s	sys 0.003s
I20260709 13:02:55.929410  2237 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:02:55.935849  1539 raft_consensus.cc:1240] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:55.942816  1541 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:02:55.939929  9038 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:55.943233  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:55.961831  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.919667  9018 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.963s	user 0.010s	sys 0.042s
W20260709 13:02:55.946425  9114 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 13:02:55.941583  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:55.974201  1609 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:55.943001  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.943696  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:55.976080  1541 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:55.919065  8707 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:02:55.978694  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:55.980439  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 133839us
I20260709 13:02:55.980800  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:55.944234  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:55.945147  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.982646  1688 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 22: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:55.946162  8620 raft_consensus.cc:2764] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:02:55.984382  1540 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 25: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:55.946830  8855 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:55.986840  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.945411  2201 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:55.990644  1688 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:55.990828  1540 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 21. Current term is 22. Ops: []
W20260709 13:02:55.991120  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 45947.1us
I20260709 13:02:55.992185  2201 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:55.992884  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:55.994382  2237 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 16: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:55.941296  1508 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:55.969733  1542 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:55.973086  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:55.995579  2229 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
W20260709 13:02:55.970290  9018 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.052s	user 0.010s	sys 0.043s
I20260709 13:02:55.980746  1618 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:55.970721  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:55.997375  1542 raft_consensus.cc:1275] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 8. (term mismatch)
W20260709 13:02:55.997613  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.000290  1542 pending_rounds.cc:85] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Aborting all ops after (but not including) 7
I20260709 13:02:55.970329  8983 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:56.001188  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 59973.6us
I20260709 13:02:55.997748  1539 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:02:55.971333  9076 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:56.004860  1508 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:55.978334  9038 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:56.017565  1540 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:56.019045  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:56.019913  2237 raft_consensus.cc:2468] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:02:56.021833  1609 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:56.016574  1508 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:56.021142  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:56.014017  1534 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:02:56.023772  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:56.024194  1534 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:56.024575  2237 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:02:56.004566  9082 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:02:56.000590  1542 pending_rounds.cc:107] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 16.8
I20260709 13:02:55.965473  8861 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.027658  2229 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:56.032296  1688 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:55.998740  8855 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.036854  1612 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43398 (request call id 2538) took 1055 ms. Trace:
I20260709 13:02:56.037256  1612 rpcz_store.cc:276] 0709 13:02:54.981024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.981138 (+   114us) service_pool.cc:224] Handling call
0709 13:02:56.036669 (+1055531us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b5400389854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:56.036795 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1382132608}
I20260709 13:02:56.039371  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:56.040055  2231 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:56.031064  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:56.002591  9026 raft_consensus.cc:2804] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:56.042040  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:56.059425  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:56.096990  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.027205  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:56.031953  1542 raft_consensus.cc:2917] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Aborting config change with OpId 16.8: Aborted: Op aborted by new leader
I20260709 13:02:56.034525  2229 raft_consensus.cc:606] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:56.118692  2229 raft_consensus.cc:614] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:56.034708  1508 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:56.118788  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:56.004145  8858 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:56.030349  1539 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 26.
W20260709 13:02:56.121639  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.018239  8707 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:56.127036  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.120537  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:56.121374  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 79427.1us
I20260709 13:02:56.125089  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:56.025889  9088 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:56.134442  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:56.135713  2230 raft_consensus.cc:2368] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 24. Current term is 25.
I20260709 13:02:56.140077  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:56.142383  1691 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:56.150532  1542 raft_consensus.cc:2889] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:56.043301  8855 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:56.126152  8858 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.155460  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:56.164955  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 130345us
I20260709 13:02:56.124624  9048 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:56.178961  1508 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:56.162731  2229 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:56.122582  9076 raft_consensus.cc:2764] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
W20260709 13:02:56.131281  8947 consensus_meta.cc:287] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.686s	user 0.008s	sys 0.003s
I20260709 13:02:56.021600  8983 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.182183  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:56.141108  9088 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.186203  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:56.150233  9001 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.165653  8844 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:56.221009  1691 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:56.222720  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:56.223502  1691 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:02:56.164845  9026 raft_consensus.cc:697] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:56.182307  8858 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:56.175442  8876 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:02:56.225344  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:56.228782  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:56.233518  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:56.234084  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:56.234838  1684 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:56.186316  8789 raft_consensus.cc:2804] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:56.232870  8861 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:56.234736  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:56.287122  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 52486.7us
I20260709 13:02:56.242026  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:56.288744  1687 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:02:56.245316  9091 log.cc:927] Time spent T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.876s	user 0.003s	sys 0.000s
I20260709 13:02:56.252890  9036 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:56.256938  9058 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:56.224452  9048 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:02:56.292745  2233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:56.257783  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 1439) took 1290 ms. Trace:
I20260709 13:02:56.258206  9072 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:56.264359  2232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1441) took 1288 ms. Trace:
I20260709 13:02:56.153967  8707 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.266774  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:56.293941  2235 rpcz_store.cc:276] 0709 13:02:54.967716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.081532 (+113816us) service_pool.cc:224] Handling call
0709 13:02:56.257621 (+1176089us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54004bbed4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:56.257727 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435326336}
I20260709 13:02:56.297410  1610 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 18.
I20260709 13:02:56.267472  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:56.298846  1609 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:02:56.268122  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:56.300601  1618 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:56.268765  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:56.269771  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:56.270412  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:56.301730  1617 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:02:56.302309  1613 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:56.270573  9100 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:02:56.185541  8579 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.272039  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:56.317338  1615 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:56.276051  9108 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.277902  1657 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:56.185719  8525 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:56.190315  9088 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:02:56.323748  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 45934.3us
I20260709 13:02:56.324594  1657 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:56.287036  8789 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:56.287842  1658 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:02:56.315917  9044 consensus_meta.cc:287] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.644s	user 0.005s	sys 0.006s
I20260709 13:02:56.224753  9001 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.223488  9076 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:56.295918  2232 rpcz_store.cc:276] 0709 13:02:54.975814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.465832 (+490018us) service_pool.cc:224] Handling call
0709 13:02:55.494011 (+ 28179us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:55.537636 (+ 43625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:55.537683 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:55.538230 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 13:02:55.649381 (+111151us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:55.649409 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.263970 (+614561us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b54005361d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:56.264015 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.264072 (+    57us) raft_consensus.cc:1685] finished
0709 13:02:56.264085 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.264297 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41820,"raft.queue_time_us":9371,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":6548,"spinlock_wait_cycles":80300288,"thread_start_us":110860,"threads_started":1,"wal-append.queue_time_us":193140}
I20260709 13:02:56.292291  9035 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:56.292342  9036 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.354212  9036 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:56.355036  8892 consensus_meta.cc:287] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.702s	user 0.007s	sys 0.004s
I20260709 13:02:56.355544  8892 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.337621  2233 raft_consensus.cc:606] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:56.337355  8789 raft_consensus.cc:3060] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:56.357988  9127 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 13:02:56.357682  2233 raft_consensus.cc:614] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:56.314256  9072 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:56.359606  8707 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:56.312947  9100 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:56.360797  9100 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:56.357575  9026 consensus_queue.cc:237] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.238737  9038 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.324704  9108 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:56.322109  9058 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 18 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:56.327637  9018 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:56.229547  9124 consensus_queue.cc:1059] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 13:02:56.351547  9048 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:56.374176  9035 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:56.390527  9124 raft_consensus.cc:3055] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:56.403349  9058 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:56.407461  9108 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:56.401003  9124 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:56.440047  9100 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:02:56.446956  9056 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.703s	user 0.008s	sys 0.005s
I20260709 13:02:56.275084  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:56.636008  1536 raft_consensus.cc:3060] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:56.278628  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:56.638458  1535 raft_consensus.cc:3060] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:56.277168  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:56.647696  1609 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:56.648146  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:56.648298  1540 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:02:56.649274  1613 raft_consensus.cc:2410] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:02:56.276535  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:56.649858  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:56.651142  1540 raft_consensus.cc:2410] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
W20260709 13:02:56.651480  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.240164  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:50584 (request call id 1) took 10844 ms. Trace:
W20260709 13:02:56.504914  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 322569us
I20260709 13:02:56.506649  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:56.315310  1611 raft_consensus.cc:2393] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 20: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:02:56.510104  1543 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 1537) took 2338 ms (client timeout 1502 ms). Trace:
W20260709 13:02:56.677947  1543 rpcz_store.cc:269] 0709 13:02:54.171256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.171432 (+   176us) service_pool.cc:224] Handling call
0709 13:02:56.509507 (+2338075us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b54000ae454. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:56.509798 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1927802752}
I20260709 13:02:56.680507  9062 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:56.681048  9066 consensus_meta.cc:287] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.933s	user 0.009s	sys 0.004s
I20260709 13:02:56.528314  1683 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:56.318408  1684 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:02:56.530037  1691 raft_consensus.cc:1240] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 21. Current term is 22. Ops: []
W20260709 13:02:56.530838  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.720343  1692 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:02:56.720996  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:56.721520  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.536160  1509 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:56.722218  1684 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
W20260709 13:02:56.722688  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 186923us
I20260709 13:02:56.723677  1509 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:56.724840  8994 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:56.539944  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:56.542639  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:56.726120  1689 raft_consensus.cc:2410] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 13:02:56.726367  8994 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:02:56.548763  2228 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:49846 (request call id 1685) took 2370 ms (client timeout 2145 ms). Trace:
I20260709 13:02:56.726620  9046 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:56.726923  2228 rpcz_store.cc:269] 0709 13:02:54.177764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.177890 (+   126us) service_pool.cc:224] Handling call
0709 13:02:56.547977 (+2370087us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54006586d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:56.548372 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2425671936}
I20260709 13:02:56.551229  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39452 (request call id 1766) took 1514 ms. Trace:
I20260709 13:02:56.373634  8525 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:56.554630  2234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 1431) took 1599 ms. Trace:
I20260709 13:02:56.728919  2234 rpcz_store.cc:276] 0709 13:02:54.954920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:54.956635 (+  1715us) service_pool.cc:224] Handling call
0709 13:02:56.554058 (+1597423us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b54007102d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:56.554356 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66541824}
I20260709 13:02:56.728206  2231 rpcz_store.cc:276] 0709 13:02:55.036950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.124030 (+1087080us) service_pool.cc:224] Handling call
0709 13:02:56.124434 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.124642 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 13:02:56.293344 (+168702us) log.cc:844] Serialized 14 byte log entry
0709 13:02:56.550131 (+256787us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:56.550267 (+   136us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:02:56.550468 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.550633 (+   165us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.550913 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":247690,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":1930,"raft.queue_time_us":41307,"raft.run_cpu_time_us":394,"raft.run_wall_time_us":394,"thread_start_us":424380,"threads_started":2,"wal-append.queue_time_us":156186}
I20260709 13:02:56.555850  1687 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 25.
I20260709 13:02:56.556823  1690 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:56.559502  8025 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.561565  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1767) took 1459 ms. Trace:
W20260709 13:02:56.562731  8547 consensus_meta.cc:287] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.611s	user 0.010s	sys 0.008s
I20260709 13:02:56.325179  8579 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.725890  8974 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:56.734603  8974 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.732929  8025 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.735054  8974 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:56.569460  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1440) took 1596 ms. Trace:
I20260709 13:02:56.589835  8876 raft_consensus.cc:697] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:56.735553  2236 rpcz_store.cc:276] 0709 13:02:54.972705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.349536 (+376831us) service_pool.cc:224] Handling call
0709 13:02:55.349872 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:55.808150 (+458278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.568797 (+760647us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54003d3bd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:02:56.569201 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1052267264}
I20260709 13:02:56.573697  8707 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:56.736711  8876 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.447440  9124 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:56.731626  8994 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:56.594651  8892 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:56.461947  9129 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.229s	user 0.000s	sys 0.002s
I20260709 13:02:56.640252  8858 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:56.460340  9131 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.235s	user 0.000s	sys 0.003s
I20260709 13:02:56.592797  9058 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:56.584787  2233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:56.399628  9048 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.742089  9048 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:56.638137  9001 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:56.274551  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:56.274758  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:56.658490  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.658550  8975 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:56.749980  8975 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:56.751768  8975 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:56.659907  9040 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.655s	user 0.004s	sys 0.002s
I20260709 13:02:56.665609  9035 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:56.667382  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:56.672039  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:56.672583  9126 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:56.521571  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:56.682134  1688 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:02:56.682260  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:56.683495  1616 raft_consensus.cc:1275] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:02:56.686769  9138 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.689244  8857 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.693818  1542 rpcz_store.cc:276] 0709 13:02:45.395557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:45.403609 (+  8052us) service_pool.cc:224] Handling call
0709 13:02:45.540038 (+136429us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b2c00056c38. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f4a0a 00007fe08e3f4693 00007fe08e3f4631 00007fe08fb273f1 00007fe08fa609fe 00007fe08fae7749 00007fe08fae876a 00007fe08fae791b 00007fe08faf303c 00007fe08faf26b3
0709 13:02:45.890037 (+349999us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b4c00016f80. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08ddb8a20 00007fe08ddc9a29 00007fe08dd8f644 00007fe08ddb6f2d 00007fe08ddb6b93 00007fe08fa451af 00007fe08fa44f3e 00007fe08faca888 00007fe08fae775c 00007fe08fae876a 00007fe08fae791b
0709 13:02:45.944545 (+ 54508us) log.cc:844] Serialized 506 byte log entry
0709 13:02:46.240975 (+296430us) spinlock_profiling.cc:243] Waited 7.26 ms on lock 0x7b540038e620. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa6cb26 00007fe08fa6649a 00007fe08fa65714 00007fe08fa45eaa 00007fe08fa4cd53 00007fe08fa4ccd9 00007fe08fa4cc69 00007fe08fa4cc31 00007fe08fa4bdfd 000000000038f354
0709 13:02:46.249139 (+  8164us) spinlock_profiling.cc:243] Waited 6.53 ms on lock 0x7b18003a8ca0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9ff3f3 00007fe08ca017c1 00007fe08ca01779 00007fe08ca01709 00007fe08ca016d1 00007fe08ca009cd 000000000038f354 000000000038f199 00007fe08c9e283a
0709 13:02:46.250912 (+  1773us) spinlock_profiling.cc:243] Waited 9.59 ms on lock 0x7b540038e620. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa6cb26 00007fe08fa6649a 00007fe08fa65714 00007fe08fa45eaa 00007fe08fa4cd53 00007fe08fa4ccd9 00007fe08fa4cc69 00007fe08fa4cc31 00007fe08fa4bdfd 000000000038f354
0709 13:02:56.240075 (+9989163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74944,"raft.queue_time_us":117995,"raft.run_cpu_time_us":15261,"raft.run_wall_time_us":195370,"spinlock_wait_cycles":760797184,"thread_start_us":58888,"threads_started":1,"wal-append.queue_time_us":139500,"wal-append.run_cpu_time_us":3387,"wal-append.run_wall_time_us":1132290}
I20260709 13:02:56.696497  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2482) took 1081 ms. Trace:
W20260709 13:02:56.701789  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.680747  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:56.704983  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 380290us
I20260709 13:02:56.707485  8960 consensus_queue.cc:260] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:56.545756  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:56.753737  9040 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.799s	user 0.004s	sys 0.002s
I20260709 13:02:56.755867  9126 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.753381  8975 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:56.758733  8857 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.757879  8957 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:56.760087  1686 rpcz_store.cc:276] 0709 13:02:55.614838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.630721 (+ 15883us) service_pool.cc:224] Handling call
0709 13:02:56.695835 (+1065114us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b540014a854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:56.696127 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2206211968}
I20260709 13:02:56.765414  9001 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:56.727561  9046 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.733618  2230 rpcz_store.cc:276] 0709 13:02:55.101823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.122093 (+ 20270us) service_pool.cc:224] Handling call
0709 13:02:56.560907 (+1438814us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b54005c0dd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:56.561293 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2158721408}
I20260709 13:02:56.737012  8707 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.596264  8983 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:56.758215  9138 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.739429  9124 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:56.754470  1609 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
W20260709 13:02:56.460594  9130 log.cc:927] Time spent T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.194s	user 0.000s	sys 0.002s
I20260709 13:02:56.765611  8960 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:02:56.456583  9132 log.cc:927] Time spent T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: Append to log took a long time: real 0.102s	user 0.002s	sys 0.000s
I20260709 13:02:56.742110  2233 raft_consensus.cc:606] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:02:56.741817  9058 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.777182  2233 raft_consensus.cc:614] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:56.772744  9001 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.778579  9001 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:56.770620  8957 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:56.765110  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 224847us
I20260709 13:02:56.780519  8957 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:56.781414  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:56.783347  2203 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:56.760594  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:56.784976  9040 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:56.785491  9040 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.785895  9040 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:56.788584  2233 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "474d3b57c33542aba459b97733122bf0"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "83f6f5810950417195f707ff308a8125"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:56.789398  2233 raft_consensus.cc:606] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Received request to transfer leadership to TS 83f6f5810950417195f707ff308a8125
I20260709 13:02:56.789919  2233 raft_consensus.cc:614] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:56.796294  2230 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eed65fd1add844e7821095e2215fb393"
dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
mode: GRACEFUL
new_leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
 from {username='slave'} at 127.0.0.1:40336
I20260709 13:02:56.797289  2230 raft_consensus.cc:606] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Received request to transfer leadership to TS 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:56.797827  2230 raft_consensus.cc:614] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:02:56.800221   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:02:56.652197  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:56.818197  1540 raft_consensus.cc:3060] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:02:56.862306  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 79101.5us
I20260709 13:02:56.872423  8025 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:56.897826  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 135187us
W20260709 13:02:56.905762  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68072us
I20260709 13:02:56.922361  9126 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:56.923657  8579 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:56.926227  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:56.928385  1657 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:56.930480  1687 raft_consensus.cc:1240] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:56.930742  9058 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:56.933528  9075 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:02:56.933754  8525 raft_consensus.cc:697] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:56.935235  8525 consensus_queue.cc:237] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.936729  1691 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:56.937779  9138 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:56.943631  1541 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3559) took 1006 ms. Trace:
W20260709 13:02:56.944943  8855 consensus_meta.cc:287] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.761s	user 0.009s	sys 0.013s
I20260709 13:02:56.947679  8857 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:56.933182  9081 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:56.949044  9081 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 22 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:56.950847   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:02:56.950502  9081 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.956882  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:56.957777  1542 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54758 (request call id 3563) took 1019 ms. Trace:
I20260709 13:02:56.944175  1541 rpcz_store.cc:276] 0709 13:02:55.937514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.976915 (+ 39401us) service_pool.cc:224] Handling call
0709 13:02:56.036096 (+ 59181us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:02:56.125328 (+ 89232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:02:56.125359 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:02:56.125958 (+   599us) log.cc:844] Serialized 52 byte log entry
0709 13:02:56.283574 (+157616us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b18004515a0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c0d1 00007fe08faa7a0f
0709 13:02:56.644882 (+361308us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:02:56.647419 (+  2537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.647559 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:56.746672 (+ 99113us) raft_consensus.cc:1685] finished
0709 13:02:56.746701 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:56.943251 (+196550us) spinlock_profiling.cc:243] Waited 58.7 ms on lock 0x7b54000adf50. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:56.943390 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8365,"raft.queue_time_us":113366,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":13832,"spinlock_wait_cycles":138400256,"thread_start_us":341940,"threads_started":1,"wal-append.queue_time_us":550019}
W20260709 13:02:56.958195  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 59673.9us
W20260709 13:02:56.958611  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 276435us
I20260709 13:02:56.959906  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:56.961639  9150 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:56.962371  9150 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:56.962749  9150 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:56.962975  9046 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:56.963689  9046 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:56.964076  9046 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:56.964018  9058 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:02:56.964632  9046 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:56.958227  1542 rpcz_store.cc:276] 0709 13:02:55.938382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.996543 (+ 58161us) service_pool.cc:224] Handling call
0709 13:02:55.996921 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.897652 (+900731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:56.957707 (+ 60055us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:02:56.982717  1657 leader_election.cc:400] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 24. Current term is 25.
I20260709 13:02:56.983193  1657 leader_election.cc:403] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:56.985436  9018 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:56.986140  9018 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:56.987284  8789 consensus_meta.cc:287] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.587s	user 0.007s	sys 0.010s
I20260709 13:02:56.987924  8789 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:56.988854  9072 consensus_meta.cc:287] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.629s	user 0.015s	sys 0.010s
I20260709 13:02:56.989637  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:56.990581  1543 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 26: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:56.986552  8707 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:56.891139  8858 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:56.999804  1535 raft_consensus.cc:2468] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:02:57.000984  1536 raft_consensus.cc:2468] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:57.004423  8832 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.009980  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:56.986590  8859 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 24. Current term is 25.
I20260709 13:02:56.990521  8789 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:57.012171  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.022044  9138 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:57.025926  9100 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.019s
I20260709 13:02:56.989351  9072 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:56.994920  8855 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.039618  1540 raft_consensus.cc:2468] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 25.
W20260709 13:02:57.041229  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 369999us
I20260709 13:02:57.050873  9138 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.062250  8832 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.064335  9072 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:57.064723  8832 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:57.067176  8855 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:57.069175  9026 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:57.059302  9138 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:57.069875  9026 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.072252  9026 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:57.078389  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 68528.4us
I20260709 13:02:57.079420  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:57.080827  8983 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:57.081441  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:57.083281  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:57.090816  1686 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 18 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:57.092060  2237 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 1444) took 1074 ms. Trace:
I20260709 13:02:57.092455  2237 rpcz_store.cc:276] 0709 13:02:56.017118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.026976 (+  9858us) service_pool.cc:224] Handling call
0709 13:02:57.091741 (+1064765us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b54002f49d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:57.091892 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2080545024}
I20260709 13:02:57.032368  9001 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.129693  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:57.137142  9048 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.138291  9001 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:57.139672  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 55750us
I20260709 13:02:57.140594  1584 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:57.142246  9048 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:57.142899  8832 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:57.143246  8789 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.143965  8789 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:57.142943  9046 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.148363  9046 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:57.142516  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet 7ba0620abe844f5bb71ba9dd947c3b9c: Aborted: Op aborted by new leader
I20260709 13:02:57.152391  1689 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:57.160365  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:57.167135  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.180748  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:57.175072  1538 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
W20260709 13:02:57.183480  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.181856  1543 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:02:57.186385  1536 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
W20260709 13:02:57.186714  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.187193  1540 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:57.153451  9001 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:57.194288  1535 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 21. Current term is 22. Ops: []
I20260709 13:02:57.196321  1535 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: []
I20260709 13:02:57.196981  8789 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:57.198079  1534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:57.198994  1534 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:57.199236  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:57.200578  1687 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 22: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:57.207274  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2486) took 1591 ms. Trace:
I20260709 13:02:57.208009  1692 rpcz_store.cc:276] 0709 13:02:55.616027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:55.720297 (+104270us) service_pool.cc:224] Handling call
0709 13:02:57.206835 (+1486538us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b540028e6d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:57.207150 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2188196224}
I20260709 13:02:57.210299  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 1662) took 1210 ms. Trace:
I20260709 13:02:57.210886  1688 rpcz_store.cc:276] 0709 13:02:55.999976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.120202 (+120226us) service_pool.cc:224] Handling call
0709 13:02:57.210018 (+1089816us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b54002ccc54. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:57.210213 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1261642112}
W20260709 13:02:57.213343  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.220000  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:57.221621  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:57.222302  2230 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:02:57.225356  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:57.226073  2237 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:57.227087  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:57.229935  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:57.231791  2230 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 26.
W20260709 13:02:57.235174  9036 consensus_meta.cc:287] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.880s	user 0.010s	sys 0.013s
I20260709 13:02:57.235733  9036 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:57.236173  9018 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:57.184895  9126 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:57.191087  9048 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.240593  9048 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:57.239792  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:57.242791  2231 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 13:02:57.244982  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.245741  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.249550  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:57.242877  9048 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:57.254395  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:57.255584  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:57.257300  1692 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:57.259524  2230 raft_consensus.cc:2368] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 19. Current term is 20.
I20260709 13:02:57.262950  1508 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:57.263877  2234 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:57.266254  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:57.268496  1507 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:57.269779  8983 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:57.270310  8983 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.270720  8983 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:57.271749  9150 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.273562  2233 raft_consensus.cc:2393] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:57.273880  9040 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.274327  9046 consensus_queue.cc:1059] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:02:57.275748  9046 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:57.276288  9040 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:57.276518  9046 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:57.239846  9126 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:57.279268  8960 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.502s	user 0.012s	sys 0.004s
W20260709 13:02:57.286826  8974 consensus_meta.cc:287] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.551s	user 0.018s	sys 0.005s
W20260709 13:02:57.297250  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.298820  1540 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:57.299782  1542 raft_consensus.cc:1240] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:57.303414  9038 raft_consensus.cc:3055] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:57.303929  9038 raft_consensus.cc:740] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:57.305066  9038 consensus_queue.cc:260] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:57.306241  9088 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.966s	user 0.019s	sys 0.005s
I20260709 13:02:57.306804  9088 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.309047  9088 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:57.289036  9126 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:57.312278  8974 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.315971  1539 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:57.317987  8974 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:57.321666  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:57.322731  1613 raft_consensus.cc:2393] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 26: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:57.326020  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:57.327793  9036 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:57.184263  9124 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 13:02:57.258322  9001 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.317250  1539 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33722 (request call id 1551) took 1044 ms. Trace:
I20260709 13:02:57.261161  8957 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.336627  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:57.332767  1539 rpcz_store.cc:276] 0709 13:02:56.272869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.274557 (+  1688us) service_pool.cc:224] Handling call
0709 13:02:57.317009 (+1042452us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b5400343fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:57.317183 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1302023424}
I20260709 13:02:57.338956  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:57.340261  2233 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:57.341925  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:57.306747  9038 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:57.342016  1612 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:57.342846  1613 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
W20260709 13:02:57.346349  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 88819.1us
I20260709 13:02:57.350339  9124 consensus_queue.cc:1243] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 17.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:57.350353  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:57.351467  1609 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:02:57.353029  1537 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 23. Current term is 24. Ops: []
I20260709 13:02:57.354962  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:57.359087  2236 raft_consensus.cc:2410] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:02:57.359710  8957 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:57.353152  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:57.362833  1507 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:57.349258  8960 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.330335  9138 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.355098  9150 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:57.376649  9100 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:57.377390  9100 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.377908  9100 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:57.378098  8547 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:57.379546  8579 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:02:57.379957  8579 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:57.381008  8579 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:57.382261  8579 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:57.390228  8957 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:02:57.396160  1507 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:57.396353  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:57.398612  1613 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:57.397715  2201 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:57.403272  9138 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:57.406512  2236 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:02:57.408486  1612 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:57.393551  8960 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:57.421228  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 48697.7us
I20260709 13:02:57.422386  2203 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:57.424154  8858 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:57.424393  9040 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:57.406849  9046 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:57.425405  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:57.407691  9150 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:57.432286  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:57.432986  1613 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
W20260709 13:02:57.434921  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 45734.2us
I20260709 13:02:57.411834  9126 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.430415  9001 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:57.431828  9040 raft_consensus.cc:697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:57.447816  8579 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.485817  1507 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:57.492050  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:57.495291  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:57.488243  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 57956.9us
I20260709 13:02:57.500065  8579 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:57.494211  9048 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Stepping down as leader of term 16
W20260709 13:02:57.510740  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 78391.2us
I20260709 13:02:57.526448  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:57.558525  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:57.552425  2201 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:57.570271  1609 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:57.516405  9048 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:57.529834  8974 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:02:57.576680  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.577858  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.577857  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3521) took 1293 ms. Trace:
I20260709 13:02:57.570349  8788 tablet_bootstrap.cc:492] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=1 applied=7 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:02:57.585994  1684 rpcz_store.cc:276] 0709 13:02:56.284949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.723100 (+438151us) service_pool.cc:224] Handling call
0709 13:02:56.723644 (+   544us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.724070 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:02:56.974193 (+250123us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.364487 (+390294us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.577340 (+212853us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b18004c0900. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c979c0a 00007fe0960b9219 00007fe08ca01a40 00007fe08c9fc4e4 00007fe08c9fc455 00007fe08fae4060 00007fe08faf6716 00007fe08fb07287 00007fe08fb07208
0709 13:02:57.577509 (+   169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.577574 (+    65us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:02:57.577616 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.577680 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.577802 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":188655,"prepare.queue_time_us":465069,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":217320,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":21239,"spinlock_wait_cycles":9878912,"thread_start_us":599903,"threads_started":2}
W20260709 13:02:57.597654  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 90170.1us
I20260709 13:02:57.607180  1509 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:02:57.619686  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:57.583873  8525 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.014s	sys 0.054s
I20260709 13:02:57.600485  9072 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:57.637156  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 161651us
I20260709 13:02:57.640228  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3522) took 1350 ms. Trace:
I20260709 13:02:57.627522  8994 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:57.664024  9145 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:02:57.638900  8525 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.014s	sys 0.055s
I20260709 13:02:57.640741  1685 rpcz_store.cc:276] 0709 13:02:56.289835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.726386 (+436551us) service_pool.cc:224] Handling call
0709 13:02:56.884176 (+157790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:56.945957 (+ 61781us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 13:02:57.174419 (+228462us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.639823 (+465404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:02:57.639893 (+    70us) raft_consensus.cc:1645] Marking committed up to 17
0709 13:02:57.639948 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.640019 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:57.640155 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29690,"prepare.queue_time_us":280780,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"thread_start_us":554674,"threads_started":2}
I20260709 13:02:57.600697  9088 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:02:57.675226  9026 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:57.675580  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:57.676534  1613 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:57.637104  8855 raft_consensus.cc:2804] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:02:57.610471  9046 consensus_queue.cc:1059] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 13:02:57.681084  8876 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:57.684974  9073 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:02:57.685753  9164 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
W20260709 13:02:57.688599  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.690079  9048 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:57.695667  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:57.692256  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 110489us
I20260709 13:02:57.650389  8960 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:57.658886  8994 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.656242  9018 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.713910  1692 raft_consensus.cc:2410] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
W20260709 13:02:57.707312  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:57.716840  1688 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:57.721261  1609 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:57.724398  8788 tablet_bootstrap.cc:492] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 13:02:57.728657  8788 ts_tablet_manager.cc:1403] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 2.150s	user 0.050s	sys 0.015s
I20260709 13:02:57.724761  9073 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:57.724993  8876 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:57.724809  8620 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.608032  8983 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.727285  2201 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:57.729502  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 588972us
W20260709 13:02:57.734393  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 78441.4us
I20260709 13:02:57.737052  1615 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:02:57.736825  9018 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:57.734982  8579 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:57.692873  9126 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:57.766083  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:57.768568  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:57.666487  9072 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.769388  1584 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:57.751787  2233 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:57.706422  8855 raft_consensus.cc:697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 25 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:57.705763  9026 consensus_queue.cc:1059] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:02:57.637673  9040 consensus_queue.cc:237] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.718428  8994 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:57.771198  8832 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:57.786487  8957 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:57.786576  9001 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:57.787550  9001 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:57.788116  9001 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:57.787249  8957 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.735277  8960 consensus_queue.cc:1059] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
W20260709 13:02:57.791591  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 72804.4us
I20260709 13:02:57.784806  8832 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.800761  2203 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:57.801259  2201 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:57.780197  8620 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.791663  9088 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:57.794970  8579 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:02:57.809285  9138 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:57.813388  9026 raft_consensus.cc:2804] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:57.814049  9026 raft_consensus.cc:697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:57.814829  9145 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:57.815377  9145 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:57.815117  9026 consensus_queue.cc:237] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.802814  8832 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:57.830793  1613 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:02:57.830556  1509 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:57.843618  8975 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.844278  8975 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:57.847311  8974 raft_consensus.cc:3055] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:57.847762  8974 raft_consensus.cc:740] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:57.848215  8788 raft_consensus.cc:359] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e [term 17 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:57.848649  8974 consensus_queue.cc:260] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.849710  8788 raft_consensus.cc:740] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 13:02:57.850713  8547 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.850445  8788 consensus_queue.cc:260] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:02:57.851274  8547 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:57.852653  8788 ts_tablet_manager.cc:1434] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.124s	user 0.007s	sys 0.000s
I20260709 13:02:57.806944  9100 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:57.808673  8960 raft_consensus.cc:3055] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:57.811427  9138 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:57.869158  1509 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:02:57.864846  8579 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:57.812449  9073 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:57.870529  1507 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:57.867090  8547 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:57.867554  9046 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:02:57.874248  9138 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:57.877926  8789 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:57.866478  8960 raft_consensus.cc:740] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:57.874735  9126 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:02:57.909801  8858 raft_consensus.cc:3055] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:02:57.914918  9088 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:57.907799  9170 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:57.859766  8855 consensus_queue.cc:237] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:57.917480  8974 raft_consensus.cc:3060] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:57.941574  8620 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:57.941668  8579 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:57.942582  9166 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
W20260709 13:02:57.945747  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 82701.5us
W20260709 13:02:57.940781  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 50587.6us
W20260709 13:02:57.939257  1508 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.0.107.196:39293 timed out after 2.039s (SENT) [suppressed 34 similar messages]
W20260709 13:02:57.972496  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 182910us
I20260709 13:02:57.961057  9100 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:57.937793  8858 raft_consensus.cc:740] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:57.977555  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:57.978760  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:57.972187  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 167873us
I20260709 13:02:57.991732   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:02:57.972222  8876 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:57.997632  9038 consensus_meta.cc:287] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.649s	user 0.012s	sys 0.002s
I20260709 13:02:57.981791  9018 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:57.969928  9170 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.011459  8975 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:58.021864  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:58.021013  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:58.018357  9046 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:58.038450  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3520) took 1760 ms. Trace:
W20260709 13:02:58.022243  9177 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.226s	user 0.001s	sys 0.001s
I20260709 13:02:58.040176  1683 rpcz_store.cc:276] 0709 13:02:56.269108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:56.722556 (+453448us) service_pool.cc:224] Handling call
0709 13:02:56.723151 (+   595us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:02:56.937866 (+214715us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:02:57.448099 (+510233us) log.cc:844] Serialized 14 byte log entry
0709 13:02:57.762425 (+314326us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:02:57.802125 (+ 39700us) log.cc:844] Serialized 26 byte log entry
0709 13:02:57.802632 (+   507us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:02:57.802755 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:57.802898 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:02:58.029747 (+226849us) raft_consensus.cc:1685] finished
0709 13:02:58.029770 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:02:58.029908 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":213858,"prepare.queue_time_us":288591,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":216081,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":2193,"thread_start_us":757742,"threads_started":2,"wal-append.queue_time_us":379089}
I20260709 13:02:58.044186  9046 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.059376  9170 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.060137  9183 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:58.060604  9183 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.060974  9183 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:58.062839  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:58.066481  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:58.066876  9185 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:58.067407  9185 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:58.067771  9185 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:58.083061  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:58.083613  1692 raft_consensus.cc:1275] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 13:02:58.085407  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 69936.2us
I20260709 13:02:58.084827  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:58.021221  8789 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 21 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:58.094813  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:58.013756  9100 consensus_queue.cc:1048] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:02:58.023893  8957 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.091014  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:58.099192  1692 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:58.091207  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:58.092211  1537 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:02:58.092504  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.092706  9186 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.092868  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:58.093251  1691 raft_consensus.cc:3060] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:58.093307  1541 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:58.094283  2230 raft_consensus.cc:1240] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 20. Current term is 21. Ops: []
I20260709 13:02:58.091218  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:58.090042  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.104063  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.107378  2228 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:58.030319  8547 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:02:58.113636  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 174394us
W20260709 13:02:58.113543  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.110018  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:58.111270  2234 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:58.082561  8975 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.118860  2232 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:02:58.120839  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 106533us
I20260709 13:02:58.113786  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.125366  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:58.126085  2237 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:58.126989  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:58.127331  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.128971  2235 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:58.129086  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.031543  8858 consensus_queue.cc:260] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.130658  8975 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:58.132200  8994 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.133039  8975 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:58.120967  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:58.132820  1507 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:58.136271  1539 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:58.134960  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:58.142735  1692 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:58.118788  9150 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:58.124567  8525 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.150022  1542 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:02:58.143858  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:58.107890  8832 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.176211  9038 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.176908  9038 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:58.178166  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.143934  9036 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.103287  8960 consensus_queue.cc:260] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.185117  9038 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:58.185779  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:58.187088  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:58.181991  9126 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.188089  9126 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:58.188443  1553 tablet_copy_service.cc:342] P 83f6f5810950417195f707ff308a8125: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-7ba0620abe844f5bb71ba9dd947c3b9c received from {username='slave'} at 127.0.0.1:33722
I20260709 13:02:58.189042  1536 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:58.189956  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.189109  1553 tablet_copy_service.cc:434] P 83f6f5810950417195f707ff308a8125: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-7ba0620abe844f5bb71ba9dd947c3b9c on tablet 7ba0620abe844f5bb71ba9dd947c3b9c with peer cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:58.190333  9126 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:58.182283  8957 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:58.191519  8957 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:58.192404  8957 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:58.185103  8960 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:58.199644  1536 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 24.
I20260709 13:02:58.186745  8789 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.201357  2232 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:02:58.201985  9100 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:02:58.202818  9100 consensus_queue.cc:1243] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 22.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 13:02:58.203367  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 70697.1us
I20260709 13:02:58.179747  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:58.206238  1539 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 17: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:58.138792  8547 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:58.163862  9150 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:58.219019  2228 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
W20260709 13:02:58.209311  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 70493.6us
I20260709 13:02:58.158371  9186 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:58.205164  9018 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.243153  1536 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:58.243630  1543 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:02:58.243340  9018 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:58.225031  1583 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:58.244916  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 100854us
I20260709 13:02:58.245649  9018 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:58.250762  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:58.251977  9038 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:02:58.252511  9038 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:02:58.253013  9038 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:58.256561  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.258201  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:58.258436  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.259373  1692 raft_consensus.cc:2393] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 25: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:58.259473  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.260368  1690 raft_consensus.cc:2410] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:02:58.260219  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:58.222982  9040 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.270361  9018 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:58.274340  2229 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:02:58.274629  9040 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.274735  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.274824  9018 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:58.276647  1687 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:02:58.276770  8547 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:58.272919  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:58.276961  9018 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:58.278206  9184 consensus_queue.cc:1048] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 13:02:58.276961  9040 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:58.283768  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.285341  1581 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:58.285799  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.288975  1583 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:58.301100  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:58.303053  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 138891us
I20260709 13:02:58.309304  2201 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:58.310416  9040 raft_consensus.cc:2804] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:58.311223  9040 raft_consensus.cc:697] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:58.312515  1688 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:58.312599  1691 raft_consensus.cc:2468] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 17.
I20260709 13:02:58.313199  1508 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:58.244963  9145 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:58.212831  8525 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.224416  9036 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.319934  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:58.320111  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:58.297873  9081 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:58.302605  9126 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.327867  2201 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:58.336046  1684 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 19.
I20260709 13:02:58.318550  8547 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:58.334057  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:58.339175  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.345767  1691 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 16.
W20260709 13:02:58.350670  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 93254.6us
I20260709 13:02:58.345165  2229 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
W20260709 13:02:58.354072  1582 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.360527  8975 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.349553  9190 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:58.326363  9081 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.364714  2201 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:58.383263  2203 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:02:58.368065  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 54945.8us
I20260709 13:02:58.406739  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.428133  1687 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 21: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:58.426788  2237 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 24.
W20260709 13:02:58.429242  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 140359us
I20260709 13:02:58.432443  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:58.368038  9194 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:02:58.445642  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:58.446424  1536 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:58.442013  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.444617  1508 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:58.449723  1691 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:02:58.390551  9048 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:58.365325  8789 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:58.365768  9201 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.367784  9196 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:02:58.387804  9036 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.480029  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.408810  9200 raft_consensus.cc:2804] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:02:58.455475  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.456975  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.384761  8525 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.367775  8994 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.361372  9126 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.497102  1538 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:58.460743  1683 raft_consensus.cc:1275] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:02:58.453258  1539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.450296  1691 pending_rounds.cc:85] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 8
I20260709 13:02:58.465149  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.465507  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:58.499763  1610 raft_consensus.cc:3060] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:58.468115  1541 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:02:58.470445  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:58.422132  9208 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.534332  9208 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.537632  9213 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:58.537703  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:58.422866  9195 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.539907  9195 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.500937  9201 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.542346  9195 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.542891  9195 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.543658  9201 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:58.398865  9100 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:58.544245  9201 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:02:58.471859  8832 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.544608  9100 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:02:58.464733  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:58.545956  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:58.546221  1583 leader_election.cc:400] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 19. Current term is 20.
I20260709 13:02:58.474406  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:58.546635  1583 leader_election.cc:403] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:02:58.477633  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:58.477814  9191 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:58.474272  1508 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:58.478750  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:58.454260  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:58.547999  1613 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:02:58.548354  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:58.548466  1583 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:58.548804  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:58.455089  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.369634  9190 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.549458  1613 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:58.549652  1690 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:02:58.550231  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.550464  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:58.551208  9190 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.551396  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.551616  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:58.485464  1535 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:58.552042  1686 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:58.502962  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.552942  1688 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:58.469254  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:58.366223  8975 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:58.545712  9174 log.cc:927] Time spent T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:02:58.546317  1612 raft_consensus.cc:2393] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:58.476810  1685 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:02:58.474080  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.546936  9100 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:58.548964  1617 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:58.552073  9186 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:58.330309  9040 consensus_queue.cc:237] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.381898  9183 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.433583  9038 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:58.557626  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 131891us
I20260709 13:02:58.553246  1542 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:58.558199  1616 raft_consensus.cc:2393] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 19: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:02:58.558053  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 83855.4us
I20260709 13:02:58.470541  8960 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.539939  1618 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: []
I20260709 13:02:58.499095  1691 pending_rounds.cc:107] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:02:58.592146  1691 raft_consensus.cc:2889] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:58.350013  9150 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.543396  9195 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.572173  1539 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:02:58.554431  9036 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.577775  1687 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:58.476557  9196 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:58.537051  9208 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.587515  1508 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:58.544621  9201 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:58.609061  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 225925us
I20260709 13:02:58.608742  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:58.613742  1689 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 13:02:58.575510  9191 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:02:58.574012  8547 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 19. Current term is 20.
W20260709 13:02:58.623288  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 63253.1us
I20260709 13:02:58.618726  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:02:58.611891  9185 consensus_meta.cc:287] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.544s	user 0.013s	sys 0.000s
I20260709 13:02:58.624361  1612 raft_consensus.cc:2393] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 21: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:02:58.594573  9138 consensus_meta.cc:287] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.002s
I20260709 13:02:58.625368  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:58.648904  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 89351.1us
I20260709 13:02:58.626758  1611 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:58.641599  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:58.652638  1610 raft_consensus.cc:2468] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:58.580087  8525 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:58.627957  2231 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
W20260709 13:02:58.614516  8974 consensus_meta.cc:287] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.659s	user 0.012s	sys 0.000s
I20260709 13:02:58.656567  1536 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:58.642972  1583 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:58.642084  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 277480us
I20260709 13:02:58.647284  1690 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:02:58.569862  8832 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 13:02:58.623803  9201 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:02:58.607955  9100 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:02:58.667280  1618 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:02:58.608670  9186 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:02:58.640723  9208 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:58.664264  1537 raft_consensus.cc:2468] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:02:58.670594  1508 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:58.671659  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 46256.8us
I20260709 13:02:58.609167  8994 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:02:58.669031  1610 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:02:58.619575  9198 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:02:58.667786  9181 log.cc:927] Time spent T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.356s	user 0.004s	sys 0.000s
I20260709 13:02:58.655540  9213 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:58.679497  1612 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:02:58.656418  9145 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:58.348786  9171 consensus_queue.cc:1175] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "83f6f5810950417195f707ff308a8125" responder_term: 21 status { last_received { term: 19 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:02:58.613689  9036 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.659776  9124 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:58.673310  8974 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:58.672112  9199 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.693882  1583 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:02:58.698237  1507 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Timed out: RequestConsensusVote RPC to 127.0.107.195:33859 timed out after 1.604s (SENT)
I20260709 13:02:58.698789  1507 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:58.641819  8960 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.666369  8547 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 13:02:58.653921  9185 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.655345  9138 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.730825  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.732777  9138 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:58.686761  9190 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:58.686951  9208 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:58.691694  9200 raft_consensus.cc:697] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:58.740286  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.716547  9217 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:02:58.718724  9195 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.725251  8960 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.755914  8789 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:58.756028  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 61538.9us
I20260709 13:02:58.700906  9197 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:02:58.727087  9215 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:58.708303  9214 raft_consensus.cc:2804] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:58.729763  1540 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:02:58.730201  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:58.772934  1536 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:58.773348  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.700093  9124 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:58.730877  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 60416.2us
I20260709 13:02:58.774528  1616 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:02:58.732787  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:58.738736  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 120146us
I20260709 13:02:58.680459  9191 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 13:02:58.754232  1617 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:02:58.707497  9199 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:58.755280  1538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.725605  1614 raft_consensus.cc:3060] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:58.763279  9036 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.764297  9183 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:58.780380  1542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.766906  9185 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:02:58.770541  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 77907.4us
I20260709 13:02:58.771548  1611 raft_consensus.cc:2468] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:58.774557  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:58.775383  1535 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:02:58.682662  8525 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.749828  8974 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:58.758315  9072 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.793292  9038 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:58.793871  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:58.794466  9220 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:58.794740  8975 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:58.795954  9126 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.809823  9223 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:58.810144  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:58.799151  1536 raft_consensus.cc:2468] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
I20260709 13:02:58.785825  9217 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:58.809319  9145 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:58.818742  1536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.824235  1536 raft_consensus.cc:2410] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 20 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 13:02:58.821950  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:58.825757  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.790189  8957 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:58.826787  1537 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:02:58.786684  9214 raft_consensus.cc:697] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:58.810441  9223 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.822721  9201 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.831526  1543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125"
I20260709 13:02:58.801954  9199 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:58.825631  9217 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.837838  9036 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:58.827551  9038 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:02:58.842190  9219 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:02:58.845053  1614 raft_consensus.cc:2468] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 24.
I20260709 13:02:58.851922  9215 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:02:58.817461  8994 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:58.854059  9214 consensus_queue.cc:237] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.856827  8525 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:58.857367  9208 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:58.886808  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:58.858794  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.853058  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:58.860891  1543 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:02:58.811364  8876 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:58.866998  8957 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 24 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:58.854566  9124 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:58.879741  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:58.881685  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:58.881248  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:58.885262  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:58.806278  9183 consensus_queue.cc:1048] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:02:58.887486  2234 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:02:58.822804  9088 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:02:58.893405  2228 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 26.
I20260709 13:02:58.830370  9145 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:58.898110  2233 raft_consensus.cc:2468] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:02:58.904932  2229 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:58.907265  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:58.884148  9200 consensus_queue.cc:237] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.909605  9225 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:58.910048  9201 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:58.911756  9217 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:02:58.913339  9193 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.785s	user 0.010s	sys 0.000s
I20260709 13:02:58.916932  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:58.917970  2235 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:58.831712  8974 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.878187  9036 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:58.937996  1543 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:02:58.942696  2229 raft_consensus.cc:1240] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 20. Current term is 22. Ops: []
I20260709 13:02:58.939579  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:58.949040  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:58.951061  2235 raft_consensus.cc:2410] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:02:58.951081  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:58.953575  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:58.954321  2228 raft_consensus.cc:2393] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 20: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:58.957125  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.957523  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:58.958295  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:02:58.945909  9001 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 1.149s	user 0.015s	sys 0.000s
I20260709 13:02:58.958848  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.960089  1690 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:02:58.894548  8994 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.966508  1684 raft_consensus.cc:2393] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 17: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:58.962757  1687 raft_consensus.cc:2393] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 25: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:58.971091  2229 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:02:58.972702  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:58.973553  2235 raft_consensus.cc:2393] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 24: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:58.974154  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.975281  9036 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:58.979036  1543 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:02:58.979967  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:58.981464  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:58.982803  1508 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:278: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
I20260709 13:02:58.985106  2233 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 23. Current term is 24. Ops: [21.14-23.15]
I20260709 13:02:58.975335  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.898562  8876 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.939672  9124 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:02:58.998803  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:58.999627  2231 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:58.999426  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:59.002540  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:59.003894  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:58.965648  9145 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:58.946412  8525 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 22.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.044s
I20260709 13:02:59.007265  1537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
W20260709 13:02:58.999205  9048 consensus_meta.cc:287] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.532s	user 0.000s	sys 0.011s
/apache/dev/git/kudu/src/kudu/util/test_util.cc:402: Failure
Failed
Timed out waiting for assertion to pass.
I20260709 13:02:59.025844  1539 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:59.007460  1535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:59.025584  1540 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:58.998070  9046 raft_consensus.cc:2804] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:58.991081  9191 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:02:58.968230  8789 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:59.009403  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.035632  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.026532  1537 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:59.033390  1540 raft_consensus.cc:2393] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 22: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:02:59.039958  2233 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:59.038878  1692 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:02:59.040294  2231 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 22: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:59.040438  1692 pending_rounds.cc:85] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 8
I20260709 13:02:59.040831  1692 pending_rounds.cc:107] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:02:58.995404  9036 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.041080  1692 raft_consensus.cc:2889] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:02:58.987601  9001 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:58.997056  9244 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:02:59.039028  9191 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.053171  1541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "83f6f5810950417195f707ff308a8125" is_pre_election: true
I20260709 13:02:59.026095  9088 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:02:59.058831  1543 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:02:58.994016  9126 consensus_queue.cc:1059] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:02:59.036800  9046 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.012526  9199 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:02:59.038672  9145 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:59.060407  1541 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:02:59.042775  9234 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:02:59.064953  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 80849.6us
I20260709 13:02:59.066551  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.062674   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:02:59.028039  9072 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:02:59.061303  9191 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:02:59.042830  9036 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:59.085983  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.065497  9145 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.065493  9046 raft_consensus.cc:3060] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:02:59.095577  9182 log.cc:927] Time spent T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.076s	user 0.005s	sys 0.004s
W20260709 13:02:59.096843  9242 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:02:59.068449  9223 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:59.007529  8957 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.065056  9199 consensus_queue.cc:1243] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:02:59.067373  9234 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.122818  2235 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 22. Current term is 23. Ops: []
I20260709 13:02:59.109880  9145 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:59.124722  1683 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
W20260709 13:02:59.126889  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 128949us
W20260709 13:02:59.132932  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 133557us
I20260709 13:02:59.134347  1658 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:59.136052  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.136758  2235 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:02:59.116791  9234 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:02:59.144517  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 76827.8us
I20260709 13:02:59.151289  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:59.151717  9001 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:59.152356  8876 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:59.153357  8994 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:59.158510  9036 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:59.164077  8858 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:59.170231  2230 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 23. Current term is 24. Ops: [21.14-23.15]
I20260709 13:02:59.142197  8983 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:59.174211  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:59.179692  1507 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:59.172163  9073 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:02:59.180986  9073 raft_consensus.cc:697] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:59.181864  9073 consensus_queue.cc:237] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:59.183105  9081 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:59.184948  9196 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.185853  9220 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.186300  9220 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:59.186899  9246 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:59.188083  9246 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:59.189491  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:59.190268  9246 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:02:59.190797  9246 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:59.191731  9246 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:59.190356  9145 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:59.194248  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.194481  9081 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:59.195004  9081 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.196198  9145 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:59.197410  9213 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:02:59.198880  9081 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:59.201965  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.206367  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:59.207310  1686 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 22: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:02:59.207625  9167 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 1.250s	user 0.003s	sys 0.003s
I20260709 13:02:59.208601  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:59.209256  1685 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:02:59.209758  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:59.210479  2232 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:59.210415  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.211229  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.211829  2233 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:02:59.209286  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.216045  2230 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 22: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:59.173467  9199 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
I20260709 13:02:59.218178  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.227003  2228 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 21.
W20260709 13:02:59.217371  9187 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 1.096s	user 0.003s	sys 0.007s
W20260709 13:02:59.220836  9230 log.cc:927] Time spent T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:02:59.237911  8983 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:02:59.256875  2203 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:59.264147  8960 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:59.264860  8960 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:59.267122  8960 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:59.247008  9081 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:02:59.280886  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2505) took 1026 ms. Trace:
I20260709 13:02:59.217620  9234 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:59.284507  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:59.285091  1683 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:59.286356  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:59.286906  1692 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:02:59.283113  2201 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:02:59.290748  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:59.282505  1657 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:59.281338  1689 rpcz_store.cc:276] 0709 13:02:58.254858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.270525 (+ 15667us) service_pool.cc:224] Handling call
0709 13:02:58.605288 (+334763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:02:58.618971 (+ 13683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:02:59.280650 (+661679us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b5400028d50. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:59.280811 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":760752000}
I20260709 13:02:59.294373  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:59.295696  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:59.296181  1692 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:02:59.294293  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:59.297783  1614 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
W20260709 13:02:59.302137  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet 22357a3949344a1792e5bae4a0291987: Illegal state: Replica 984c9089d9804652846f3caeb609d56e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 25 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:02:59.311467  2201 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:59.311589  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:59.312791  2201 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:59.313901  9217 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:59.314306  9217 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.314639  9217 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:59.317991  9201 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:59.318495  9201 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:59.318909  9201 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:59.293154  9073 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.006s
I20260709 13:02:59.322952  9223 raft_consensus.cc:2764] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:02:59.323415  9036 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:59.324260  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:02:59.330933  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 48535.6us
I20260709 13:02:59.336619  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:59.337280  1609 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:59.281445  9191 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.303032  9150 raft_consensus.cc:2804] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:02:59.342595  9150 raft_consensus.cc:697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 24 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:59.343961  9234 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:59.343648  9150 consensus_queue.cc:237] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.330888  1615 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:02:59.351485  1683 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:02:59.352890  2237 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:02:59.353889  1609 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:02:59.356103  1686 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:02:59.309170  9138 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:59.342641  9191 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 27 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:59.369571  9248 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:02:59.374895  9138 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:02:59.375379  8525 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:59.393643  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:59.400431  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:59.401310  1614 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:02:59.364526  9036 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:59.405897  1611 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:02:59.406849  1542 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 24 LEADER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 13:02:59.392649  8994 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.377735  9246 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 13:02:59.400846  9138 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:59.427927  9245 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.414371  8983 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.421147  1611 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:02:59.427315  9036 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:59.439232  1614 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:59.441962  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:59.455345  1609 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 22: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:02:59.406167  9046 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.445312  8994 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:59.446450  9220 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.410997  8525 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.466691  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:02:59.468905  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:02:59.471026  2203 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:59.468116  1509 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:59.476011  2203 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:59.480249  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:02:59.454900  9245 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.483858  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:59.485148  1691 raft_consensus.cc:2410] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:02:59.489259  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:59.492035  2203 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:59.495543  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:59.493780  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:59.488276  9246 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:02:59.500265  8858 raft_consensus.cc:2804] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:02:59.501098  8858 raft_consensus.cc:697] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:59.503688  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:59.456940  8983 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:59.487207  9215 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:02:59.508297  1509 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:59.488547  9203 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:59.511082  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:59.493634  9224 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:02:59.502277  9190 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:59.510185  9088 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:02:59.514259  9214 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.529239  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:59.512665  9203 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:59.519872  9190 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.519619  9226 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:02:59.533835  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:02:59.502138  8858 consensus_queue.cc:237] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:59.549191  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.538094  9190 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:59.562619  9217 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.563613  9245 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:59.528456  9088 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.520700  9201 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.532311  9203 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:59.541973  9191 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 13:02:59.578367  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.579977  1582 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:59.583437  9046 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:59.584851  8525 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:02:59.570128  9088 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:59.585716  9191 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:02:59.587014  9191 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:59.590276  9258 log.cc:927] Time spent T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:02:59.549590  9226 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:59.533833  9214 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:59.599869  9214 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:59.601699  8983 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:59.610460  9220 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:02:59.612342  9253 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:02:59.615937  9150 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:59.616685  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39466 (request call id 1449) took 1560 ms. Trace:
I20260709 13:02:59.616636  9150 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.625003  9150 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:02:59.568136  9217 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:59.617161  2236 rpcz_store.cc:276] 0709 13:02:58.056511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.086501 (+ 29990us) service_pool.cc:224] Handling call
0709 13:02:59.616448 (+1529947us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54005b0254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:02:59.616620 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2369058048}
I20260709 13:02:59.636042  9048 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:59.636574  9048 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:59.597025  9226 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:59.637485  9048 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.649714  8579 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:02:59.655107  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 146879us
W20260709 13:02:59.655843  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 73950.1us
I20260709 13:02:59.663552  1509 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:59.663493  8983 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:02:59.652184  9072 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:02:59.666486  8983 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:02:59.666916  9046 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:02:59.673003  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 339955us
I20260709 13:02:59.679186  1657 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:59.680087  1657 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:59.681968  9197 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:02:59.682384  9197 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.682706  9197 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:02:59.685730  9124 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:02:59.673918  9234 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:02:59.687500  9234 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:02:59.687898  9234 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:02:59.674731  9245 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:02:59.688813  9234 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.681133  1657 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:59.693733  9124 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:02:59.698678  1657 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:59.702507  1657 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:59.692453  9234 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:02:59.705951  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 44134us
I20260709 13:02:59.711196  1582 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:59.713871  1582 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:59.707106  9124 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:02:59.715730  9072 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:02:59.717094  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:02:59.718072  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.716007  9124 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:59.719563  9124 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:59.721304  1687 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:02:59.717367  9072 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.718652  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:59.707726  9248 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:59.658591  9138 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.724666  8579 consensus_queue.cc:1048] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:02:59.726195  9073 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:02:59.728133  9138 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:59.625190  9190 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:59.730006  8960 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:02:59.736302  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:02:59.723455  9072 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:59.715057  9246 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:02:59.732870  9248 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:59.744800  9220 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:59.724985  9226 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:59.749076  1582 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:59.733115  9145 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:02:59.752657  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.760664  9205 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:02:59.748675  9248 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:02:59.755123  1658 consensus_peers.cc:597] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:59.771184  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:02:59.771708  1658 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:02:59.772488  1658 consensus_peers.cc:597] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:59.776468  2236 raft_consensus.cc:3055] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:02:59.776800  1686 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:02:59.776940  2236 raft_consensus.cc:740] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:59.780342  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:59.781275  2236 consensus_queue.cc:260] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:59.781895  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 84241.5us
I20260709 13:02:59.785152  9048 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:02:59.785816  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.741478  8960 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:02:59.770925  8957 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:02:59.778743  1683 raft_consensus.cc:1275] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:02:59.778569  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.775698  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:59.777776  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:59.753074  9226 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.779768  1687 raft_consensus.cc:1275] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:02:59.789985  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.795905  2233 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:02:59.791078  2231 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:02:59.804244  2229 raft_consensus.cc:1275] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:02:59.763139  9145 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.808327  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:02:59.813867  9018 consensus_queue.cc:1048] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:02:59.825239  2235 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 13:02:59.825644  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 1.17546e+06us
I20260709 13:02:59.807361  2201 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:59.829339  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:02:59.831496  1684 raft_consensus.cc:1275] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:02:59.833108  2233 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: [19.14-19.14]
W20260709 13:02:59.835691  1658 leader_election.cc:336] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:59.836503  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:02:59.837122  1687 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:02:59.800880  9226 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:59.837242  2201 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:02:59.841959  2229 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:02:59.835538  1584 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:02:59.845489  1584 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:59.833335  9018 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:02:59.791371  9088 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:59.805735  9190 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:59.831257  9145 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:02:59.810622  8957 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:02:59.852639  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:59.859526  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:02:59.847536  9040 raft_consensus.cc:2804] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:02:59.848244  9205 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:02:59.852979  8960 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:02:59.848564  8975 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.011s
I20260709 13:02:59.855291  8620 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:02:59.852610  9220 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:02:59.836426  9124 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:59.876673  1658 consensus_peers.cc:597] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:59.874347  8960 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:02:59.898114  8960 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:02:59.888689  9040 raft_consensus.cc:697] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:02:59.901715  9267 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:02:59.902292  9040 consensus_queue.cc:237] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.854990  9018 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.910027  8620 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:02:59.862375  9001 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:02:59.862005  9217 consensus_queue.cc:1048] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:02:59.896859  8579 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:02:59.864598  9138 consensus_queue.cc:1048] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 13:02:59.917589  2229 raft_consensus.cc:1275] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 13:02:59.920596  1658 leader_election.cc:336] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:59.898751  9205 raft_consensus.cc:2804] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:02:59.921579  8620 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:02:59.927513  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 178951us
I20260709 13:02:59.863811  8957 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:02:59.940861  1582 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:02:59.942462  2235 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: []
I20260709 13:02:59.949616  9220 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:02:59.954582  1658 leader_election.cc:336] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:02:59.900110  9197 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:02:59.970476  1508 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:02:59.977255  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.933061  8579 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:02:59.967746  1658 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:02:59.939353  9205 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.007804  9220 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:00.013779  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:02:59.954190  9203 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.021826  9203 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:02:59.996570  8579 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:00.023593  9088 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:00.045157  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 74780.8us
I20260709 13:02:59.987661  9046 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:00.006855  9205 raft_consensus.cc:3060] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:00.048734  1687 raft_consensus.cc:2468] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
W20260709 13:03:00.049850  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 109075us
I20260709 13:03:00.116428  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:00.116822  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:00.117254  1617 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 23: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:03:00.117609  1616 raft_consensus.cc:2393] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:03:00.103947  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 157836us
W20260709 13:03:00.092231  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 45397.2us
I20260709 13:03:00.119987  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:00.049850  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 190419us
I20260709 13:03:00.120639  1692 raft_consensus.cc:2468] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:03:00.112787  2231 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 27.
I20260709 13:03:00.127120  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:03:00.127656  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet ebc9ea99300141e883fcf2f1b0d7a744: Illegal state: Replica 8a6a5ae1b7f240408b3ddd30d8a18262 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:00.130107  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:00.131901  1584 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:00.135535  1683 raft_consensus.cc:1240] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 13:03:00.137295  2233 raft_consensus.cc:1240] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 26. Current term is 27. Ops: []
W20260709 13:03:00.138525  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.105185  8983 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:00.142155  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:00.142820  1690 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:03:00.049374  9246 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:00.147202  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.056419  9277 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:00.148659  1688 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 13:03:00.149362  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.152913  1688 pending_rounds.cc:85] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 15
I20260709 13:03:00.147568  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:00.153131  1611 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:03:00.153546  1615 raft_consensus.cc:3060] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:00.153167  1688 pending_rounds.cc:107] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 13:03:00.154309  1688 raft_consensus.cc:2889] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:00.092372  9048 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:00.155759  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:00.156349  1685 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:00.141597  9274 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:00.151050  1614 raft_consensus.cc:1275] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:03:00.152086  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:00.161523  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:00.128489  8975 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.164615  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:00.165306  1687 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:03:00.166077  1690 raft_consensus.cc:1240] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 13:03:00.169341  1616 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:00.168952  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:00.171900  1683 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:03:00.178218  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.077939  9046 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.091609  8876 consensus_queue.cc:1059] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 13:03:00.123200  9220 consensus_queue.cc:1048] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 13:03:00.190166  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:00.190856  1616 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:00.142499  9088 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 13:03:00.196159  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.181565  9124 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:00.194486  9276 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:03:00.163633  9048 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.181053  9280 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:00.176693  8975 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.211119  1614 raft_consensus.cc:1275] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:03:00.214872  1613 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:03:00.159600  9277 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:00.201651  9018 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.205631  9281 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:00.237067  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 85069us
W20260709 13:03:00.239184  1613 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2931) took 1686 ms (client timeout 1790 ms). Trace:
W20260709 13:03:00.239796  1613 rpcz_store.cc:269] 0709 13:02:58.548196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.551404 (+  3208us) service_pool.cc:224] Handling call
0709 13:03:00.214271 (+1662867us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54003c7420. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa63dc0 00007fe08faefe18 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6
0709 13:03:00.234631 (+ 20360us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317178624}
I20260709 13:03:00.206097  8957 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:00.228525  8789 raft_consensus.cc:3055] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:03:00.229580  9197 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.235160  9170 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:00.256098  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.266197  1616 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:03:00.248090  8789 raft_consensus.cc:740] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:00.268954  9088 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:00.299759  1616 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:03:00.244984  9274 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:03:00.254525  9205 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:00.299783  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:00.305757  9072 consensus_meta.cc:287] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.554s	user 0.005s	sys 0.006s
I20260709 13:03:00.288309  9272 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:00.308208  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.304461  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:00.294385  9145 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:00.243665  9281 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.317245  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.324574  9274 consensus_queue.cc:1243] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Peer 83f6f5810950417195f707ff308a8125 log is divergent from this leader: its last log entry 24.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:03:00.325975  8789 consensus_queue.cc:260] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.327100  8789 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:00.269116  9170 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.327278  9272 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:00.330219  1616 raft_consensus.cc:1240] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: []
I20260709 13:03:00.303682  8876 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.295075  9268 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:03:00.293934  9197 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:03:00.340749  1618 raft_consensus.cc:2393] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 21: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:00.361186  1685 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
I20260709 13:03:00.305613  9088 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:00.311380  9248 consensus_meta.cc:287] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.549s	user 0.005s	sys 0.007s
I20260709 13:03:00.308969  9214 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:00.363093  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.366546  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:00.372895  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:00.363559  9283 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 13:03:00.381268  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:00.392508  1509 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:00.392954  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:00.311579  9072 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:00.388311  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:03:00.439004  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 50808.3us
W20260709 13:03:00.440462  2237 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39452 (request call id 1774) took 1533 ms (client timeout 1707 ms). Trace:
I20260709 13:03:00.441183  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:00.441118  2237 rpcz_store.cc:269] 0709 13:02:58.906738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:58.906927 (+   189us) service_pool.cc:224] Handling call
0709 13:03:00.440227 (+1533300us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54007404d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:03:00.440383 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":985922432}
I20260709 13:03:00.443976  1615 raft_consensus.cc:2468] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:03:00.446391  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:03:00.450055  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 57659.6us
I20260709 13:03:00.369262  9046 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.448617  1507 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:00.459836  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:00.460276  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:00.462586  2203 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:00.402853  9199 raft_consensus.cc:3055] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:00.523823  1614 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:00.525797  1692 raft_consensus.cc:1240] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 13:03:00.463776  2203 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:00.382249  9196 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:00.406174  9048 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.411072  9088 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:00.556849  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:00.528146  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 64437.9us
I20260709 13:03:00.380712  9205 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:00.403386  9277 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:00.536592  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:00.453636  9208 raft_consensus.cc:2804] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:00.536662  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:00.398244  9018 consensus_queue.cc:1059] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.142s
I20260709 13:03:00.541538  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.372225  9248 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:00.557797  2203 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:00.572624  2203 leader_election.cc:336] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:00.563759  2228 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 23: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:00.564970  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.466732  8620 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:00.464843  9290 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:03:00.378626  9281 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.568004  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.473491  9046 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:00.565034  9088 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.566881  9208 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.628826  9208 raft_consensus.cc:3060] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:00.367014  8876 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:00.630674  9289 log.cc:927] Time spent T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.265s	user 0.003s	sys 0.000s
I20260709 13:03:00.633322  9072 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:00.453302  9223 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.446590  9275 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:00.570340  1615 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:03:00.407949  8579 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:00.562397  8975 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:00.584116  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:00.584913  9279 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 13:03:00.586380  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:00.573041  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:00.614853  1610 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:03:00.552937  9272 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:00.654101  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 81504.9us
W20260709 13:03:00.654870  2203 leader_election.cc:336] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:00.655483  2203 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:00.655872  2203 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:00.581897  9018 consensus_queue.cc:1243] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:03:00.656477  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:00.656980  2203 leader_election.cc:336] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:00.622210  9170 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.632253  8876 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.558064  9199 raft_consensus.cc:740] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:00.668081  9199 consensus_queue.cc:260] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.671216  9199 raft_consensus.cc:3060] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:00.572010  9277 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:00.627002  1688 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:03:00.456684  8957 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:00.692237  9277 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.694628  9277 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.641708  9223 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.696719  8579 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:00.634240  9046 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:00.696751  9223 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:00.697616  9046 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:00.701016  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:00.702142  1683 raft_consensus.cc:2368] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 22. Current term is 23.
I20260709 13:03:00.620021  9088 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.705451  9088 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.709658  9208 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.712080  9208 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.453724  9214 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.632627  8620 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.455670  8983 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.182s
I20260709 13:03:00.672789  9145 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:00.674335  8994 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:00.624439  1689 raft_consensus.cc:3060] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:00.677770  2230 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:00.624114  9248 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:00.678576  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.684279  1508 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:00.791582  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 107390us
I20260709 13:03:00.793716  8994 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.794059  1689 raft_consensus.cc:2468] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:03:00.794263  8994 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:00.794677  9208 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:00.795258  9208 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.796572  8994 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:00.797147  8994 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:03:00.716280  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.717100  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.717206  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:00.832834  1609 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:00.717921  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.834856  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.835805  1610 raft_consensus.cc:2368] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22.
I20260709 13:03:00.836789  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:00.838064  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:00.839604  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:00.840476  1612 raft_consensus.cc:2368] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22.
I20260709 13:03:00.840574  2236 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:00.718150  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.842926  1614 raft_consensus.cc:2393] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 22: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:00.718796  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.720077  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.844450  1611 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:03:00.844882  1613 raft_consensus.cc:2410] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
W20260709 13:03:00.720688  2202 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.846082  1610 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:00.688033  9001 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:00.708681  9046 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:00.729708  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:00.848137  1618 raft_consensus.cc:2376] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22. Re-sending same reply.
I20260709 13:03:00.466485  9234 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.781086  1615 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:03:00.849045  9046 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:00.849388  9234 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.783332  9214 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.785247  8620 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:00.688784  9026 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:03:00.852010  9234 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.852334  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.739769  9215 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.853364  9215 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.854275  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:00.639448  8960 consensus_meta.cc:287] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.736s	user 0.007s	sys 0.006s
I20260709 13:03:00.797353  9208 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.807160  9246 raft_consensus.cc:3055] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:03:00.816608  9226 consensus_meta.cc:287] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.965s	user 0.008s	sys 0.005s
I20260709 13:03:00.785966  9040 consensus_queue.cc:1048] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 13:03:00.876053  9280 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.876694  9280 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.878913  9215 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:00.880170  1610 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 24.
I20260709 13:03:00.804225  9201 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:00.824692  2230 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
I20260709 13:03:00.827678  2201 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:03:00.829737  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 143637us
I20260709 13:03:00.855788  9246 raft_consensus.cc:740] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:00.857309  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:03:00.892463  2203 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:00.893007  1609 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:03:00.858526  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:00.859823  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.894070  1613 raft_consensus.cc:2368] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 22. Current term is 23.
I20260709 13:03:00.860052  8876 consensus_queue.cc:1059] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:00.862772  1615 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:03:00.895902  1440 catalog_manager.cc:5697] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:00.863960  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.865579  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:00.898073  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 1703) took 1131 ms. Trace:
I20260709 13:03:00.898504  1689 rpcz_store.cc:276] 0709 13:02:59.766731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.766845 (+   114us) service_pool.cc:224] Handling call
0709 13:03:00.897897 (+1131052us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b540028f0d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:03:00.898016 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1939617664}
I20260709 13:03:00.869789  9234 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:00.899767  9234 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:00.855901  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:00.817900  9293 consensus_queue.cc:1059] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:00.902390  9214 raft_consensus.cc:3055] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:03:00.902876  9214 raft_consensus.cc:740] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:00.904072  9214 consensus_queue.cc:260] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.881666  9226 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:00.882956  9280 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:00.855744  8960 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:00.905613  9214 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:03:00.906241  1616 raft_consensus.cc:1240] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 13:03:00.907344  9226 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:00.895745  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68189.8us
I20260709 13:03:00.909970  2201 leader_election.cc:400] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 22. Current term is 23.
I20260709 13:03:00.910408  2201 leader_election.cc:403] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:03:00.911217  2201 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:00.912070  9040 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859) with higher term. Message: Invalid argument: T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 22. Current term is 23.
I20260709 13:03:00.913179  9040 raft_consensus.cc:2804] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:00.913337  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:00.914186  9040 raft_consensus.cc:697] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:00.915485  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:00.915524  9040 consensus_queue.cc:237] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:00.893311  2203 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:00.901021  9246 consensus_queue.cc:260] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:00.908030  9234 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:00.908104  8960 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:00.895025  9201 raft_consensus.cc:2804] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:00.872682  8994 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 27 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:00.922158  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:00.976897  2232 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:03:00.978221  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:00.923990  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:00.925410  2237 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:03:00.925895  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:00.924003  2233 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:00.900327  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:00.927616  1658 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:00.980789  2234 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 24.
I20260709 13:03:00.948400  2231 raft_consensus.cc:1240] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 13:03:00.981498  1658 leader_election.cc:336] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.534s (SENT)
W20260709 13:03:00.938246  1617 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.016s	sys 0.037s
W20260709 13:03:00.982077  1617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.016s	sys 0.037s
W20260709 13:03:00.934703  2203 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:00.982885  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 54444.3us
W20260709 13:03:00.983500  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:00.984707  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:00.986999  2203 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:00.988346  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:00.989231  1689 raft_consensus.cc:2468] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
W20260709 13:03:00.975803  8974 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.343s	user 0.020s	sys 0.061s
I20260709 13:03:00.990671  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:00.991008  2203 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:00.997340  2203 leader_election.cc:336] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:00.977885  1440 catalog_manager.cc:5697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 9 to 12, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 12 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:00.996446  2232 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:03:01.006002  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:01.006506  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:01.008780  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:01.012390  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:00.995987  8974 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.363s	user 0.020s	sys 0.062s
I20260709 13:03:00.987699  9201 raft_consensus.cc:697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:03:01.014329  1507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.000s	sys 0.051s
W20260709 13:03:01.017709  1507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.000s	sys 0.051s
W20260709 13:03:01.018015  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 569461us
I20260709 13:03:01.015519  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:01.011219  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:01.022856  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:01.023687  2236 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:03:01.024055  2236 pending_rounds.cc:85] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 14
I20260709 13:03:01.024253  2236 pending_rounds.cc:107] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:03:01.024459  2236 raft_consensus.cc:2889] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:01.024659  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:01.025151  2236 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1471) took 1257 ms. Trace:
W20260709 13:03:01.026718  1655 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.006s	sys 0.044s
W20260709 13:03:01.027554  1655 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.007s	sys 0.044s
I20260709 13:03:01.025586  2236 rpcz_store.cc:276] 0709 13:02:59.768271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:59.768432 (+   161us) service_pool.cc:224] Handling call
0709 13:02:59.776344 (+  7912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.657794 (+881450us) spinlock_profiling.cc:243] Waited 85.1 ms on lock 0x7b18002cd660. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fb27646 00007fe08fa6158e 00007fe08fae2f9e 00007fe08fae47de 00007fe08faeeb49 00007fe08faef416 00007fe08faebd8f 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4
0709 13:03:00.708229 (+ 50435us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b5800904dd0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa480aa 00007fe08faca447 00007fe08fae2fe7 00007fe08fae47de 00007fe08faeeb49 00007fe08faef416 00007fe08faebd8f 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4
0709 13:03:00.840277 (+132048us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b58009050d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa480aa 00007fe08faca447 00007fe08fae2fe7 00007fe08fae47de 00007fe08faeeb49 00007fe08faef416 00007fe08faebd8f 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4
0709 13:03:01.024876 (+184599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.025098 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551902592}
I20260709 13:03:01.010388  9246 raft_consensus.cc:3060] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:01.040424  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.011947  9124 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:01.028973  9234 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:01.010054  8525 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:03:01.045295  9018 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:03:01.069521  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.037634  8974 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.111083  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:01.045205  8876 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.045044  9048 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.064824  1658 leader_election.cc:400] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22.
I20260709 13:03:01.111914  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:01.148383  1612 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:03:01.116923  1583 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:01.136600  1440 catalog_manager.cc:5697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 12 to 14, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 14 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:01.141544  1434 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40172 (request call id 235) took 23249 ms (client timeout 15000 ms). Trace:
W20260709 13:03:01.148072  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 102020us
I20260709 13:03:01.158679  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:01.044713  9268 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:01.159703  1609 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:01.158253  9048 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.168378  1611 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:03:01.171224  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:01.172062  1610 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:01.173985  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:01.179572  1440 catalog_manager.cc:5697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 8 to 9, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 9 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.148010  9264 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.148648  9244 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:01.194377  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 77722.8us
I20260709 13:03:01.196944  1610 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
W20260709 13:03:01.189711  1658 leader_election.cc:336] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.653s (SENT)
I20260709 13:03:01.199120  1658 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:01.103981  9190 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 13:03:01.117920  9245 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 13:03:01.211325  1440 catalog_manager.cc:5697] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:01.212742  1658 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.707s (SENT)
I20260709 13:03:01.122509  8983 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:03:01.249835  1440 catalog_manager.cc:5697] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 9 to 10, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 10 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:01.216535  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:01.178164  9268 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.215240  8525 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:01.121889  9191 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 13:03:01.185999  9208 raft_consensus.cc:2804] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:01.167337  9201 consensus_queue.cc:237] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.193957  8620 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.153927  8876 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:01.223142  9215 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.206265  8975 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:01.146023  8974 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.263548  8789 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.049141  8994 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.234059  2229 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 13:03:01.281136  1692 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.016s	sys 0.058s
W20260709 13:03:01.281494  1692 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.016s	sys 0.058s
I20260709 13:03:01.237401  9048 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:01.284333  9295 log.cc:927] Time spent T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:03:01.214481  9244 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:01.256863  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 57441.5us
I20260709 13:03:01.259985  1610 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 13:03:01.299789  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:01.300916  1610 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:03:01.263253  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:01.264889  1684 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.008s	sys 0.008s
I20260709 13:03:01.268493  1609 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19. Ops: []
W20260709 13:03:01.260757  9272 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.000s	sys 0.039s
W20260709 13:03:01.275700  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 62999.1us
I20260709 13:03:01.275739  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:01.208132  9264 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.263130  9208 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
W20260709 13:03:01.155292  1434 rpcz_store.cc:269] 0709 13:02:37.897778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:37.918177 (+ 20399us) service_pool.cc:224] Handling call
0709 13:02:39.399974 (+1481797us) spinlock_profiling.cc:243] Waited 96.3 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:03:01.141265 (+21741291us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:02:54.988335 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:02:55.151773 (+163438us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:02:55.155337 (+  3564us) write_op.cc:435] Acquired schema lock
0709 13:02:55.162103 (+  6766us) tablet.cc:662] Decoding operations
0709 13:02:55.597475 (+435372us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:02:55.600498 (+  3023us) write_op.cc:641] Partition lock acquired for write op
0709 13:02:55.603751 (+  3253us) tablet.cc:685] Acquiring locks for 18 operations
0709 13:02:55.830976 (+227225us) tablet.cc:701] Row locks acquired
0709 13:02:55.830995 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:02:55.854903 (+ 23908us) write_op.cc:270] Start()
0709 13:02:55.867644 (+ 12741us) write_op.cc:276] Timestamp: P: 1783602175850541 usec, L: 0
0709 13:02:55.867671 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:02:55.951042 (+ 83371us) log.cc:844] Serialized 7044 byte log entry
0709 13:02:56.454625 (+503583us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:02:57.351156 (+896531us) op_driver.cc:464] REPLICATION: finished
0709 13:02:57.797502 (+446346us) write_op.cc:301] APPLY: starting
0709 13:02:57.797648 (+   146us) tablet.cc:1367] starting BulkCheckPresence
0709 13:02:58.839446 (+1041798us) tablet.cc:1370] finished BulkCheckPresence
0709 13:02:58.839493 (+    47us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:02:59.223117 (+383624us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:02:59.223564 (+   447us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=36,mrs_lookups=0
0709 13:02:59.223634 (+    70us) write_op.cc:312] APPLY: finished
0709 13:02:59.224215 (+   581us) log.cc:844] Serialized 164 byte log entry
0709 13:02:59.481463 (+257248us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:59.569181 (+ 87718us) write_op.cc:454] Released schema lock
0709 13:02:59.641511 (+ 72330us) write_op.cc:341] FINISH: Updating metrics
0709 13:02:59.645922 (+  4411us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:02:59.645954 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5787548,"reported_tablets":39,"spinlock_wait_cycles":221561728,"child_traces":[["op",{"apply.queue_time_us":442167,"apply.run_cpu_time_us":63608,"apply.run_wall_time_us":1848452,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79475,"num_ops":18,"prepare.queue_time_us":274104,"prepare.run_cpu_time_us":139694,"prepare.run_wall_time_us":1252561,"raft.queue_time_us":529066,"raft.run_cpu_time_us":105881,"raft.run_wall_time_us":313408,"replication_time_us":1481287,"spinlock_wait_cycles":30488960,"thread_start_us":1356617,"threads_started":5,"wal-append.queue_time_us":652707,"wal-append.run_cpu_time_us":54073,"wal-append.run_wall_time_us":3058158}]]}
I20260709 13:03:01.311244  9048 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:01.314046  1658 leader_election.cc:336] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:01.315426  1684 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.008s	sys 0.009s
I20260709 13:03:01.324283  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1362) took 1199 ms. Trace:
I20260709 13:03:01.326974  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:01.328174  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:01.328783  1610 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:01.318131  9272 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.000s	sys 0.040s
I20260709 13:03:01.282363  8975 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:01.338840  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:01.324986  1617 rpcz_store.cc:276] 0709 13:03:00.117216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.124915 (+  7699us) service_pool.cc:224] Handling call
0709 13:03:00.127212 (+  2297us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:00.349544 (+222332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.349595 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:00.364940 (+ 15345us) log.cc:844] Serialized 52 byte log entry
0709 13:03:00.982460 (+617520us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:00.982504 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.003337 (+ 20833us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b5400105254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:03:01.003394 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.285527 (+282133us) raft_consensus.cc:1685] finished
0709 13:03:01.285560 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.316443 (+ 30883us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22866,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":10073,"spinlock_wait_cycles":486521600,"thread_start_us":544713,"threads_started":1,"wal-append.queue_time_us":795093}
I20260709 13:03:01.337874  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:01.329473  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:01.351539  1610 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:03:01.273120  8620 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:01.369786  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.321853  9208 raft_consensus.cc:3060] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:01.370422  1509 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
W20260709 13:03:01.373955  1658 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.790s (SENT)
W20260709 13:03:01.378211  1658 consensus_peers.cc:597] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:01.285279  9046 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 13:03:01.290143  8789 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.367228  9190 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:03:01.280717  9215 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.370666  8525 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:01.327705  9048 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:01.393669  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 64329.6us
I20260709 13:03:01.405830  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:01.387825  9298 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.414652  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3563) took 1279 ms. Trace:
I20260709 13:03:01.371898  9299 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:03:01.397665  9191 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.402125  9272 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:01.407435  8983 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:03:01.421082  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.423604  1617 raft_consensus.cc:1275] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:03:01.425781  9299 consensus_queue.cc:1243] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer 8a6a5ae1b7f240408b3ddd30d8a18262 log is divergent from this leader: its last log entry 23.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 13:03:01.329946  9275 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.001s	sys 0.015s
I20260709 13:03:01.427098  1612 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:01.419180  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:01.412174  9302 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.444749  9302 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.446776  8876 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:01.447979  9264 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:01.448134  9302 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:01.427230  9300 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 13:03:01.425323  8960 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.429006  9272 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:01.407691  8525 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.431005  9191 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:01.457525  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 459415us
W20260709 13:03:01.468202  2203 leader_election.cc:336] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:01.472126  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:01.421528  9280 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 13:03:01.415272  1685 rpcz_store.cc:276] 0709 13:03:00.135295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.438665 (+303370us) service_pool.cc:224] Handling call
0709 13:03:00.452418 (+ 13753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.452639 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:00.932186 (+479547us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.409144 (+476958us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:01.409213 (+    69us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:01.409276 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.409369 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.414551 (+  5182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":709993,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":99,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":7159,"thread_start_us":955714,"threads_started":2}
I20260709 13:03:01.441685  8983 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:01.478415  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:01.479269  2237 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
W20260709 13:03:01.443763  9275 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.002s	sys 0.015s
I20260709 13:03:01.487396  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:01.458505  9272 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:01.493515  1685 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 23: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:03:01.418624  9298 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:01.503185  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:01.437005  8975 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.457340  8960 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.486042  8620 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:01.507458  9199 consensus_meta.cc:287] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.002s
I20260709 13:03:01.542356  1691 raft_consensus.cc:1275] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 13:03:01.542856  1691 pending_rounds.cc:85] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 13
I20260709 13:03:01.543143  1691 pending_rounds.cc:107] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:03:01.543416  1691 raft_consensus.cc:2889] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:01.502043  8974 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:01.545506  1689 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.015s	sys 0.003s
W20260709 13:03:01.546615  1689 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.015s	sys 0.003s
W20260709 13:03:01.548866  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 143136us
I20260709 13:03:01.533465  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2541) took 1311 ms. Trace:
I20260709 13:03:01.534353  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:01.421664  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 479079us
I20260709 13:03:01.563349  1685 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:03:01.564862  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:01.565888  1685 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:03:01.580963  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:01.582350  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:01.583063  1685 raft_consensus.cc:2468] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:03:01.630834  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:01.567358  1690 rpcz_store.cc:276] 0709 13:03:00.222165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.607574 (+385409us) service_pool.cc:224] Handling call
0709 13:03:00.607918 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.608149 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:01.067467 (+459318us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.532964 (+465497us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:01.533041 (+    77us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:01.533105 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.533201 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.533373 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6626,"prepare.queue_time_us":555196,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":19582,"thread_start_us":916784,"threads_started":2,"wal-append.queue_time_us":424746}
I20260709 13:03:01.588085  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2536) took 1442 ms. Trace:
I20260709 13:03:01.594877  9215 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:01.595896  9305 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:01.586211  9226 consensus_queue.cc:1059] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:03:01.546898  8789 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:01.598140  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:01.560851  9280 raft_consensus.cc:2764] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
W20260709 13:03:01.612056  1687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.009s	sys 0.012s
I20260709 13:03:01.576956  1582 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:01.587455  9124 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.601218  8983 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:01.792836  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:01.717052  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 75670.9us
I20260709 13:03:01.818011  1685 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:03:01.820534  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:03:01.818739  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 241851us
W20260709 13:03:01.709009  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.776669  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:01.759488  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:01.821892  1687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.012s	sys 0.022s
I20260709 13:03:01.781158  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1710) took 1097 ms. Trace:
I20260709 13:03:01.822686  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2539) took 1624 ms. Trace:
I20260709 13:03:01.776319  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:01.858695  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:01.859939  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:01.823061  1687 rpcz_store.cc:276] 0709 13:03:00.200209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.604319 (+404110us) service_pool.cc:224] Handling call
0709 13:03:00.604680 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.604897 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:03:01.093463 (+488566us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.822382 (+728919us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:01.822435 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:03:01.822476 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.822537 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.822637 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":558211,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"thread_start_us":1189899,"threads_started":2,"wal-append.queue_time_us":706478}
I20260709 13:03:01.861688  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:01.844177  1658 leader_election.cc:336] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:01.865561  1658 leader_election.cc:336] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:01.822986  1691 rpcz_store.cc:276] 0709 13:03:00.683990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.715890 (+ 31900us) service_pool.cc:224] Handling call
0709 13:03:00.716231 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.543882 (+827651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.780799 (+236917us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b54000f6254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:03:01.781063 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1899032832}
I20260709 13:03:01.867789  9274 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:01.868362  9274 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:01.868079  9026 consensus_queue.cc:1048] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:01.868819  9274 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:01.857797  9124 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.872435  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:01.648204  9244 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:01.649212  9306 log.cc:927] Time spent T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 13:03:01.783272  9311 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:03:01.877089  1688 raft_consensus.cc:2393] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 23: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:03:01.879112  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:01.782481  9246 consensus_meta.cc:287] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.737s	user 0.012s	sys 0.000s
I20260709 13:03:01.879223  1692 rpcz_store.cc:276] 0709 13:03:00.145529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.542018 (+396489us) service_pool.cc:224] Handling call
0709 13:03:00.542328 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:00.542540 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:01.282218 (+739678us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.587641 (+305423us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:01.587713 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:01.587763 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.587826 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.587956 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1450,"prepare.queue_time_us":620337,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":31,"raft.queue_time_us":3458,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":125,"thread_start_us":1016859,"threads_started":2,"wal-append.queue_time_us":587420}
I20260709 13:03:01.882915  1582 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:01.745909  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:01.884079  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:01.760016  9268 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:01.783485  9298 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:01.882304  1658 leader_election.cc:336] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:01.868049  8620 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:01.813907  9191 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:01.881906  9040 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:01.894904  1685 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:03:01.769362  9214 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:01.895458  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:01.895915  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:01.811630  8525 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:01.886118  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:01.784853  9001 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:01.898842  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:01.899173  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:01.784284  8960 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:01.903558  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:01.882103  9026 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:01.690519  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3562) took 1556 ms. Trace:
W20260709 13:03:01.904286  1507 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.021s	sys 0.021s
I20260709 13:03:01.905490  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:01.926167  1507 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.546s	user 0.021s	sys 0.021s
I20260709 13:03:01.906559  2201 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:01.925689  1686 rpcz_store.cc:276] 0709 13:03:00.134539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.411215 (+276676us) service_pool.cc:224] Handling call
0709 13:03:00.411708 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.412193 (+   485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.412234 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:00.446959 (+ 34725us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.037395 (+590436us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:01.500082 (+462687us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.645801 (+145719us) spinlock_profiling.cc:243] Waited 8.62 ms on lock 0x7b540030eba0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fa67733 00007fe08faecdbf 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c
0709 13:03:01.645888 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.646027 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.646108 (+    81us) raft_consensus.cc:1685] finished
0709 13:03:01.690186 (+ 44078us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.690427 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2955,"prepare.queue_time_us":141299,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":12241,"raft.queue_time_us":724,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":21673,"spinlock_wait_cycles":19836800,"thread_start_us":880782,"threads_started":2,"wal-append.queue_time_us":698299}
I20260709 13:03:01.926388  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:01.929396  1688 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:01.650291  9248 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:01.907637  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:01.909714  1509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.011s	sys 0.022s
I20260709 13:03:01.746528  9288 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.719646  9234 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.647918  9048 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:01.910914  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:01.889243  9199 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.942922  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:01.916307  1687 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
I20260709 13:03:01.813781  9272 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.806638  9208 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.873517  8579 consensus_queue.cc:1059] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:03:01.943856  1686 raft_consensus.cc:2410] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 13:03:01.914743  9214 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:01.923604  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:01.948261  1685 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:03:01.949687  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:01.931255  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:01.933848  1509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.011s	sys 0.022s
I20260709 13:03:01.936040  2235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1487) took 1010 ms. Trace:
W20260709 13:03:01.938050  1658 leader_election.cc:336] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 2.113s (SENT)
I20260709 13:03:01.939271  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2537) took 1778 ms. Trace:
I20260709 13:03:01.921787  9124 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:01.953804  1684 rpcz_store.cc:276] 0709 13:03:00.160721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.595567 (+434846us) service_pool.cc:224] Handling call
0709 13:03:00.596010 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.596444 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.596475 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:00.596994 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.326546 (+729552us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:01.326615 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.326705 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.939023 (+612318us) raft_consensus.cc:1685] finished
0709 13:03:01.939053 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.939204 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19045,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":89,"thread_start_us":729036,"threads_started":1,"wal-append.queue_time_us":894075}
I20260709 13:03:01.956440  1658 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:01.924611  1692 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 23
W20260709 13:03:01.958544  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 588146us
I20260709 13:03:01.898190  9215 consensus_queue.cc:1048] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 13:03:01.959717  1509 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:01.976519  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:01.979266  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:01.979965  9246 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:01.980305  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1717) took 1258 ms. Trace:
I20260709 13:03:01.980713  9246 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.964357  9199 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:01.944031  9191 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.982570  9191 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:01.931999  8960 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:01.983955  1685 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:03:01.981040  1687 rpcz_store.cc:276] 0709 13:03:00.721641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.978085 (+1256444us) service_pool.cc:224] Handling call
0709 13:03:01.979381 (+  1296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.979715 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.979762 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:01.979832 (+    70us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:01.979889 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.979979 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.980175 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:01.992089  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1715) took 1295 ms. Trace:
I20260709 13:03:01.994197  1690 rpcz_store.cc:276] 0709 13:03:00.697030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.951025 (+1253995us) service_pool.cc:224] Handling call
0709 13:03:01.990757 (+ 39732us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.991450 (+   693us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.991599 (+   149us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:01.991753 (+   154us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:01.991799 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.991874 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.992015 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:01.998757  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1716) took 1301 ms. Trace:
I20260709 13:03:01.946348  9280 raft_consensus.cc:3055] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:02.003803  1684 rpcz_store.cc:276] 0709 13:03:00.697329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.956000 (+1258671us) service_pool.cc:224] Handling call
0709 13:03:01.998102 (+ 42102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.998330 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.998367 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:01.998423 (+    56us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:01.998466 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.998536 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.998684 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:01.934265  9018 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:01.952075  2235 rpcz_store.cc:276] 0709 13:03:00.925337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.925480 (+   143us) service_pool.cc:224] Handling call
0709 13:03:00.925797 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.926148 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:00.926175 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:00.931870 (+  5695us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.397935 (+466065us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:01.397974 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.401133 (+  3159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:01.935728 (+534595us) raft_consensus.cc:1685] finished
0709 13:03:01.935756 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:01.935970 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":78,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"thread_start_us":415257,"threads_started":1,"wal-append.queue_time_us":581771}
I20260709 13:03:02.028637  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1718) took 1304 ms. Trace:
I20260709 13:03:02.029225  1686 rpcz_store.cc:276] 0709 13:03:00.723805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.003035 (+1279230us) service_pool.cc:224] Handling call
0709 13:03:02.027932 (+ 24897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.028180 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.028217 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.028274 (+    57us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:02.028328 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.028402 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.028563 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:02.030884  2203 leader_election.cc:336] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:02.032688  1507 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:02.033035  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:01.985664  9040 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 13:03:02.037139  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 114016us
I20260709 13:03:02.044745  1658 leader_election.cc:400] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22.
I20260709 13:03:02.045179  1658 leader_election.cc:403] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 13:03:02.045662  1657 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:02.047943  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:02.050333  1687 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:02.051199  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1719) took 1327 ms. Trace:
I20260709 13:03:02.051584  1687 rpcz_store.cc:276] 0709 13:03:00.724094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.023330 (+1299236us) service_pool.cc:224] Handling call
0709 13:03:02.050098 (+ 26768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.050888 (+   790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.051129 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:02.049716  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:02.063895  1692 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:03:01.927881  9001 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:02.049692  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 185558us
I20260709 13:03:02.069902  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:02.010815  9280 raft_consensus.cc:740] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:02.076263  1657 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:01.945199  9234 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:02.120688  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 50954.9us
W20260709 13:03:02.119791  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 71974.6us
I20260709 13:03:01.942782  9288 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:02.019728  9048 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:01.981328  9246 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:02.185214  1692 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:03:02.186112  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:02.186712  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1721) took 1081 ms. Trace:
I20260709 13:03:02.187348  1692 rpcz_store.cc:276] 0709 13:03:01.105806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.125069 (+1019263us) service_pool.cc:224] Handling call
0709 13:03:02.131287 (+  6218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.186358 (+ 55071us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.186633 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:02.092401  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2591) took 1007 ms. Trace:
I20260709 13:03:02.128969  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2545) took 1261 ms. Trace:
I20260709 13:03:02.191231  1686 rpcz_store.cc:276] 0709 13:03:00.867138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.067357 (+1200219us) service_pool.cc:224] Handling call
0709 13:03:02.128193 (+ 60836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.128433 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.128469 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.128526 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:02.128590 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.128662 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.128898 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:02.185246  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:02.010129  8960 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:02.189108  1614 rpcz_store.cc:276] 0709 13:03:01.085177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.085285 (+   108us) service_pool.cc:224] Handling call
0709 13:03:01.110369 (+ 25084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.110641 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:03:01.547736 (+437095us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.091898 (+544162us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.091982 (+    84us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:03:02.092043 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.092144 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.092318 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":383508,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"thread_start_us":860876,"threads_started":2}
I20260709 13:03:02.042411  9275 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.216600  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 278663us
I20260709 13:03:02.219118  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1723) took 1071 ms. Trace:
I20260709 13:03:02.229391  1692 rpcz_store.cc:276] 0709 13:03:01.147287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.217168 (+1069881us) service_pool.cc:224] Handling call
0709 13:03:02.218059 (+   891us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.218534 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.218606 (+    72us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.218727 (+   121us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:02.218798 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.218887 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.219041 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:02.122437  8876 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:02.041056  9040 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:02.071318  9299 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:02.233023  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:02.066612  8579 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:02.119026  9124 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22.
I20260709 13:03:02.122803  9340 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:02.237440  1692 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:03:02.226790  9298 raft_consensus.cc:2804] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:02.053234  8789 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.134s
I20260709 13:03:02.240962  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2590) took 1156 ms. Trace:
I20260709 13:03:02.066964  8974 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:02.126665  9268 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:02.071509  9329 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:02.213059  9302 raft_consensus.cc:3055] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:03:02.234843  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 351963us
I20260709 13:03:02.257092  1692 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:02.263518  1509 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:02.261729  1616 rpcz_store.cc:276] 0709 13:03:01.084874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.085028 (+   154us) service_pool.cc:224] Handling call
0709 13:03:01.085349 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:01.085577 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 13:03:01.489223 (+403646us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.239854 (+750631us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.239921 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:02.239987 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.240711 (+   724us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.240884 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":408289,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":23875,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":1153453,"threads_started":3,"wal-append.queue_time_us":486514}
I20260709 13:03:02.068444  9244 consensus_queue.cc:1059] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.170s
W20260709 13:03:02.250082  9330 log.cc:927] Time spent T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 13:03:02.264592  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:03:02.246686  9040 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:02.242698  9331 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.340s	user 0.001s	sys 0.000s
I20260709 13:03:02.055755  9018 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:02.266743  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:02.297202  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:02.298141  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.300022  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.291177  9244 raft_consensus.cc:3055] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:02.290712  9268 consensus_queue.cc:1059] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:03:02.265586  9302 raft_consensus.cc:740] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:03:02.302783  9338 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 13:03:02.304054  9335 log.cc:927] Time spent T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 13:03:02.270995  9048 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:02.253047  8579 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.311479  1692 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:03:02.309804  1613 raft_consensus.cc:2376] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 24. Re-sending same reply.
I20260709 13:03:02.313799  1615 raft_consensus.cc:1275] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:03:02.314779  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 111090us
W20260709 13:03:02.320137  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 55606.3us
I20260709 13:03:02.258925  9299 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:02.324482  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:02.254729  9339 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 13:03:02.264887  9298 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:02.256328  8620 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:02.343549  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:02.327651  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.373203  1683 raft_consensus.cc:2410] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 13:03:02.376950  1688 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:03:02.390712  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40114 (request call id 3575) took 1008 ms. Trace:
I20260709 13:03:02.391160  1688 rpcz_store.cc:276] 0709 13:03:01.382280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.926119 (+543839us) service_pool.cc:224] Handling call
0709 13:03:02.390617 (+464498us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:03:02.393807  1684 raft_consensus.cc:1240] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:03:02.313687  9244 raft_consensus.cc:740] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:03:02.395735  9341 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:03:02.377468  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:02.266987  9329 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:02.400437  9329 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:02.402061  9299 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:02.403239  9245 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:02.403878  9336 log.cc:927] Time spent T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.349s	user 0.000s	sys 0.001s
W20260709 13:03:02.406212  9337 log.cc:927] Time spent T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:02.380203  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:02.320237  9191 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:02.395213  9244 consensus_queue.cc:260] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:02.341701  9264 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:02.378072  9268 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:02.378733  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.410099  1688 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 13:03:02.410396  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 200218us
I20260709 13:03:02.411172  2229 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1500) took 1033 ms. Trace:
W20260709 13:03:02.385613  9326 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.440s	user 0.000s	sys 0.001s
I20260709 13:03:02.437574  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.438500  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.415853  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:02.418490  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.407464  9208 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:02.380884  1692 raft_consensus.cc:1240] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 24. Current term is 25. Ops: []
I20260709 13:03:02.235844  9275 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:02.261608  9280 consensus_queue.cc:260] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:02.378999  8579 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:02.380637  9340 raft_consensus.cc:3055] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:02.275300  8876 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:02.390215  9298 raft_consensus.cc:3060] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:02.414870  9334 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.390s	user 0.001s	sys 0.001s
I20260709 13:03:02.419866  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.436403  2229 rpcz_store.cc:276] 0709 13:03:01.378434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.378636 (+   202us) service_pool.cc:224] Handling call
0709 13:03:01.381225 (+  2589us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.397974 (+ 16749us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.398008 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.398499 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.830545 (+432046us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:01.830602 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.869187 (+ 38585us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b54005b0254. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:03:01.869240 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.390682 (+521442us) raft_consensus.cc:1685] finished
0709 13:03:02.390715 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.411100 (+ 20385us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":282,"raft.queue_time_us":6650,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":5977,"spinlock_wait_cycles":24558464,"thread_start_us":431683,"threads_started":1,"wal-append.queue_time_us":499154}
I20260709 13:03:02.467868  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.493485  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:03:02.469240  1618 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.000s	sys 0.014s
W20260709 13:03:02.495807  1618 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.001s	sys 0.014s
I20260709 13:03:02.457124  1616 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:03:02.482577  1692 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3578) took 1085 ms. Trace:
I20260709 13:03:02.472487  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:55314 (request call id 1) took 2276 ms. Trace:
I20260709 13:03:02.487112  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:03:02.439399  9332 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.398s	user 0.000s	sys 0.001s
I20260709 13:03:02.458515  9264 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:03:02.485140  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 69369.6us
I20260709 13:03:02.498050  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.497028  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.532161  1617 raft_consensus.cc:2393] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 19: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:02.511461  1613 raft_consensus.cc:2368] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 22. Current term is 23.
I20260709 13:03:02.511801  1612 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 13:03:02.457394  9048 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:02.514521  1685 rpcz_store.cc:276] 0709 13:03:00.182364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.425512 (+2243148us) service_pool.cc:224] Handling call
0709 13:03:02.458550 (+ 33038us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:02.438973  9346 log.cc:927] Time spent T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.171s	user 0.001s	sys 0.001s
I20260709 13:03:02.514134  1692 rpcz_store.cc:276] 0709 13:03:01.403659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.379566 (+975907us) service_pool.cc:224] Handling call
0709 13:03:02.380387 (+   821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.482073 (+101686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.482485 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:03:02.559266  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 291175us
W20260709 13:03:02.519893  1583 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.430919  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 354745us
I20260709 13:03:02.536754  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:02.487260  9244 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:02.486810  9340 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:02.491511  9191 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.567883  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1502) took 1096 ms. Trace:
I20260709 13:03:02.568519  2231 rpcz_store.cc:276] 0709 13:03:01.471315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.471462 (+   147us) service_pool.cc:224] Handling call
0709 13:03:01.471821 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:01.472253 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.472291 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:01.491820 (+ 19529us) log.cc:844] Serialized 26 byte log entry
0709 13:03:01.921982 (+430162us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:01.922024 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.922097 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.567570 (+645473us) raft_consensus.cc:1685] finished
0709 13:03:02.567602 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.567802 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":303,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":12267,"thread_start_us":411754,"threads_started":1,"wal-append.queue_time_us":758241}
I20260709 13:03:02.578879  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:02.457711  9268 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:02.517242  9293 raft_consensus.cc:493] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.533020  8525 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:02.430919  9302 consensus_queue.cc:260] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:02.531667  9246 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:02.549768  9296 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:03:02.601933  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet c8377b7234e4472ea27005f16dede669: Illegal state: Replica 984c9089d9804652846f3caeb609d56e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:02.558100  9001 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:02.636090  9302 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:02.638068  9345 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 13:03:02.640157  1689 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1713) took 1955 ms. Trace:
W20260709 13:03:02.622929  9274 consensus_meta.cc:287] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.754s	user 0.012s	sys 0.000s
I20260709 13:03:02.641839  9366 consensus_queue.cc:1048] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:02.642156  9274 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:02.585498  9245 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.594761  9191 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:02.650975  2200 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.650795  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.612939  9296 consensus_queue.cc:1059] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:02.615383  9344 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:02.616896  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2549) took 1311 ms. Trace:
I20260709 13:03:02.670454  1683 rpcz_store.cc:276] 0709 13:03:01.305700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.917704 (+612004us) service_pool.cc:224] Handling call
0709 13:03:02.616599 (+698895us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b540003e154. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:03:02.616812 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":968584448}
I20260709 13:03:02.617892  9264 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:02.619030  9343 log.cc:927] Time spent T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 13:03:02.618865  9248 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:02.624140  9201 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:02.622929  8620 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.007s
I20260709 13:03:02.630265  9268 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:02.631249  9324 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.381s	user 0.000s	sys 0.002s
I20260709 13:03:02.633121  9124 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.640659  1689 rpcz_store.cc:276] 0709 13:03:00.684693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.989984 (+305291us) service_pool.cc:224] Handling call
0709 13:03:00.990319 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.999631 (+  9312us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 13:03:01.547111 (+547480us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.123497 (+576386us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.153620 (+ 30123us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.153778 (+   158us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:02.153821 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.153903 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.639871 (+485968us) raft_consensus.cc:1685] finished
0709 13:03:02.639899 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.640078 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1131,"prepare.queue_time_us":164111,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":12878,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":34028,"thread_start_us":999411,"threads_started":2,"wal-append.queue_time_us":694239}
W20260709 13:03:02.611802  9348 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 13:03:02.592428  9234 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:02.610486  8983 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:02.610620  9352 raft_consensus.cc:493] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.650137  9246 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:02.601333  9354 consensus_queue.cc:1048] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.083s
I20260709 13:03:02.711781  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:02.712755  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:02.715529  1692 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:03:02.716511  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:02.716516  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:02.604048  9293 raft_consensus.cc:515] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:02.672808  9332 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:03:02.720518  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:02.721397  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:02.720530  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:02.723033  1692 raft_consensus.cc:2393] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 20: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:03:02.722608  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 144239us
W20260709 13:03:02.718308  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 156277us
I20260709 13:03:02.725626  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.729521  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.731732  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.702466  9268 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:02.673409  9245 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:02.734018  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:02.712978  9234 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:02.694168  9248 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:02.736923  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:02.738375  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.739064  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.733703  1614 raft_consensus.cc:2410] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 21 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:03:02.708897  9296 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:02.745873  1617 raft_consensus.cc:1240] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 24. Current term is 25. Ops: [22.15-24.16]
I20260709 13:03:02.747814  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:02.734798  1658 leader_election.cc:336] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:02.736033  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:02.750324  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:02.749540  1683 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:03:02.756112  1612 raft_consensus.cc:2410] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 25 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 13:03:02.755725  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:02.699831  9201 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:02.760627  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.762152  1616 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:02.724035  9342 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.708933  9124 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:02.766848  1613 raft_consensus.cc:2393] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 22: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:02.767419  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.741581  9361 raft_consensus.cc:3055] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:03:02.739729  8974 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.768692  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 44833.8us
I20260709 13:03:02.778402  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:02.782469  2228 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 13:03:02.782848  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:02.785941  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:02.786661  2234 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:03:02.786957  1692 raft_consensus.cc:1275] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 13:03:02.788929  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.791915  2232 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 22. Current term is 23. Ops: []
W20260709 13:03:02.779613  1658 leader_election.cc:336] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:02.786330  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:02.798485  1617 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:03:02.717247  9352 raft_consensus.cc:515] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:02.805447  1617 raft_consensus.cc:1275] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:02.806295  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:02.806684  1609 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:02.797526  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:02.816082  2230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:49846 (request call id 1734) took 1903 ms. Trace:
I20260709 13:03:02.821102  2230 rpcz_store.cc:276] 0709 13:03:00.912228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:00.918976 (+  6748us) service_pool.cc:224] Handling call
0709 13:03:00.919428 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:00.925822 (+  6394us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:03:01.396883 (+471061us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.065717 (+668834us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.112158 (+ 46441us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.112385 (+   227us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:02.112435 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.112530 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.815762 (+703232us) raft_consensus.cc:1685] finished
0709 13:03:02.815794 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.816000 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1545,"prepare.queue_time_us":613201,"prepare.run_cpu_time_us":1593,"prepare.run_wall_time_us":5602,"raft.queue_time_us":83825,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":11687,"thread_start_us":1113489,"threads_started":3,"wal-append.queue_time_us":543933}
I20260709 13:03:02.689494  9340 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:02.824472  9324 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:03:02.826889  9248 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:02.827229  9364 consensus_queue.cc:1048] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:02.808866  1582 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:02.827546  9248 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:02.841806  1612 raft_consensus.cc:1275] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:03:02.778441  9361 raft_consensus.cc:740] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:02.845836  1618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2596) took 1738 ms. Trace:
I20260709 13:03:02.749348  9268 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:02.787321  9358 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:02.874001  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2556) took 1566 ms. Trace:
I20260709 13:03:02.874619  1684 rpcz_store.cc:276] 0709 13:03:01.307871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.227852 (+919981us) service_pool.cc:224] Handling call
0709 13:03:02.393520 (+165668us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.394288 (+   768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.873712 (+479424us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b540028e6d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:03:02.873918 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336251776}
I20260709 13:03:02.757719  9234 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:02.852854  1618 rpcz_store.cc:276] 0709 13:03:01.107106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.107310 (+   204us) service_pool.cc:224] Handling call
0709 13:03:01.107716 (+   406us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:01.153950 (+ 46234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.153985 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:01.226442 (+ 72457us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b2c0005b3b8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:03:01.227183 (+   741us) log.cc:844] Serialized 52 byte log entry
0709 13:03:01.600274 (+373091us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:01.745531 (+145257us) log.cc:844] Serialized 14 byte log entry
0709 13:03:02.497340 (+751809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.507927 (+ 10587us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:02.828651 (+320724us) raft_consensus.cc:1685] finished
0709 13:03:02.845571 (+ 16920us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:02.845763 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":192765,"prepare.queue_time_us":137524,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":2042,"raft.queue_time_us":291832,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":13308,"spinlock_wait_cycles":7711488,"thread_start_us":964838,"threads_started":2,"wal-append.queue_time_us":898796}
I20260709 13:03:02.779394  9001 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 13:03:02.799566  8974 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:02.877839  9357 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:03:02.778828  9296 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:02.779392  9342 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:02.918113  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:02.932776  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 166634us
W20260709 13:03:02.947067  1584 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 114 similar messages]
I20260709 13:03:02.906122  9274 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:02.905630  9215 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:02.935453  9344 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:02.957482  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 148506us
W20260709 13:03:02.967265  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 165438us
W20260709 13:03:02.980581  1584 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:02.981120  1584 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:02.959045  9360 log.cc:927] Time spent T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
W20260709 13:03:03.025501  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 44942.6us
I20260709 13:03:03.028410  1691 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1714) took 2339 ms. Trace:
W20260709 13:03:03.030185  1584 leader_election.cc:336] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.029893  1691 rpcz_store.cc:276] 0709 13:03:00.689127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.945347 (+1256220us) service_pool.cc:224] Handling call
0709 13:03:01.961482 (+ 16135us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.005122 (+ 43640us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.005163 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.038240 (+ 33077us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.518514 (+480274us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:02.518697 (+   183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.518935 (+   238us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.028186 (+509251us) raft_consensus.cc:1685] finished
0709 13:03:03.028212 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.028343 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56806,"raft.queue_time_us":52537,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":12738,"thread_start_us":364687,"threads_started":1,"wal-append.queue_time_us":539245}
I20260709 13:03:03.031298  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:02.884609  9358 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:02.979168  9350 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:02.966503  8789 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.040994  1582 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:03:03.042726  1584 consensus_peers.cc:597] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:03.056731  1584 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:02.969846  9349 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.055462  1582 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:03.057210  1584 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:03.087625  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:03.024049  9362 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.053481  9367 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:02.970494  9215 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.179620  9377 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.092100  9293 leader_election.cc:290] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:02.985847  9344 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.240736  9344 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:03.022521  9214 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.242394  9214 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:03.240808  9370 log.cc:927] Time spent T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.343s	user 0.000s	sys 0.000s
I20260709 13:03:03.087854  9350 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.246807  1690 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2558) took 1938 ms. Trace:
I20260709 13:03:03.247756  1687 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2557) took 1939 ms. Trace:
I20260709 13:03:03.247215  1690 rpcz_store.cc:276] 0709 13:03:01.308521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.227696 (+919175us) service_pool.cc:224] Handling call
0709 13:03:02.243227 (+ 15531us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.243460 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.243491 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.257298 (+ 13807us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.548092 (+290794us) raft_consensus.cc:1645] Marking committed up to 19
0709 13:03:02.548156 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.548263 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.246589 (+698326us) raft_consensus.cc:1685] finished
0709 13:03:03.246610 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.246748 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":222227,"threads_started":1,"wal-append.queue_time_us":530300}
I20260709 13:03:03.248577  1687 rpcz_store.cc:276] 0709 13:03:01.308207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.227772 (+919565us) service_pool.cc:224] Handling call
0709 13:03:02.244088 (+ 16316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.417129 (+173041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.417379 (+   250us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:02.424399 (+  7020us) log.cc:844] Serialized 26 byte log entry
0709 13:03:02.607036 (+182637us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:02.607098 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:02.607210 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.247495 (+640285us) raft_consensus.cc:1685] finished
0709 13:03:03.247525 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.247688 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":69226,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":9147,"thread_start_us":354124,"threads_started":2,"wal-append.queue_time_us":777355}
I20260709 13:03:03.040609  9347 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.193s
W20260709 13:03:03.252746  9302 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.616s	user 0.013s	sys 0.001s
I20260709 13:03:03.253635  9302 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:03.254014  1611 raft_consensus.cc:2410] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:03:03.254192  9302 raft_consensus.cc:697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:03:03.260470  9040 consensus_meta.cc:287] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.929s	user 0.009s	sys 0.000s
W20260709 13:03:03.269002  8983 consensus_meta.cc:287] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.548s	user 0.004s	sys 0.004s
I20260709 13:03:03.270709  9234 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:03.271674  9124 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.274017  9199 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.275494  9358 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.278900  9367 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:03.144299  9362 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:03.198662  9245 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.209501  9201 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:03.286916  9268 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.215567  9352 leader_election.cc:290] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.146505  9191 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.240916  9379 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:03.092593  8789 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.291975  1615 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:03:03.292222  9342 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.252652  9347 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:03.255234  9302 consensus_queue.cc:237] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:03.323698  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 266985us
W20260709 13:03:03.329209  1506 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.347963  9018 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:03.351285  1614 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.007s	sys 0.002s
I20260709 13:03:03.296708  9350 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:03.118558  9349 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:03.308941  9377 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:03.374027  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:03.355633  8983 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:03.389950  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:03.355741  9214 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:03.391628  1614 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.007s	sys 0.002s
I20260709 13:03:03.409761  2229 raft_consensus.cc:2393] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 17: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:03.362280  9275 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:03.391845  9347 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.356030  9040 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.372929  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:02.959600  9361 consensus_queue.cc:260] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:03.376374  9201 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:03.404310  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:03.377025  9191 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:03.386377  1508 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:03:03.410094  8579 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.911s	user 0.006s	sys 0.006s
W20260709 13:03:03.378022  1584 consensus_peers.cc:597] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:03.408337  9394 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:03.430855  2230 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
W20260709 13:03:03.438849  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 60911.1us
W20260709 13:03:03.429689  2203 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.445578  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:03.446282  2203 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:03.425776  9347 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:03.442796  1584 consensus_peers.cc:597] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:03.448558  1584 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.424387  9296 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:03.448943  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 62703.6us
I20260709 13:03:03.426947  9274 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:03.450708  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 363138us
W20260709 13:03:03.446970  2203 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.449132  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:03.461790  1584 leader_election.cc:336] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:03.462586  1584 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.415402  9214 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:03.473873  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 677745us
I20260709 13:03:03.479144  1692 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
W20260709 13:03:03.481243  1658 leader_election.cc:336] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.481722  1658 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:03.463264  1584 consensus_peers.cc:597] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:03.485215  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:03.485561  1584 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.488409  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.438510  9201 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.461303  9296 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:03.472118  9367 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:03.483901  9329 consensus_meta.cc:287] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.082s	user 0.013s	sys 0.000s
W20260709 13:03:03.534018  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.488086  1584 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:03.503381  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:03.489952  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:03.498939  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:03.535828  1690 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:03.536074  1691 raft_consensus.cc:2410] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 23 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 13:03:03.502229  1612 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:03:03.507920  2203 leader_election.cc:336] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.504449  9208 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:03.515234  9377 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:03.484486  9352 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:03.489229  9396 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:03.501267  9400 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:03.489459  9293 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:03.486791  9268 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:03.523144  9201 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:03.569082  9404 log.cc:927] Time spent T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:03:03.569721  9387 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.008s
I20260709 13:03:03.570936  9040 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:03.504096  9354 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.030s	sys 0.053s
W20260709 13:03:03.560305  9244 consensus_meta.cc:287] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.977s	user 0.013s	sys 0.000s
I20260709 13:03:03.526069  9362 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:03.477546  9353 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:03.478041  9374 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 13:03:03.448551  9394 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.466892  8983 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:03.447543  8579 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.446612  9191 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.564687  9396 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:03:03.562680  9215 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:03.565491  9400 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:03.559907  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 74385.3us
I20260709 13:03:03.484752  9389 consensus_queue.cc:1059] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.089s
I20260709 13:03:03.562453  9329 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:03.561070  2203 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.566139  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.566973  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.559652  9296 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:03.533697  9298 consensus_meta.cc:287] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 1.016s	user 0.012s	sys 0.001s
I20260709 13:03:03.564143  9352 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.571874  9293 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.575181  9403 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:03.567565  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 66644.5us
I20260709 13:03:03.523132  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2984) took 1056 ms. Trace:
I20260709 13:03:03.575153  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:03.531172  9349 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:03.603006  1615 rpcz_store.cc:276] 0709 13:03:02.467059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.469971 (+  2912us) service_pool.cc:224] Handling call
0709 13:03:03.522930 (+1052959us) spinlock_profiling.cc:243] Waited 760 ms on lock 0x7b5400389354. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:03:03.523055 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1747245824}
I20260709 13:03:03.604619  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:03.588400  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1725) took 2357 ms. Trace:
W20260709 13:03:03.606392  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 122157us
I20260709 13:03:03.594177  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.588109  1616 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 22: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
W20260709 13:03:03.592733  2203 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:03.590350  1584 leader_election.cc:336] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.610369  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:03.574369  9354 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.031s	sys 0.053s
I20260709 13:03:03.612396  1509 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:03.612793  1617 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
W20260709 13:03:03.610195  2203 leader_election.cc:336] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:03.611518  1584 consensus_peers.cc:597] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:03.616894  1584 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.617831  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:03.618083  1584 leader_election.cc:336] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.618510  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:03.619488  1584 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.619837  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:03.593149  1618 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
W20260709 13:03:03.624135  1584 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.625154  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:03.622498  9403 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:03.587582  9191 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.628455  9040 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:03.633528  9264 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:03.583348  9394 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:03.638527  9244 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:03.587358  9396 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:03.600495  9352 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.639253  9381 consensus_queue.cc:1059] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:03:03.676882  9381 consensus_queue.cc:1243] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer 8a6a5ae1b7f240408b3ddd30d8a18262 log is divergent from this leader: its last log entry 23.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:03:03.677481  9381 raft_consensus.cc:3055] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:03:03.677834  9381 raft_consensus.cc:740] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:03.678752  9381 consensus_queue.cc:260] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.680136  9381 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:03.594785  8579 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.682580  9350 consensus_queue.cc:1048] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:03.684218  9350 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:03.624028  8620 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:03.686921  8525 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.687460  8620 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.687600  8525 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.589741  9288 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.690730  8525 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.585317  9347 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.694859  1612 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:03.591249  9280 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:03.696489  9347 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.595629  8789 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.605834  9293 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.599759  9214 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.597856  9296 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.596549  9298 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.613901  9349 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:03.583554  8983 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.657369  9379 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:03.606451  1686 rpcz_store.cc:276] 0709 13:03:01.230753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.193149 (+962396us) service_pool.cc:224] Handling call
0709 13:03:02.236362 (+ 43213us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:02.314070 (+ 77708us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.314141 (+    71us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:02.343092 (+ 28951us) log.cc:844] Serialized 52 byte log entry
0709 13:03:02.675080 (+331988us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:02.675134 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.587930 (+912796us) spinlock_profiling.cc:243] Waited 73.4 ms on lock 0x7b54001dbfd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:03:03.587987 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.588065 (+    78us) raft_consensus.cc:1685] finished
0709 13:03:03.588079 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.588337 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3546,"raft.queue_time_us":34298,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":15708,"spinlock_wait_cycles":168838400,"thread_start_us":295202,"threads_started":1,"wal-append.queue_time_us":904277}
I20260709 13:03:03.660367  1584 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:03.615664  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0631055540000034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
I20260709 13:03:03.578241  8960 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.594980  9389 raft_consensus.cc:3055] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:03:03.593302  9329 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.665737  1690 raft_consensus.cc:2468] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:03:03.668056  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:03.590180  9400 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:03.579282  8974 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:03.671165  2203 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:03.712023  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 87911.8us
W20260709 13:03:03.700704  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 240513us
I20260709 13:03:03.685036  9350 consensus_queue.cc:1048] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:03.713030  2230 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:03:03.675405  9201 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:03.668695  8876 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.037s
W20260709 13:03:03.710347  1658 connection.cc:256] RPC call timeout handler was delayed by -1.0055055540000239s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
W20260709 13:03:03.715670  1658 leader_election.cc:336] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.509s (SENT)
W20260709 13:03:03.716768  1658 leader_election.cc:336] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Timed out: RequestConsensusVote RPC to 127.0.107.194:43887 timed out after 1.543s (SENT)
W20260709 13:03:03.713194  1584 consensus_peers.cc:597] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:03.712217  2203 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.721529  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:03.718171  9349 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:03.731392  8620 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:03.729949  9389 raft_consensus.cc:740] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:03:03.733126  9365 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.804s	user 0.007s	sys 0.007s
I20260709 13:03:03.710215  8983 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:03.731103  9296 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:03.732929  9389 consensus_queue.cc:260] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:03.728544  9361 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:03.735625  9298 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:03.719483  9201 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:03.743271  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:03.744002  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 175437us
I20260709 13:03:03.747315  2233 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1501) took 2341 ms. Trace:
I20260709 13:03:03.748972  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:03.749162  2203 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:03.803313  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 54185.8us
W20260709 13:03:03.804395  2203 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.750358  1507 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:03.745690  1657 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:03.771572  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:03.806701  1686 raft_consensus.cc:2410] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:03:03.807595  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:03:03.772399  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 51136.7us
I20260709 13:03:03.802181  1683 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:03:03.803324  2230 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:03:03.796207  2201 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:03.813527  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 70395.5us
I20260709 13:03:03.813794  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.815102  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:03.815290  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 69736.8us
W20260709 13:03:03.816885  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 66648.5us
I20260709 13:03:03.817238  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:03.817972  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:03.803049  2233 rpcz_store.cc:276] 0709 13:03:01.400351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:01.400484 (+   133us) service_pool.cc:224] Handling call
0709 13:03:01.400887 (+   403us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:03:01.406989 (+  6102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:01.407022 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:03:01.418674 (+ 11652us) log.cc:844] Serialized 78 byte log entry
0709 13:03:01.717467 (+298793us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:01.748775 (+ 31308us) log.cc:844] Serialized 14 byte log entry
0709 13:03:01.751275 (+  2500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:01.751392 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:03.739869 (+1988477us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b240030b4f8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c8859c4 00007fe08c883acb 00007fe08ca3edc9 00007fe08ca3ea55 00007fe08ca3dfb0 00007fe08f6b4488 00007fe08f6b4be8 00007fe08f68c398 00007fe08f68776a 00007fe08f686bb6
0709 13:03:03.741479 (+  1610us) raft_consensus.cc:1685] finished
0709 13:03:03.741603 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.742046 (+   443us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2147,"prepare.queue_time_us":21195,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":9822,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":10748,"spinlock_wait_cycles":4898176,"thread_start_us":275144,"threads_started":1,"wal-append.queue_time_us":504602}
I20260709 13:03:03.824743  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:03.825762  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.824798  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:03.827629  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:03.830403  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.831140  1618 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:03:03.831440  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:03.833757  1507 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:03.836687  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:03.831974  1612 raft_consensus.cc:2376] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 24. Re-sending same reply.
W20260709 13:03:03.842480  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:03.843130  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.846258  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:03.846320  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.846978  1683 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:03.848546  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:03.812873  9201 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:03.845146  1582 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:03.851663  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.822674  9366 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:03.821799  9298 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:03.853749  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:03.855342  1658 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:03.830219  9296 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:03.837997  8974 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:03.829847  9392 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.856208  9124 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:03.837596  9362 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:03.909286  1507 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:03.839505  9001 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:03.839751  9374 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:03.840333  9018 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:03.918092  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.849005  9268 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:03.856338  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:03:03.862730  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.824397  9329 consensus_queue.cc:1059] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:03:03.906665  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:03.925104  9365 log.cc:927] Time spent T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:03:03.824527  9358 consensus_queue.cc:1048] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:03:03.919276  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2621) took 1182 ms. Trace:
I20260709 13:03:03.941550  1614 rpcz_store.cc:276] 0709 13:03:02.736545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.787189 (+ 50644us) service_pool.cc:224] Handling call
0709 13:03:02.787449 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:03.408985 (+621536us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:03:03.664501 (+255516us) log.cc:844] Serialized 14 byte log entry
0709 13:03:03.918804 (+254303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:03.918876 (+    72us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:03:03.918941 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:03.919032 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:03.919203 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73556,"prepare.queue_time_us":41597,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":178216,"raft.run_cpu_time_us":266,"raft.run_wall_time_us":17364,"thread_start_us":1021030,"threads_started":3}
I20260709 13:03:03.944506  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.857702  9389 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:03.902192  8525 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:03.901979  9296 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:03:03.959550  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 50402.3us
I20260709 13:03:03.960379  1507 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:03.852706  9199 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:03.939121  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 82858us
I20260709 13:03:03.906032  8960 raft_consensus.cc:2804] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:03.945350  1614 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:03:03.917634  9245 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:03.947587  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.947652  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.928779  9124 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:03.957499  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 102257us
I20260709 13:03:03.973989  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:03.974210  1584 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:03.972934  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 150916us
I20260709 13:03:03.958051  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:03.937047  1686 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 20.
I20260709 13:03:03.961782  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:03.975142  1584 leader_election.cc:336] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:03.975844  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:03.930393  9001 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:03.947717  8789 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:03.972661  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:03.977587  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:03.976429  1683 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:03:03.980268  8960 raft_consensus.cc:697] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
W20260709 13:03:03.980998  9411 log.cc:927] Time spent T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:03:03.981767  9412 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 13:03:03.973310  8876 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:03.930543  9392 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.982560  9124 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:03.983515  9358 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:03.957010  9214 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:03.983826  9340 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:03.984892  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:03.989588  1618 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:03.986563  8789 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:04.046996  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 69485.2us
I20260709 13:03:04.133720  9199 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:03:04.031075  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:04.029932  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:04.049610  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 60779.1us
W20260709 13:03:04.052234  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 83122.7us
W20260709 13:03:04.252755  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 205059us
I20260709 13:03:03.965996  9299 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:04.253572  2201 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:03.962330  9040 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:03.937855  9362 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:03.967356  9275 raft_consensus.cc:3055] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:04.260557  9275 raft_consensus.cc:740] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:03.964206  9268 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:04.121346  1614 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:03:03.977300  9400 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.264873  1686 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:03:03.982376  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:04.133889  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:04.154604  1655 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.265767  1684 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 26.
I20260709 13:03:04.266187  9430 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:04.267236  9400 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:03.994511  9001 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:04.134224  8876 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.151469  9329 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:04.173657  8960 consensus_queue.cc:237] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:03.964385  9296 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:04.213908  1688 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40124 (request call id 2563) took 1499 ms. Trace:
I20260709 13:03:04.200433  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:04.203542  2231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39466 (request call id 1509) took 1421 ms. Trace:
I20260709 13:03:04.211211  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:04.196991  9214 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:04.209734  9423 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:03:04.199481  1658 leader_election.cc:400] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 22. Current term is 23.
I20260709 13:03:04.303380  1658 leader_election.cc:403] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:04.203562  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:04.304004  1689 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:04.202288  9366 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:04.205693  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:04.200719  9364 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:04.207909  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:04.215108  1617 raft_consensus.cc:1240] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:03:04.203022  9392 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:04.231369  9415 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.249s	user 0.002s	sys 0.000s
I20260709 13:03:04.201318  8789 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:04.233443  8974 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:04.197400  9280 consensus_meta.cc:287] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.502s	user 0.008s	sys 0.002s
I20260709 13:03:04.240332  9344 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:04.240927  9427 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:04.241396  9359 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.202183  9272 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:04.241640  9428 consensus_queue.cc:1059] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:03:04.222435  9381 consensus_meta.cc:287] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.542s	user 0.006s	sys 0.006s
I20260709 13:03:04.245507  9018 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.165779  9358 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:04.070513  9397 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:04.259193  2201 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:04.336269  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 77203us
I20260709 13:03:04.295934  1688 rpcz_store.cc:276] 0709 13:03:02.714054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.714183 (+   129us) service_pool.cc:224] Handling call
0709 13:03:02.714498 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.903234 (+188736us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.903283 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.903339 (+    56us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:03:02.903386 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.213655 (+1310269us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54001dd3d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:03:04.213719 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.213850 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":433459072}
I20260709 13:03:04.261411  9275 consensus_queue.cc:260] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:04.297451  2231 rpcz_store.cc:276] 0709 13:03:02.782020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.782171 (+   151us) service_pool.cc:224] Handling call
0709 13:03:02.804254 (+ 22083us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:02.806207 (+  1953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.806255 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:02.806315 (+    60us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:03:02.806362 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.203304 (+1396942us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b5400730fd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faece01 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6
0709 13:03:04.203347 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:04.203483 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3977472}
I20260709 13:03:04.297797  1692 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 24.
I20260709 13:03:04.271108  9400 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:04.306159  1691 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 17.
W20260709 13:03:04.307261  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 107923us
I20260709 13:03:04.294811  8876 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:04.309965  1609 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 17.
I20260709 13:03:04.263332  9040 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:04.263731  9124 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:04.325049  1610 raft_consensus.cc:2410] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:03:04.310160  9268 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 22. Current term is 23.
I20260709 13:03:04.332067  9026 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:04.298305  9329 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:04.337131  2201 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:04.347899  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:04.348591  1688 raft_consensus.cc:1275] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:03:04.310986  9201 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:04.345592  9429 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:04.337852  9358 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:04.353111  8983 consensus_meta.cc:287] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: Time spent flushing consensus metadata: real 0.553s	user 0.011s	sys 0.000s
I20260709 13:03:04.369378  1689 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
I20260709 13:03:04.372022  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:04.375350  1611 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2975) took 2048 ms (client timeout 1609 ms). Trace:
W20260709 13:03:04.375809  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.376473  2201 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:04.315975  9366 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:04.326985  9344 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:04.379140  1611 rpcz_store.cc:269] 0709 13:03:02.326579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.327312 (+   733us) service_pool.cc:224] Handling call
0709 13:03:04.375175 (+2047863us) spinlock_profiling.cc:243] Waited 864 ms on lock 0x7b540028ffd4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:03:04.375296 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1986099072}
I20260709 13:03:04.385053  1611 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:03:04.385985  1609 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:03:04.382656  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:04.388304  1658 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:04.328925  9359 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:04.353690  9403 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.362026  9275 raft_consensus.cc:3060] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:04.393014  1683 raft_consensus.cc:2393] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 24: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:03:04.364809  9026 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:04.337924  9018 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:04.377985  9428 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:04.396261  9403 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:04.384979  9361 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:04.392743  9397 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:04.393024  8525 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:04.426606  9366 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:04.427212  9214 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:04.429836  8525 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:04.430488  8525 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 24 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:04.430034  9026 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:04.431497  1610 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43412 (request call id 2986) took 1962 ms (client timeout 2148 ms). Trace:
I20260709 13:03:04.431442  8525 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:04.431996  1610 rpcz_store.cc:269] 0709 13:03:02.468866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.493115 (+ 24249us) service_pool.cc:224] Handling call
0709 13:03:04.431289 (+1938174us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b5400102554. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:03:04.431436 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4209140096}
I20260709 13:03:04.384325  8983 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:04.433123  9268 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:04.433836  9268 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:04.434572  8983 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:04.436174  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:04.404661  9375 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:03:04.460165  9416 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.486s	user 0.000s	sys 0.004s
I20260709 13:03:04.473889  9361 raft_consensus.cc:493] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:04.474642  9361 raft_consensus.cc:515] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:04.477061  9361 leader_election.cc:290] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:04.426630  9040 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:04.327781  9427 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:04.482358  9403 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:04.493691  9268 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:04.495286  9329 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:04.496902  9344 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:04.499781  9389 consensus_meta.cc:287] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.000s
I20260709 13:03:04.488999  9299 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:04.431531  9026 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:04.529745  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:04.530058  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:04.437631  9362 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:04.454489  9375 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
W20260709 13:03:04.541973  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 159520us
I20260709 13:03:04.547629  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:04.489061  8876 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:04.543172  9375 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:04.591950  1616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45356 (request call id 1373) took 1853 ms. Trace:
I20260709 13:03:04.593639  1616 rpcz_store.cc:276] 0709 13:03:02.738232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.788703 (+ 50471us) service_pool.cc:224] Handling call
0709 13:03:04.591686 (+1802983us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b540008cad4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:03:04.591869 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1835947264}
W20260709 13:03:04.595548  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 48042.1us
I20260709 13:03:04.607491  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:04.610726  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:04.613101  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:04.614058  1685 raft_consensus.cc:2410] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 13:03:04.557798  9362 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:04.610920  9366 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:03:04.594928  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 206706us
I20260709 13:03:04.628963  1658 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:04.623828  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:03:04.658473   431 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 1568)
I20260709 13:03:04.656620  9389 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:03:04.639088  1658 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:04.659072  1685 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40124 (request call id 2561) took 1948 ms. Trace:
I20260709 13:03:04.661688  1685 rpcz_store.cc:276] 0709 13:03:02.711024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.716212 (+  5188us) service_pool.cc:224] Handling call
0709 13:03:04.658263 (+1942051us) spinlock_profiling.cc:243] Waited 1.9 s on lock 0x7b540014a854. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:03:04.659013 (+   750us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4361621376}
I20260709 13:03:04.662858  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:04.664677  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:04.663417  1685 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:03:04.673949  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:04.674381  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:04.674762  1689 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
I20260709 13:03:04.675184  1685 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:03:04.625090  9366 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:04.679785  9366 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:04.682126  1584 leader_election.cc:336] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:04.682633  1584 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:04.625485  9124 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:04.685032  1584 consensus_peers.cc:597] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:04.685760  1584 consensus_peers.cc:597] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:04.685669  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 49184.2us
W20260709 13:03:04.686510  1584 consensus_peers.cc:597] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:04.676505  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:04.687026  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:03:04.692005  1584 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:04.698709  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.699927  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:04.702538  9124 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:04.703217  9124 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.703714  9124 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:04.701833  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:04.715965  9352 consensus_queue.cc:1048] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:04.727200  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:04.728838  9414 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:04.729395  9414 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.729997  9414 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:04.747424  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.751783  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:04.661285  9389 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:04.661547  9359 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:04.754174  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:04.759325  1689 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:04.695434  9293 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:03:04.756491  9441 log.cc:927] Time spent T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:03:04.695084  8960 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:04.691756  9299 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.756583  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:04.746414  9214 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:04.746333  9361 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:04.757597  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:04.787901  1612 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:04.622591  9362 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:04.795924  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:04.750329  9201 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:04.750337  9329 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:04.758422  9389 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:03:04.820323  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:04.820775  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:04.821242  2233 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:04.821463  2230 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:03:04.821847  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:04.822118  2229 raft_consensus.cc:1240] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:04.810950  1616 pending_rounds.cc:85] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 14
W20260709 13:03:04.760655  9419 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:03:04.752965  8983 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:04.810797  1690 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 16.
I20260709 13:03:04.828207  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:04.817171  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:04.829697  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:04.823349  1616 pending_rounds.cc:107] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 26.15
I20260709 13:03:04.830430  1616 raft_consensus.cc:2889] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:04.824148  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:04.824363  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:04.825830  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:04.833709  2228 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:04.765554  9302 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:04.835647  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:04.762233  9427 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:04.762650  8960 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.842541  2234 raft_consensus.cc:2410] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 13:03:04.850309  1687 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:03:04.857749  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:04.765384  9214 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:04.853295  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:04.812261  9363 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:04.858060  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:04.858572  2233 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:03:04.873555  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:04.759495  8789 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:04.759574  8974 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.077s
I20260709 13:03:04.850682  9302 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.827216  9447 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:04.897913  2233 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:03:04.861177  8960 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:04.813274  9299 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:04.856065  9361 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:04.860970  9428 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:04.868746  9214 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:04.868320  9201 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:04.870101  9363 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:04.921185  2228 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:03:04.844177  8983 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:04.828243  9329 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:04.902453  9302 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:04.902159  9447 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:04.907511  9427 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:04.937280  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:04.911834  9414 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:04.912673  8876 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:04.958727  2228 raft_consensus.cc:1240] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 21. Current term is 22. Ops: []
I20260709 13:03:04.926208  9363 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:04.974531  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:04.982076  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:04.936915  9447 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:04.920712  9428 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:04.925791  9201 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:04.977527  9450 log.cc:927] Time spent T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 13:03:05.017647  1683 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40014 (request call id 1745) took 1225 ms. Trace:
I20260709 13:03:05.022554  1683 rpcz_store.cc:276] 0709 13:03:03.792625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:03.989082 (+196457us) service_pool.cc:224] Handling call
0709 13:03:05.017352 (+1028270us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54000749d4. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faf02fa 00007fe094ab1b17 00007fe08e02c6d4 00007fe08e02c662 00007fe08e02c591 00007fe08e02c50c 00007fe08e02b7c2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:03:05.017568 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321970560}
I20260709 13:03:05.023139  1684 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 13:03:05.029811  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:05.031454  9437 log.cc:927] Time spent T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 13:03:05.033108  1615 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43398 (request call id 2638) took 1222 ms. Trace:
I20260709 13:03:04.987682  8525 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:05.033828  8525 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:05.033550  1615 rpcz_store.cc:276] 0709 13:03:03.810691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.268486 (+457795us) service_pool.cc:224] Handling call
0709 13:03:04.296212 (+ 27726us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:04.296612 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:03:04.296639 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:04.308037 (+ 11398us) log.cc:844] Serialized 52 byte log entry
0709 13:03:04.552799 (+244762us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:03:04.552856 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.552949 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.032858 (+479909us) raft_consensus.cc:1685] finished
0709 13:03:05.032881 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.033046 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3588,"raft.queue_time_us":1443,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":5298,"thread_start_us":223641,"threads_started":1,"wal-append.queue_time_us":434667}
I20260709 13:03:05.035617  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
W20260709 13:03:05.039292  9443 log.cc:927] Time spent T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:03:05.040745  1613 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1379) took 2292 ms. Trace:
I20260709 13:03:05.042200  9362 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.041143  1613 rpcz_store.cc:276] 0709 13:03:02.748709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.805985 (+ 57276us) service_pool.cc:224] Handling call
0709 13:03:02.806296 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:02.816263 (+  9967us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:02.816305 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:03:04.250024 (+1433719us) spinlock_profiling.cc:243] Waited 964 us on lock 0x7b2c0005b3b8. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08e3e5a11 00007fe08e3f5372 00007fe08fb27a01 00007fe08faec43a 00007fe08faeb83e 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2
0709 13:03:04.256793 (+  6769us) log.cc:844] Serialized 26 byte log entry
0709 13:03:04.560275 (+303482us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:03:04.560336 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.560442 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.040419 (+479977us) raft_consensus.cc:1685] finished
0709 13:03:05.040441 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.040663 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9587,"raft.queue_time_us":118759,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":11226,"spinlock_wait_cycles":2216832,"thread_start_us":288898,"threads_started":1,"wal-append.queue_time_us":405123}
W20260709 13:03:05.043291  9452 log.cc:927] Time spent T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
W20260709 13:03:05.044214  9451 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 13:03:04.965299  9427 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.055068  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:05.079589  1689 raft_consensus.cc:2468] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:03:05.083882  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:05.085736  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:05.086894  1689 raft_consensus.cc:2410] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
W20260709 13:03:05.060168  9358 consensus_meta.cc:287] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.676s	user 0.010s	sys 0.000s
W20260709 13:03:05.049578  9448 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
W20260709 13:03:05.046875  9438 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
I20260709 13:03:05.059706  9299 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.076078  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:05.122242  2228 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:05.114477  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:05.127031  1688 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 25.
W20260709 13:03:05.063515  9296 consensus_meta.cc:287] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.764s	user 0.011s	sys 0.000s
I20260709 13:03:05.127305  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:05.128028  2236 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 22: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:03:05.094244  2232 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 13:03:05.095574  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:05.127718  9296 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.046499  9463 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:05.129856  9001 consensus_meta.cc:287] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.831s	user 0.010s	sys 0.003s
I20260709 13:03:05.129815  9463 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:05.130380  9001 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.131049  9296 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.131958  1685 raft_consensus.cc:1240] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:05.097895  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8343e6d9eb5348b99ed8b5dc38a6087d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.129323  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:05.044286  9329 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:05.069941  9018 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:05.133932  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:05.045066  9453 log.cc:927] Time spent T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:03:05.100574  9201 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:05.066872  9366 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.132011  9463 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:05.135586  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.136730  9302 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:05.098507  9358 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.107612  9427 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:05.054180  9124 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:05.116135  1617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1394) took 1271 ms. Trace:
I20260709 13:03:05.123131  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:05.123545  9340 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:05.125985  8983 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:05.090003  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 162136us
I20260709 13:03:05.090430  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:05.047586  9359 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.133217  1683 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 22: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:05.134027  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:05.138469  2235 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:05.101861  9428 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.136598  9366 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:05.139670  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.143541  9214 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:05.181301  1609 raft_consensus.cc:1240] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:05.144585  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:05.182543  2233 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 23: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:05.146255  1508 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.183600  8983 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:05.187832  8789 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:05.170239  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 187904us
I20260709 13:03:05.195407  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:05.197103  1618 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:05.146816  9329 consensus_queue.cc:1059] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:05.171816  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:03:05.197803  1658 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:05.198012  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 51845.3us
I20260709 13:03:05.198256  1691 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:05.198541  1658 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.151700  1684 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40114 (request call id 3590) took 1345 ms. Trace:
I20260709 13:03:05.199275  1684 rpcz_store.cc:276] 0709 13:03:03.805901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.271929 (+466028us) service_pool.cc:224] Handling call
0709 13:03:05.018168 (+746239us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:03:05.023850 (+  5682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.151414 (+127564us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b54000749d0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08faeb8ad 00007fe094ab0419 00007fe08e027bd4 00007fe08e027b62 00007fe08e027a91 00007fe08e027a0c 00007fe08e026cc2 00007fe08de2748c 00007fe08de268c6 00007fe08de2626f
0709 13:03:05.151613 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1715114880}
I20260709 13:03:05.193764  2228 raft_consensus.cc:2410] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
W20260709 13:03:05.151484  9375 consensus_meta.cc:287] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.001s
I20260709 13:03:05.132441  9001 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:05.154136  9026 consensus_meta.cc:287] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Time spent flushing consensus metadata: real 0.621s	user 0.010s	sys 0.001s
I20260709 13:03:05.154876  9363 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:05.126161  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d7cdb06308e4f21a67e6a685abdc179" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.156181  9389 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:03:05.174384  1688 raft_consensus.cc:1240] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:05.205246  1689 raft_consensus.cc:3055] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:03:05.224637  1689 raft_consensus.cc:740] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
W20260709 13:03:05.224795  2203 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.152987  9447 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:05.198800  1508 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.218940  1690 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:03:05.222774  1658 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.226016  1658 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:05.229691  1613 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 13:03:05.231326  2229 raft_consensus.cc:2410] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 for term 25 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 13:03:05.170871  1617 rpcz_store.cc:276] 0709 13:03:03.844478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.311065 (+466587us) service_pool.cc:224] Handling call
0709 13:03:04.311473 (+   408us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:04.311969 (+   496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.312011 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:04.320555 (+  8544us) log.cc:844] Serialized 52 byte log entry
0709 13:03:04.534939 (+214384us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:03:04.545896 (+ 10957us) log.cc:844] Serialized 14 byte log entry
0709 13:03:04.546225 (+   329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.546325 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.115892 (+569567us) raft_consensus.cc:1685] finished
0709 13:03:05.115915 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.116073 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7468,"prepare.queue_time_us":10671,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":4927,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":5877,"thread_start_us":196099,"threads_started":1,"wal-append.queue_time_us":304015}
I20260709 13:03:05.234200  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.234735  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:05.236258  2232 raft_consensus.cc:1240] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 20. Current term is 21. Ops: []
W20260709 13:03:05.223675  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 46677.7us
I20260709 13:03:05.239130  1688 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:03:05.231832  1583 leader_election.cc:304] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:05.244340  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 45624us
I20260709 13:03:05.249342  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:05.249750  1508 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:05.250546  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:05.251796  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:05.252328  9447 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:05.252812  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:05.245265  9018 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:05.235702  9463 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:05.172873  9359 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:05.253937  9018 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.256167  9018 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.256337  9359 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:05.257249  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.257968  9363 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:05.244706  9293 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.014s
I20260709 13:03:05.259619  9214 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:05.260987  2203 leader_election.cc:336] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.261479  9389 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:05.261668  2203 leader_election.cc:336] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.232185  1689 consensus_queue.cc:260] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:05.262490  1689 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:05.262590  2203 leader_election.cc:336] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:05.264168  9436 log.cc:927] Time spent T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.580s	user 0.000s	sys 0.003s
I20260709 13:03:05.265494  1614 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45356 (request call id 1391) took 1453 ms. Trace:
I20260709 13:03:05.228735  9375 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.266824  1611 raft_consensus.cc:1240] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 26. Current term is 27. Ops: []
I20260709 13:03:05.265899  1614 rpcz_store.cc:276] 0709 13:03:03.812349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.268476 (+456127us) service_pool.cc:224] Handling call
0709 13:03:04.268846 (+   370us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:03:04.269239 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:03:04.269275 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:03:04.296642 (+ 27367us) log.cc:844] Serialized 52 byte log entry
0709 13:03:04.515070 (+218428us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:03:04.515117 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:04.515191 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:03:05.263630 (+748439us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08c887db1 00007fe08c925c85 00007fe08c9266a2 00007fe08f6981db 00007fe08f693aaf 00007fe08f68786e 00007fe08f686bb6 00007fe08f6867d8 00007fe08f690221 00007fe08f6901d9 00007fe08f690169
0709 13:03:05.265175 (+  1545us) raft_consensus.cc:1685] finished
0709 13:03:05.265201 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.265409 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3372,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":3351,"spinlock_wait_cycles":25765248,"thread_start_us":205291,"threads_started":1,"wal-append.queue_time_us":310728}
I20260709 13:03:05.267442  9375 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:05.267695  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:05.268185  2231 raft_consensus.cc:1240] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 20. Current term is 21. Ops: []
I20260709 13:03:05.268800  9329 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:05.268985  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:05.271880  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:05.242165  8789 raft_consensus.cc:3055] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:05.274013  8789 raft_consensus.cc:740] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:05.274741  9299 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:05.274848  8789 consensus_queue.cc:260] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.276209  8789 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:05.227242  9428 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:05.278200  1689 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:03:05.278640  1689 pending_rounds.cc:85] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 10
I20260709 13:03:05.278944  1689 pending_rounds.cc:107] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:03:05.279614  1689 raft_consensus.cc:2889] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:05.226353  9353 consensus_queue.cc:1059] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 13:03:05.282666  9414 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:05.286042  9275 consensus_queue.cc:1059] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:03:05.287494  9329 raft_consensus.cc:3055] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:05.287878  9329 raft_consensus.cc:740] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:05.288774  8983 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:05.288900  9018 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:05.288666  9329 consensus_queue.cc:260] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.289559  9018 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.290660  9329 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:05.292455  9018 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:05.294955  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.300057  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:05.300743  2231 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:05.301635  1616 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:05.261590  9358 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.304643  1691 raft_consensus.cc:2410] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:03:05.307971  2231 raft_consensus.cc:2468] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 24.
I20260709 13:03:05.309376  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.315387  2237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:05.158555  8960 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.312080  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.312428  1618 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
W20260709 13:03:05.238042  1658 leader_election.cc:336] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.313462  2233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.327014  1687 raft_consensus.cc:1240] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 22. Current term is 23. Ops: []
I20260709 13:03:05.313947  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:03:05.327066  1658 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.314672  2228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:03:05.328749  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 91411.8us
W20260709 13:03:05.315225  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 76207.2us
I20260709 13:03:05.289917  9428 raft_consensus.cc:493] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:05.270771  8525 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:05.337663  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:05.316972  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.322422  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:05.340093  1691 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:05.221918  9124 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:05.326700  2232 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
W20260709 13:03:05.333956  1584 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.014s	sys 0.019s
W20260709 13:03:05.348973  1584 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.014s	sys 0.019s
W20260709 13:03:05.349339  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 654894us
W20260709 13:03:05.353257  1509 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:05.331027  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.333990  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:05.275411  9299 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:05.268826  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:05.355008  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.355342  1614 raft_consensus.cc:2393] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 25: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:05.345356  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:05.337196  9358 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:05.358788  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:05.359544  1507 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:05.351627  1584 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:05.360567  1584 leader_election.cc:336] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:05.361145  1584 leader_election.cc:336] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.361523  1584 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.362308  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.363655  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:05.363296  2232 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:03:05.368342  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:05.370132  1507 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.353780  9215 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:05.353945  9435 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:05.381394  2201 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:05.425381  1658 leader_election.cc:336] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.426024  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:05.427675  1658 leader_election.cc:336] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.428198  1658 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.358194  8789 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:05.358374  9359 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:05.340987  9428 raft_consensus.cc:515] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:05.432314  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 70280.9us
W20260709 13:03:05.434710  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:05.435290  2203 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.458478  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:05.370921  9366 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:05.376205  1657 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:03:05.460826  2203 leader_election.cc:336] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.372951  8983 raft_consensus.cc:2804] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:05.370604  9381 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:03:05.465241  2203 leader_election.cc:336] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:05.465721  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 89603.5us
I20260709 13:03:05.430454  9018 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:05.463474  9463 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:05.463665  8983 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.437527  9375 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:05.475276  9358 raft_consensus.cc:2804] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:05.499764  2201 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:05.476527  9431 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:05.460479  9435 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:05.481339  8789 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:05.487866  2203 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.530042  1657 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:03:05.531890  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 44059.8us
W20260709 13:03:05.532546  2203 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.486650  9302 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:05.498183  8983 raft_consensus.cc:3060] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:05.498205  9329 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.547099  1508 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:05.528699  9431 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:03:05.551744  9479 log.cc:927] Time spent T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 13:03:05.565639  9199 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
I20260709 13:03:05.537804  9296 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:05.568536  9296 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.568945  9296 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:05.548774  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 178708us
I20260709 13:03:05.574914  1507 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:05.488564  9299 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.537981  9358 raft_consensus.cc:697] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 24 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:05.543826  9339 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:03:05.586800  1658 leader_election.cc:336] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.591786  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:05.594745  9477 log.cc:927] Time spent T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:03:05.602061  1507 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.576125  9431 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:05.596719  9359 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 13:03:05.659758   431 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 1568)
I20260709 13:03:05.587746  9485 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:03:05.652689  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.601764  9199 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:05.553614  9329 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.600432  8789 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:05.606541  9381 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.651357  8876 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:05.664873  9299 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:05.663141  9435 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.663964  9485 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.682572  9485 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:05.663684  9359 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:03:05.675395  9480 log.cc:927] Time spent T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 13:03:05.672021  8960 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.690599  9359 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:05.674161  8789 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.710357  1507 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:05.675724  9381 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:05.715672  9478 log.cc:927] Time spent T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 13:03:05.716336  9428 leader_election.cc:290] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.716939  8789 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:05.719377  9482 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:03:05.728612  1691 raft_consensus.cc:1240] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:05.729461  9329 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.734731  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.735154  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.735726  2235 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 25.
I20260709 13:03:05.736948  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:05.732492  9486 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:05.693854  9358 consensus_queue.cc:237] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.749581  9456 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:05.750209  9456 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.755492  9296 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.756479  9456 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:05.762326  9381 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.767799  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.775115  1507 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:05.781504  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:05.783591  2229 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 13:03:05.783991  2229 pending_rounds.cc:85] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 15
I20260709 13:03:05.784200  2229 pending_rounds.cc:107] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 13:03:05.770875  8974 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:05.784415  2229 raft_consensus.cc:2889] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:05.784812  8974 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:05.771230  8876 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 13:03:05.779621  9381 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:03:05.809652  9381 consensus_queue.cc:1243] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer 984c9089d9804652846f3caeb609d56e log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:03:05.811861  8983 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:05.814581  9431 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.815759  2230 raft_consensus.cc:1240] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:05.797588  8974 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:05.817142  9431 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:05.798520  9264 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:03:05.843115  9471 log.cc:927] Time spent T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.598s	user 0.004s	sys 0.000s
I20260709 13:03:05.854266  9296 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.860350  9485 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.860859  8974 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.882076  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:05.861936  8983 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:05.864193  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 613456us
W20260709 13:03:05.892542  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 105072us
I20260709 13:03:05.893973  9361 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.894114  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:05.894773  1684 raft_consensus.cc:3060] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:05.894621  9361 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:05.895426  9214 raft_consensus.cc:2749] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:05.864936  9379 consensus_queue.cc:1059] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:03:05.896988  9361 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:05.897295  9379 raft_consensus.cc:3055] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:05.897765  9379 raft_consensus.cc:740] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:05.898553  9379 consensus_queue.cc:260] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.881036  9447 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.899801  9379 raft_consensus.cc:3060] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:05.900197  9447 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:05.902276  1688 raft_consensus.cc:1275] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:05.902532  9447 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:05.885044  9428 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:05.902603  1684 raft_consensus.cc:2468] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:03:05.903321  9428 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.910626  1509 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:05.913242  9492 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:05.872501  9353 raft_consensus.cc:3055] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:03:05.913626  9264 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.897622  9358 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:03:05.914453  9264 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:05.875360  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 514173us
W20260709 13:03:05.917464  1584 leader_election.cc:336] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:05.918525  1584 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:05.918617  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.876438  9349 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:05.886461  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 654663us
I20260709 13:03:05.923332  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:05.924495  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.923406  8974 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.925882  1583 leader_election.cc:304] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.926302  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:05.927809  9272 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:03:05.927842  9349 raft_consensus.cc:2749] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:05.928762  9272 consensus_queue.cc:1059] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:05.929838  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:05.930258  9272 raft_consensus.cc:3055] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:05.930799  9272 raft_consensus.cc:740] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:05.931521  9428 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.874396  1686 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40014 (request call id 1751) took 1275 ms. Trace:
I20260709 13:03:05.894965  9389 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.914109  9353 raft_consensus.cc:740] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:05.932457  9272 consensus_queue.cc:260] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.934234  9353 consensus_queue.cc:260] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.935200  9272 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:05.935691  9353 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:05.925906  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:05.938354  1687 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:05.932678  1686 rpcz_store.cc:276] 0709 13:03:04.599228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:04.599388 (+   160us) service_pool.cc:224] Handling call
0709 13:03:04.599763 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:03:04.600007 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 13:03:04.761537 (+161530us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.570847 (+809310us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b180013b460. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c9e1e07 00007fe09583cab9 00007fe08c9fe69b 00007fe08c9fc93f 00007fe08c9fec02 00007fe08f68620b 00007fe08f68bd23 00007fe08f68c23a 00007fe08faf6a2a
0709 13:03:05.662297 (+ 91450us) log.cc:844] Serialized 14 byte log entry
0709 13:03:05.873691 (+211394us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b180013b460. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08c979c0a 00007fe0960b9219 00007fe08ca01a40 00007fe08c9fc4e4 00007fe08c9fc455 00007fe08fae4060 00007fe08faf6716 00007fe08fb07287 00007fe08fb07208
0709 13:03:05.873898 (+   207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:03:05.873967 (+    69us) raft_consensus.cc:1645] Marking committed up to 18
0709 13:03:05.874032 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:03:05.874159 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:03:05.874314 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":204313,"prepare.queue_time_us":99407,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":95,"raft.queue_time_us":213576,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":6535,"spinlock_wait_cycles":58398464,"thread_start_us":228655,"threads_started":2,"wal-append.queue_time_us":941161}
I20260709 13:03:05.890280  9485 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.946756  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:05.933192  9389 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:05.948006  9426 consensus_queue.cc:1059] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:03:05.966781  9426 raft_consensus.cc:3055] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:05.967166  9426 raft_consensus.cc:740] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:05.851855  9359 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:05.947389  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55442.5us
I20260709 13:03:05.970585  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:05.970829  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:05.971166  2235 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:05.971378  2229 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:03:05.973698  1658 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.974202  1658 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:05.977377  9264 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.968070  9426 consensus_queue.cc:260] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:05.949558  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:05.978519  2229 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 26.
W20260709 13:03:05.978767  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 51162.7us
I20260709 13:03:05.979962  2235 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:03:05.948390  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:05.981638  9124 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:03:05.978636  9400 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
I20260709 13:03:05.950297  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:05.950410  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:05.983031  9400 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:05.951820  1692 pending_rounds.cc:85] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 9
I20260709 13:03:05.983587  9400 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:05.984100  1692 pending_rounds.cc:107] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 13:03:05.984589  1692 raft_consensus.cc:2889] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:05.952174  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:05.952541  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:05.987285  1691 raft_consensus.cc:2410] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 10.
I20260709 13:03:05.961696  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:05.955833  9389 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:05.991477  1658 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.992503  9299 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:05.992708  1658 leader_election.cc:336] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:05.994693  9485 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:05.995354  9485 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:05.996134  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 77635.6us
I20260709 13:03:05.998087  9485 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:05.998625  9426 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:05.998718  9485 consensus_queue.cc:1059] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 13:03:06.000550  8974 raft_consensus.cc:3055] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:06.001026  8974 raft_consensus.cc:740] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:06.002095  8974 consensus_queue.cc:260] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 23.20, Last appended by leader: 20, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.003084  8974 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:06.003810  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 53600.4us
I20260709 13:03:06.004585  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:06.006034  2203 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:06.006958  9359 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:06.009977  9460 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:06.010779  9460 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:06.012140  9427 raft_consensus.cc:2749] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:06.012702  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:06.013408  9264 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.013921  9264 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.014545  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:06.016278  9264 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:05.988978  9400 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.018021  9400 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:06.018929  9124 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:06.019734  1690 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:03:06.021935  1509 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:05.947125  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:06.023768  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.024421  1686 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:06.025621  8983 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:06.026608  8983 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:06.027451  8983 consensus_queue.cc:1059] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:06.028836  9485 raft_consensus.cc:3055] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 13:03:06.029254  9485 raft_consensus.cc:740] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:06.030503  9485 consensus_queue.cc:260] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:03:06.031901  9485 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:03:06.034598  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:06.035168  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:06.035436  2235 raft_consensus.cc:2393] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:06.038522  1687 raft_consensus.cc:2468] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:03:05.993299  1658 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:06.045498  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:06.046294  1584 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.047995  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:06.050851  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 58187.3us
W20260709 13:03:06.051594  1658 leader_election.cc:336] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:06.052767  1658 leader_election.cc:336] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.049360  9214 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:06.054299  9214 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.052039  9468 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:06.052371  9361 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:06.062664  1434 catalog_manager.cc:5697] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.052515  9124 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:06.065667  9124 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:06.065768  1434 catalog_manager.cc:5697] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.053306  1658 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:06.079114  1658 leader_election.cc:336] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.075687  9214 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:06.078362  1434 catalog_manager.cc:5697] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 19 to 24. New cstate: current_term: 24 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:06.080790  1686 raft_consensus.cc:2410] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
W20260709 13:03:06.073292  1440 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55374 (request call id 324) took 65499 ms (client timeout 15000 ms). Trace:
I20260709 13:03:06.080633  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:06.081781  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:06.079603  1658 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.083766  2229 raft_consensus.cc:1240] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Rejecting Update request from peer 8a6a5ae1b7f240408b3ddd30d8a18262 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 13:03:06.084506  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.085711  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:06.086170  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:06.085985  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.087273  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:06.087705  1618 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 22: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:06.087890  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.088498  1615 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 25.
I20260709 13:03:06.088635  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.088766  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:06.090073  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:06.050549  1509 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:03:06.083729  1658 leader_election.cc:336] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.093508  1658 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:06.094658  1658 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.094982  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:06.096206  1613 raft_consensus.cc:2410] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 22 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 13:03:06.077879  1508 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:03:06.095263  1658 leader_election.cc:336] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:06.105866  1658 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.083685  1434 catalog_manager.cc:5697] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 18 to 21, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 21 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:06.106765  1658 leader_election.cc:336] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.107980  1434 catalog_manager.cc:5697] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 16 to 23, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 23 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:06.109573  1434 catalog_manager.cc:5697] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 14 to 17, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 17 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.111405  1434 catalog_manager.cc:5697] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 19 to 23, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 23 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:03:06.111683  2203 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.112748  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e4d8a1326e854e51b5539e7addc91fe5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:06.113282  2203 leader_election.cc:336] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.113999  1616 raft_consensus.cc:1217] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Deduplicated request from leader. Original: 15.9->[17.10-19.11]   Dedup: 19.11->[]
I20260709 13:03:06.113926  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:06.114683  1616 raft_consensus.cc:1240] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 13:03:06.114347  2203 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:06.115643  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.116093  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.115304  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.117234  1609 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 20: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:03:06.118117  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.112771  1434 catalog_manager.cc:5697] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 15 to 23, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 23 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:06.119335  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:06.116295  1616 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:03:06.119956  1434 catalog_manager.cc:5697] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.117245  1614 raft_consensus.cc:2410] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 24 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 13:03:06.120502  1583 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.120597  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:06.123611  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:06.107559  1658 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.126576  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:06.126969  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:06.127014  1658 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.124851  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb767ced66f24d579d3a9c116606ef2f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.097182  1508 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:06.129316  1582 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:06.129369  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:06.130424  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.130853  1616 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:06.129608  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:06.128167  1584 leader_election.cc:336] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.132153  1584 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:06.134246  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.128192  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.133352  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:06.121552  1434 catalog_manager.cc:5697] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.135354  1615 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:06.095906  9361 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:06.096287  9358 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:06.096786  9460 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:06.136240  1434 catalog_manager.cc:5697] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 19 to 23, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 23 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:06.097569  9296 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:06.101858  8974 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:06.137516  1609 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 24. Current term is 25. Ops: []
W20260709 13:03:06.081998  1440 rpcz_store.cc:269] 0709 13:02:00.574132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:00.574389 (+   257us) service_pool.cc:224] Handling call
0709 13:03:06.073152 (+65498763us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:01.954042 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:01.981002 (+ 26960us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:01.981054 (+    52us) write_op.cc:435] Acquired schema lock
0709 13:03:01.981075 (+    21us) tablet.cc:662] Decoding operations
0709 13:03:01.981936 (+   861us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:01.982078 (+   142us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:01.982107 (+    29us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:03:01.982724 (+   617us) tablet.cc:701] Row locks acquired
0709 13:03:01.982770 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:03:01.982930 (+   160us) write_op.cc:270] Start()
0709 13:03:01.983085 (+   155us) write_op.cc:276] Timestamp: P: 1783602181982868 usec, L: 0
0709 13:03:01.983107 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:03:02.127701 (+144594us) log.cc:844] Serialized 2477 byte log entry
0709 13:03:03.239519 (+1111818us) op_driver.cc:464] REPLICATION: finished
0709 13:03:03.347896 (+108377us) write_op.cc:301] APPLY: starting
0709 13:03:03.348022 (+   126us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:03.528743 (+180721us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:03.528860 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:03.581190 (+ 52330us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:03.605830 (+ 24640us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:03:03.611623 (+  5793us) write_op.cc:312] APPLY: finished
0709 13:03:03.628196 (+ 16573us) log.cc:844] Serialized 65 byte log entry
0709 13:03:03.628405 (+   209us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:03.628532 (+   127us) write_op.cc:454] Released schema lock
0709 13:03:03.628588 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:03.629063 (+   475us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:03.629098 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":610018,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":108022,"apply.run_cpu_time_us":13023,"apply.run_wall_time_us":281160,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29243,"num_ops":6,"prepare.queue_time_us":479070,"prepare.run_cpu_time_us":20087,"prepare.run_wall_time_us":606390,"raft.queue_time_us":501584,"raft.run_cpu_time_us":8696,"raft.run_wall_time_us":111649,"replication_time_us":1256217,"thread_start_us":691144,"threads_started":4,"wal-append.queue_time_us":522492,"wal-append.run_cpu_time_us":39758,"wal-append.run_wall_time_us":1947572}]]}
I20260709 13:03:06.098315  9379 consensus_queue.cc:1059] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 13:03:06.123747  9389 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.126394  8960 raft_consensus.cc:493] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:06.117892  9359 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:03:06.153494  8974 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:06.137061  9381 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:03:06.138800  9352 consensus_queue.cc:1059] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 13:03:06.124060  9392 consensus_queue.cc:1059] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:03:06.136559  9463 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:06.097759  9264 consensus_queue.cc:1059] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 13:03:06.150050  1434 catalog_manager.cc:5697] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 20 to 24, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 24 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.191287  1434 catalog_manager.cc:5697] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 18 to 23, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 23 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.141356  9428 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:06.171367  1616 raft_consensus.cc:2468] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 24.
I20260709 13:03:06.148922  9124 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:06.137753  1658 leader_election.cc:336] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:06.173451  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 60238.2us
I20260709 13:03:06.143066  9414 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:06.175024  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 45775us
I20260709 13:03:06.183022  9001 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.195467  1434 catalog_manager.cc:5697] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 16 to 20, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:06.212077  1434 catalog_manager.cc:5697] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.143489  9302 consensus_queue.cc:1048] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 13:03:06.195284  1658 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:06.214843  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 86693.1us
W20260709 13:03:06.209738  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 67956us
W20260709 13:03:06.215659  1584 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.215587  1434 catalog_manager.cc:5697] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:06.177659  9201 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:06.216496  1584 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:06.222355  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 84652.9us
I20260709 13:03:06.208357  1615 raft_consensus.cc:2468] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 24.
I20260709 13:03:06.178140  8960 raft_consensus.cc:515] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.220103  1434 catalog_manager.cc:5697] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 16 to 20, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:06.254971  2229 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:03:06.254647  1434 catalog_manager.cc:5697] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 19 to 20, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:06.216950  9001 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.229599  8960 leader_election.cc:290] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:06.230602  9264 raft_consensus.cc:3055] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:06.212430  9379 raft_consensus.cc:2804] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:06.238374  9381 raft_consensus.cc:697] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:03:06.233407  9497 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:03:06.243458  9299 raft_consensus.cc:3055] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:06.174670  9389 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:06.218421  8974 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:06.256498  1434 catalog_manager.cc:5697] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 15 to 20, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 20 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.297731  9001 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:06.297958  1434 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 18 to 20, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 20 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:06.261819  1610 raft_consensus.cc:2468] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
W20260709 13:03:06.267772  2201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.300135  1434 catalog_manager.cc:5697] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 15 to 22. New cstate: current_term: 22 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:06.225903  9124 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:06.216982  9359 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 26 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:06.198189  9463 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:06.292685  9379 raft_consensus.cc:697] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:06.255643  9414 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:06.303237  1434 catalog_manager.cc:5697] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.258292  9358 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:06.304876  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:06.261775  9264 raft_consensus.cc:740] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:06.288491  8960 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:03:06.315945  1434 catalog_manager.cc:5697] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.323282  9379 consensus_queue.cc:237] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:06.324579  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 128490us
I20260709 13:03:06.332528  9463 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:06.339061  9363 raft_consensus.cc:3055] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:03:06.291456  9299 raft_consensus.cc:740] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:06.344587  9389 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:06.345466  9381 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.265750  9492 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 13:03:06.346772  9359 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.351363  2232 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 18.
I20260709 13:03:06.361753  1434 catalog_manager.cc:5697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 17 to 19, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 19 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:06.364324  1434 catalog_manager.cc:5697] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.366482  1434 catalog_manager.cc:5697] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 17 to 22, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 22 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:03:06.367722  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 152041us
I20260709 13:03:06.368582  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.368499  1434 catalog_manager.cc:5697] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:06.369385  2229 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 13:03:06.370502  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 154864us
W20260709 13:03:06.384811  1584 leader_election.cc:336] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:06.385708  1584 leader_election.cc:336] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:06.386548  1584 leader_election.cc:336] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.386891  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:06.389479  1584 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.390345  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:06.358210  9363 raft_consensus.cc:740] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:06.354251  9358 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.391353  8960 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:06.394289  9389 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:06.406364  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 203358us
W20260709 13:03:06.408273  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:06.412891  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.389889  1584 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:06.414746  1584 consensus_peers.cc:597] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:06.396772  9463 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
I20260709 13:03:06.433722  1582 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:06.418465  9414 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 13:03:06.439109  1584 leader_election.cc:336] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:06.448130  1584 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.435252  9389 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 13:03:06.449201  1584 leader_election.cc:336] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:06.496498  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 47309.4us
W20260709 13:03:06.500253  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 276709us
W20260709 13:03:06.503197  1656 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.508493  1507 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.416405  9264 consensus_queue.cc:260] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:06.519773  1658 leader_election.cc:304] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.530103  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.498687  9509 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:06.513849  1584 consensus_peers.cc:597] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:06.514060  1507 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:06.515506  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 81880.8us
I20260709 13:03:06.545142  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:06.545585  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:06.547399  1658 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.551637  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.552130  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:06.552393  1507 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:06.552861  1610 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:06.551076  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:06.556770  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.557006  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:06.557601  1617 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:06.440589  9363 consensus_queue.cc:260] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.559891  1616 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
W20260709 13:03:06.561564  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 47773.9us
I20260709 13:03:06.561728  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.517204  9492 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:06.563498  1616 pending_rounds.cc:85] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 14
I20260709 13:03:06.567339  1616 pending_rounds.cc:107] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 13:03:06.567440  1609 raft_consensus.cc:1240] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 13:03:06.567888  1616 raft_consensus.cc:2889] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:06.566817  1509 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:06.565384  1610 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 26.
W20260709 13:03:06.571058  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet a35934c9d0d04fbeac6fa3a58fa8fd63: Illegal state: Replica 8a6a5ae1b7f240408b3ddd30d8a18262 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:06.537928  9505 raft_consensus.cc:2804] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:06.602550  9505 raft_consensus.cc:697] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:06.603495  9505 consensus_queue.cc:237] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:06.539934  9340 raft_consensus.cc:2749] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:06.558561  9124 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:06.606058  9124 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:06.606515  9124 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:06.609072  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.558403  9486 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:06.569264  9434 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:03:06.613531  1615 raft_consensus.cc:1275] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 18. (term mismatch)
I20260709 13:03:06.613945  1615 pending_rounds.cc:85] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 17
I20260709 13:03:06.614140  1615 pending_rounds.cc:107] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 23.18
I20260709 13:03:06.614348  1615 raft_consensus.cc:2889] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:06.615655  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:06.617102  1584 leader_election.cc:336] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.551017  9509 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.619099  9509 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:06.564286  9264 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:03:06.625600  1584 leader_election.cc:336] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:06.626571  1584 leader_election.cc:336] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.626967  1584 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:06.614779  1617 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 26.
I20260709 13:03:06.628454  9299 consensus_queue.cc:260] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:06.575417  9502 log.cc:927] Time spent T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 13:03:06.631146  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:06.561142  9444 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:06.632328  9349 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:06.576526  1658 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.577919  1609 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:06.582077  8983 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.633828  8983 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:06.552913  9296 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:03:06.562788  8974 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:06.585683  9392 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:03:06.573764  9361 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:06.586849  9272 raft_consensus.cc:3055] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:03:06.587181  1509 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:06.596230  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:06.599534  9358 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:06.549783  9352 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:06.636219  9340 raft_consensus.cc:3060] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:06.697892  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 71338us
W20260709 13:03:06.698815  1584 leader_election.cc:336] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.699224  1584 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:06.643042  9299 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:06.700227  9349 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 13:03:06.699945  9124 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:06.661773   431 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 1568)
I20260709 13:03:06.635632  1686 raft_consensus.cc:1240] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Rejecting Update request from peer cecad4f86a114a7696e398b3e8cc511e for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:06.706260  9363 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:06.706914  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet 49db1a2fe0dd40eba2ac450e306a6937: Illegal state: Replica 984c9089d9804652846f3caeb609d56e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:06.708756  9329 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:06.710687  1683 raft_consensus.cc:2393] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 23: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:03:06.711467  9272 raft_consensus.cc:740] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:06.712553  9272 consensus_queue.cc:260] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:06.635558  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:06.714358  1584 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:06.636641  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 60198.1us
I20260709 13:03:06.714876  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:06.708910  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 121800us
I20260709 13:03:06.746261  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.738516  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:06.747500  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:06.748651  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:06.634266  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a1244fbe0514ba59a0164883a315bbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:06.743507  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:06.750170  1684 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 25.
I20260709 13:03:06.748381  2236 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 24.
I20260709 13:03:06.716892  1691 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:03:06.764460  1584 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.765141  1584 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:06.766129  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:03:06.767197  1584 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:06.727833  8983 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:06.780603  1582 leader_election.cc:304] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.771106  1509 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.736773  9358 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:06.782014  1582 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:06.778644  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 210371us
I20260709 13:03:06.789067  2203 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:03:06.790175  9201 raft_consensus.cc:2804] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:06.765079  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:06.790899  9201 raft_consensus.cc:697] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:06.791536  9361 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:06.792088  9201 consensus_queue.cc:237] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.793375  1658 leader_election.cc:304] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.794540  9463 raft_consensus.cc:2749] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:06.797641  9435 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:03:06.752107  9414 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:06.753826  9296 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:06.803567  1658 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:06.754029  9329 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:06.765209  8974 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:06.768252  9392 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:06.769738  9507 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:06.769917  9447 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:03:06.818349  9214 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:06.737586  9352 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.801878  9510 raft_consensus.cc:2749] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:06.799271  9435 raft_consensus.cc:3055] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:06.780153  9349 raft_consensus.cc:2764] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 13:03:06.780205  9400 raft_consensus.cc:3060] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:06.819274  9272 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:06.852016  9296 raft_consensus.cc:2804] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:06.855716  9468 consensus_queue.cc:1059] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.019s
I20260709 13:03:06.817724  9358 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:06.818240  9463 consensus_queue.cc:1059] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 13:03:06.876015  9435 raft_consensus.cc:740] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:06.852788  9329 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:06.820201  9414 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:06.904676  9520 log.cc:927] Time spent T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:03:06.906332  9414 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:06.804555  1508 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:03:06.953344  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 149270us
I20260709 13:03:06.801916  1509 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
W20260709 13:03:06.973613  1584 leader_election.cc:336] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:06.974682  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 172733us
I20260709 13:03:06.999112  9531 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:03:07.001590  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet 63835fd2de1c4dd4b0cea71fe7f09ed5: Illegal state: Replica 984c9089d9804652846f3caeb609d56e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:07.002723  9531 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.003432  9531 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:06.853936  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:07.014245  2229 raft_consensus.cc:2393] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 23: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:06.890452  9296 raft_consensus.cc:697] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 24 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:06.779667  8789 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 13:03:06.807238  8983 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:07.025336  9392 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.036267  9392 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:07.026705  9352 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:07.041066  9392 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:07.050817  9359 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 13:03:07.051918  1507 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:06.879304  9124 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:07.069736  9435 consensus_queue.cc:260] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.074950  9296 consensus_queue.cc:237] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.086114  9359 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 26 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:07.088171  9359 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:07.113404  9329 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:07.116051  9358 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:07.118930  9447 raft_consensus.cc:3055] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:03:07.119626  9447 raft_consensus.cc:740] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:07.124539  9447 consensus_queue.cc:260] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:07.128885  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 68530.6us
W20260709 13:03:07.119671  2203 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:07.147010  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 343384us
I20260709 13:03:07.148746  1658 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:07.152093  9214 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:07.159675  9428 raft_consensus.cc:2804] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:07.159976  9296 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.160707  9428 raft_consensus.cc:697] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 24 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:07.161103  9296 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.162313  9428 consensus_queue.cc:237] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:07.165529  2203 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:07.166527  9296 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:07.166445  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:07.168973  9435 raft_consensus.cc:3060] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:07.169726  9460 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:07.179822  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 169389us
I20260709 13:03:07.185246  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:07.186386  2232 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
I20260709 13:03:07.159888  9358 consensus_queue.cc:1048] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.030s
I20260709 13:03:07.190412  9447 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:07.215318  9299 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:07.216363  9299 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.219547  9299 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:07.222939  9400 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:07.177904  9340 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:07.195385  9359 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.229717  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:07.229915  9359 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.230654  1688 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:07.249909  1692 raft_consensus.cc:1240] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Rejecting Update request from peer 83f6f5810950417195f707ff308a8125 for earlier term 17. Current term is 18. Ops: []
I20260709 13:03:07.255422  9533 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:07.256538  9533 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.259896  9533 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:07.268455  1691 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:07.269078  1686 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 13. (term mismatch)
I20260709 13:03:07.269538  1686 pending_rounds.cc:85] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 12
I20260709 13:03:07.269785  1686 pending_rounds.cc:107] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:03:07.270057  1686 raft_consensus.cc:2889] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:03:07.270152  1584 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:07.270920  1584 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:07.272563  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.273787  9359 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:07.274397  9359 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.275045  9359 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:03:07.273327  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.277819  9531 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.278716  1691 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:07.279027  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.279924  1686 raft_consensus.cc:2393] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:07.280267  9531 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:07.278358  1692 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:03:07.285413  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:07.286477  1611 raft_consensus.cc:2393] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 24: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:03:07.292855  9379 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.293823  9379 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:07.298784  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet 034eef09c5fd47c9b010158a104b4028: Illegal state: Replica 8a6a5ae1b7f240408b3ddd30d8a18262 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:07.272306  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:07.292831  2203 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:07.317973  1690 raft_consensus.cc:1275] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:03:07.322625  1609 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:07.323441  1689 raft_consensus.cc:1275] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:03:07.333393  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:07.309896  2203 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:07.339583  1508 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:03:07.345352  2203 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:07.343116  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.349577  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:07.350482  1609 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 20.
I20260709 13:03:07.348171  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:07.351564  2232 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 21.
I20260709 13:03:07.359295  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:07.361831  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:07.362730  1617 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 27.
I20260709 13:03:07.376421  8974 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:07.378487  1509 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:03:07.379693  9485 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:07.380105  9485 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.380463  9485 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:07.386472  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:07.387323  2236 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:03:07.389519  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:07.390239  2230 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:03:07.392402  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 43619.9us
I20260709 13:03:07.347985  9359 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:07.399590  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:07.400955  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:07.405218  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.407202  1688 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 26.
I20260709 13:03:07.407255  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.409298  2232 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 13:03:07.401997  1584 consensus_peers.cc:597] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:07.407743  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.412346  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:07.412539  1689 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:03:07.412609  2236 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:03:07.413008  1685 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:03:07.410933  1690 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 26.
I20260709 13:03:07.414656  9381 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.415386  9381 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.402576  9379 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:07.422181  2230 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 25.
I20260709 13:03:07.424669  1685 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 25.
I20260709 13:03:07.425915  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:07.425026  9272 consensus_queue.cc:1048] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:03:07.425869  9293 consensus_queue.cc:1048] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 13:03:07.433040  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:07.433913  2235 raft_consensus.cc:2393] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 21: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:07.437244  9359 consensus_queue.cc:1059] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:07.438759  9531 raft_consensus.cc:3055] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:03:07.437026  9485 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.439250  9531 raft_consensus.cc:740] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:07.440366  9531 consensus_queue.cc:260] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:07.442155  9531 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:07.449070  9485 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:07.451306  1584 consensus_peers.cc:597] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:07.455653  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:07.463128  9381 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:07.466445  1507 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:07.469929  1685 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:03:07.470800  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:07.471043  1507 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:07.472218  9359 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:07.473367  9359 raft_consensus.cc:3060] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:07.473284  9299 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:07.468281  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:07.476224  1612 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:03:07.478407  1611 raft_consensus.cc:1240] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 13:03:07.480190  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:07.482178  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:07.483268  9533 raft_consensus.cc:3060] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:07.483901  1584 leader_election.cc:336] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:07.486002  1657 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:03:07.488569  1509 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:03:07.489521  8974 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:03:07.489737  1617 raft_consensus.cc:2468] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:03:07.490053  8974 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.482170  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:07.490741  9460 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.494765  1508 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:07.498723  1616 raft_consensus.cc:1275] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:03:07.468640  9485 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:03:07.502022  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:07.510138  9460 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.510690  1583 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:07.512637  1583 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:07.513509  8974 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:03:07.519434  1508 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:07.520454  9492 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:07.489688  1507 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:07.519098  2203 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:07.516781  1611 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for term 23 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 13:03:07.522353  2203 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:07.521065  9492 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 25 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:07.525897  9492 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.513849  9379 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:07.515460  9293 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:07.513386  9460 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:07.513706  9363 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:03:07.518553  9329 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:07.536311  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 46697.3us
I20260709 13:03:07.538168  1507 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:07.541483  1658 leader_election.cc:336] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:07.519935  9531 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 13:03:07.548245  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:07.550884  1658 leader_election.cc:336] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:07.551486  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:07.553623  9428 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:07.553402  9296 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:07.519142  9264 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:03:07.546041  9381 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:03:07.560900  1658 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:07.547384  9299 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:07.566639  1611 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 13:03:07.562844  9531 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.568485  9531 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:07.570248  8974 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:03:07.571189  9531 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:07.579689  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:07.584875  9296 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:07.585472  9296 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:07.590368  9296 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:07.597556  9201 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:07.604727  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:07.604676  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:07.606274  2236 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:03:07.606657  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:07.607242  2230 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:07.602953  9299 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:07.610219  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 49384.9us
I20260709 13:03:07.605273  1617 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:03:07.603458  9359 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:07.618017  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.620754  1685 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:03:07.627708  1658 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:07.629348  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:07.631039  1612 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:03:07.621876  8974 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 28 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:07.639886  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.639954  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:07.641598  1691 raft_consensus.cc:2410] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:03:07.641628  1686 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Advancing to term 28
I20260709 13:03:07.645557  9428 raft_consensus.cc:2804] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:07.648924  9428 raft_consensus.cc:697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:07.651615  1686 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 28.
I20260709 13:03:07.651898  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.617499  9299 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:07.652525  1687 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:03:07.632347  9531 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.665625  1507 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:07.668035  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:07.669121  1610 raft_consensus.cc:2410] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 24 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 13:03:07.668953  1687 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:03:07.671566  1507 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:07.671964  1611 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:03:07.658705  9363 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.673767  9359 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 13:03:07.674436  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:07.692518  9359 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 28 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:07.693765  9359 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:03:07.697975  9329 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:07.698446  9329 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.698613  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:07.699306  1611 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:03:07.698822  9329 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:07.709681   431 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 1568)
I20260709 13:03:07.711596  9381 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:07.658187  9530 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:07.742008  9533 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:07.674341  9531 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.742848  9264 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:07.743549  9264 consensus_queue.cc:1243] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [LEADER]: Peer 8a6a5ae1b7f240408b3ddd30d8a18262 log is divergent from this leader: its last log entry 23.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:03:07.744736  9531 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:07.739061  1617 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:03:07.746752  2230 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
W20260709 13:03:07.690047  1658 leader_election.cc:336] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:07.744607  9379 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:07.747565  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:07.650024  9428 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:07.769122  9340 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:07.746304  9363 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:07.770862  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 80851.9us
W20260709 13:03:07.771984  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 74141.1us
I20260709 13:03:07.773358  1609 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:07.757220  9299 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:07.781584  1658 leader_election.cc:336] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:07.784214  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:07.742100  9530 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.773398  9264 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 13:03:07.793318  9340 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:07.800205  9293 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.820783  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:07.820258  1658 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:07.852137  9363 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:07.833200  9435 consensus_queue.cc:1048] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 13:03:07.827783  9293 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.856415  9293 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:07.860695  1611 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 28.
I20260709 13:03:07.862787  1609 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
W20260709 13:03:07.865618  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 53224.4us
I20260709 13:03:07.860047  2229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:07.871670  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:07.873093  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:07.872701  2236 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
W20260709 13:03:07.880352  9552 log.cc:927] Time spent T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:03:07.887713  1509 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:07.898200  9426 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.898897  9426 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.900089  1508 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:07.901600  9561 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:07.902180  9530 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:07.901247  9531 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.902802  9509 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.069s
I20260709 13:03:07.903152  9531 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.903638  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.905081  9460 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:07.898245  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.910349  1688 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:03:07.910519  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.911314  1691 raft_consensus.cc:2393] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 21: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:03:07.912636  9296 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.913551  9561 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:07.914748  9561 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 20.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.916200  9296 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:07.871626  9492 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:07.891924  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:07.924968  8974 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:07.927940  9214 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:07.928835  9214 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.930236  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:07.931080  1686 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:03:07.901204  9426 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:07.906090  9460 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:07.960880  2201 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:07.873340  9329 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:07.929411  9214 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:07.960171  9460 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:07.969132  9296 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:08.016235  9296 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.018508  9296 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.019049  9492 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.022475  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:08.024412  9531 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.029004  9329 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:08.037581  2203 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:08.040853  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 72301.3us
W20260709 13:03:08.056869  2203 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:07.967306  9530 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:08.068068  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:08.069272  2231 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 24: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:08.069499  9444 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:08.072772  9400 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.074153  9530 raft_consensus.cc:493] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.081710  9400 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:08.084951  9529 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:03:08.086315  9400 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.102929  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.106272  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:08.115610  1658 leader_election.cc:336] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:08.123297  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:08.082607  9530 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:08.082191  9492 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.132458  9426 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.133208  9426 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.140627  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:08.105571  9460 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.144541  9460 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.147421  2234 raft_consensus.cc:1275] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 18. (term mismatch)
I20260709 13:03:08.147898  2234 pending_rounds.cc:85] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Aborting all ops after (but not including) 17
I20260709 13:03:08.148149  2234 pending_rounds.cc:107] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: Aborting uncommitted NO_OP operation due to leader change: 23.18
I20260709 13:03:08.148406  2234 raft_consensus.cc:2889] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:08.148602  1617 raft_consensus.cc:1275] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 13:03:08.082711  9444 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.160548  9444 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.169938  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:08.171188  1688 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:08.174041  1508 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 205657us
I20260709 13:03:08.196170  1615 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:03:08.203555  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.209630  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.210573  1610 raft_consensus.cc:2410] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 for term 23 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 13:03:08.213079  1611 pending_rounds.cc:85] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 14
I20260709 13:03:08.219682  1611 pending_rounds.cc:107] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 13:03:08.220034  1611 raft_consensus.cc:2889] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:08.218643  1609 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:03:08.230899  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:08.231345  9264 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.231667  1614 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:08.232000  9264 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.227064  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.238765  1683 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 25.
I20260709 13:03:08.243789  9296 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:08.244618  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 78741.4us
I20260709 13:03:08.245630  9340 consensus_queue.cc:1048] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:03:08.247148  9340 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.247756  9340 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.256083  1688 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 24.
W20260709 13:03:08.257560  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 141974us
W20260709 13:03:08.258487  1658 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:08.259004  1658 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:08.262274  1658 consensus_peers.cc:597] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:08.258780  2203 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 157 similar messages]
I20260709 13:03:08.269333  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.274775  9400 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.275408  9400 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:08.276341  9214 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.267907  9296 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:08.303990  9565 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.304633  9565 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:08.306928  9565 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.278172  9264 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.309551  9579 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.310216  9579 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.310843  9340 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.277396  9400 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.312573  9579 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.314787  9426 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.328590  9428 consensus_queue.cc:1048] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:08.337011  9530 raft_consensus.cc:515] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:08.345615  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 81556.6us
I20260709 13:03:08.347924  9509 raft_consensus.cc:493] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.348529  9509 raft_consensus.cc:515] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.283296  1582 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:08.355826  1658 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:08.368218  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.368911  1686 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:03:08.369697  2236 raft_consensus.cc:2368] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 23. Current term is 24.
I20260709 13:03:08.370210  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.370474  1683 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:03:08.370857  1688 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:03:08.372013  9264 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:08.372303  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:08.373108  1684 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 24: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:08.372674  9264 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:08.373617  9272 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.374392  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:08.374186  9272 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:08.374989  2234 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:03:08.375053  9264 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.376335  9272 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.376878  1507 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:08.380795  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 97628.5us
W20260709 13:03:08.381314  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 119967us
I20260709 13:03:08.366612  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.385656  1614 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 24.
I20260709 13:03:08.384161  1616 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
I20260709 13:03:08.394582  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.406642  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.379734  9428 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:08.415263  1657 leader_election.cc:400] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Vote denied by peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) with higher term. Message: Invalid argument: T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 23. Current term is 24.
I20260709 13:03:08.422391  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.422433  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.423833  1614 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:03:08.417415  2201 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:08.429060  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.429853  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:08.431718  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:08.433580  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.434363  1617 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 25: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:08.435300  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.436096  1615 raft_consensus.cc:2368] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 21. Current term is 22.
I20260709 13:03:08.437016  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.437745  1686 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
I20260709 13:03:08.437852  2231 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:03:08.438849  9562 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:08.438992  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:08.441570  1618 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 24: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:03:08.443717  9426 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:08.444239  9426 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting leader election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:08.444643  9426 raft_consensus.cc:3060] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:08.448323  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:08.449038  2231 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:03:08.439797  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:08.450031  1686 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:08.450532  9565 raft_consensus.cc:493] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.451149  9565 raft_consensus.cc:515] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.393093  9214 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.430671  2232 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 24.
I20260709 13:03:08.408236  9579 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:08.460384  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.454331  1507 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:08.470006  9124 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:08.469811  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.470719  9124 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.470970  1612 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 24.
I20260709 13:03:08.471223  9124 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:08.466501  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:08.472419  2236 raft_consensus.cc:2393] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 24: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:08.469775  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:08.473507  2234 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 22.
I20260709 13:03:08.474684  9568 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.475253  2231 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:03:08.420548  9531 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.482396  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.488394  1688 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:03:08.475234  9568 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.486927  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.483628  2203 leader_election.cc:400] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 21. Current term is 22.
W20260709 13:03:08.484447  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 106140us
I20260709 13:03:08.491158  2203 leader_election.cc:403] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:08.478879  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:08.426839  9428 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:08.453358  9565 leader_election.cc:290] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.509081  1509 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:03:08.493551  9533 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 13:03:08.529312  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 62531.6us
I20260709 13:03:08.536957  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:08.538985  9560 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:08.539552  9560 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.540012  9560 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:08.548303  1583 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:08.555171  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:08.556097  1614 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:08.564985  1686 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:03:08.491859  9579 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 13:03:08.519636  9533 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.601636  9299 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.602392  9299 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.604856  9299 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.558228  9363 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:08.606066  9363 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:08.606309  9589 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:08.606849  9589 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.607322  9589 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:08.607173  9363 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.510695  9531 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.611115  9533 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:03:08.639818  2203 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:08.644577  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:08.639446  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 123170us
I20260709 13:03:08.645489  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e
I20260709 13:03:08.652107  1690 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 28 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 13:03:08.652118  9434 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e for earlier term 21. Current term is 22.
W20260709 13:03:08.657681  9576 log.cc:927] Time spent T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:03:08.658542  9591 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.659142  9591 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:08.650079  9444 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:03:08.650174  1658 consensus_peers.cc:597] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:08.646140  9359 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:08.674054  1658 leader_election.cc:336] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:08.689617  1658 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:08.693835  1614 raft_consensus.cc:2468] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 24.
I20260709 13:03:08.702909  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:08.706215  2203 leader_election.cc:336] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:08.662420  9583 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:03:08.661466  9564 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:08.661733  9587 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.661396  9591 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:08.734027  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 63275us
W20260709 13:03:08.736742   431 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 1568)
I20260709 13:03:08.744349  9568 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.683104  9579 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:08.661844  9509 leader_election.cc:290] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.645938  9428 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:08.761216  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 62566.5us
I20260709 13:03:08.734376  9564 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.689051  9560 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.751087  9599 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:08.760452  9579 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.781890  9579 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:08.793685  9485 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:08.794476  9485 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:08.796607  9485 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.801837  9531 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.729194  9359 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:08.804692  9359 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.763834  9124 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.807751  9124 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.808913  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:08.809696  2230 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:08.778815  9560 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.771783  9564 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:08.767722  9591 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:08.814529  9591 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:08.816601  9591 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:08.767858  9299 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 13:03:08.819031  9530 leader_election.cc:290] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.773343  9587 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:08.822964  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.823709  1689 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:03:08.824393  9587 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.825251  2203 leader_election.cc:304] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:08.827276  9264 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:08.828004  9264 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:08.828980  9435 raft_consensus.cc:2804] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:08.829550  9435 raft_consensus.cc:697] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 24 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:08.830474  9435 consensus_queue.cc:237] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.830961  9264 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.855917  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:08.856630  2231 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:03:08.858417  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.858878  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.865794  1616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e0058494c1f14737a2e0e2b8f4697969" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.854733  9562 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:08.868396  9562 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:08.869652  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 167034us
I20260709 13:03:08.873039  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b833a9bd5950418885dc620b52d4558f" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:08.874121  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.878805  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.879882  1433 catalog_manager.cc:5697] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 10 to 14, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:08.883612  9426 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.886070  9533 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.893520  1611 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 28 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 13:03:08.897574  1614 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 13:03:08.901614  9587 raft_consensus.cc:493] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:08.901885  2203 leader_election.cc:336] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:08.920516  9589 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:08.922147  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.923249  1609 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:08.935552  1688 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:03:08.972087  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 70233.5us
I20260709 13:03:08.958994  1582 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:08.962970  1509 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:08.945749  2230 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:03:08.976791  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da6820413cb240fd9a4bd8655e6762f7" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.985903  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.990435  1434 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:40172 (request call id 236) took 6611 ms. Trace:
I20260709 13:03:08.991240  9589 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:08.974411  9340 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:08.993088  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:08.993343  9596 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:03:08.993755  1611 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:08.994928  9564 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:08.996743  1433 catalog_manager.cc:5697] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 11 to 14, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:08.997860  9379 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:08.998340  9379 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:08.998741  9379 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:08.999426  9533 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 election: Requested vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:08.999337  9587 raft_consensus.cc:515] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:08.971469  9599 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:08.997725  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:09.001791  1427 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet ab5b971190d342d2b2568c30f3c0c81f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cecad4f86a114a7696e398b3e8cc511e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:09.008394  9299 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:03:09.010715  1688 raft_consensus.cc:3060] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:08.993227  9340 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:09.012296  9562 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:09.014621  9579 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:09.004235  9596 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:03:08.992660  1614 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:03:09.018482  9587 leader_election.cc:290] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:09.019331  9426 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:09.024396  2203 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:09.029100  1611 raft_consensus.cc:2468] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:03:09.029524  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:09.030212  1612 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:09.037407  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:09.044428  1610 raft_consensus.cc:2393] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 25: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:09.045498  1691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:08.997232  9564 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:09.046419  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:09.050642  1508 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:09.051784  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:09.053409  1615 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:09.013234  9340 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:09.054739  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:09.057677  1690 raft_consensus.cc:3060] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:09.030196  9579 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:09.064072  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:09.062491  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:09.059099  1508 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:09.069029  1688 raft_consensus.cc:2468] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 25.
I20260709 13:03:09.070998  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:09.071509  9561 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:09.071725  1689 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:03:09.072062  9561 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.072579  9561 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:09.064335  9533 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:09.079150  9565 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.079813  9565 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:09.052688  1433 catalog_manager.cc:5697] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 7 to 14, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:09.083279  9565 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:09.083341  1433 catalog_manager.cc:5697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 10 to 14, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:08.991065  1434 rpcz_store.cc:276] 0709 13:03:02.393043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:02.393513 (+   470us) service_pool.cc:224] Handling call
0709 13:03:02.869405 (+475892us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:03:08.990127 (+6120722us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:06.617011 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:06.617779 (+   768us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:06.617830 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:03:06.617857 (+    27us) tablet.cc:662] Decoding operations
0709 13:03:06.620353 (+  2496us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:06.620458 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:06.620498 (+    40us) tablet.cc:685] Acquiring locks for 27 operations
0709 13:03:06.683991 (+ 63493us) tablet.cc:701] Row locks acquired
0709 13:03:06.684024 (+    33us) write_op.cc:260] PREPARE: finished
0709 13:03:06.684235 (+   211us) write_op.cc:270] Start()
0709 13:03:06.684392 (+   157us) write_op.cc:276] Timestamp: P: 1783602186684166 usec, L: 0
0709 13:03:06.684421 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 13:03:06.720055 (+ 35634us) log.cc:844] Serialized 10558 byte log entry
0709 13:03:07.160160 (+440105us) op_driver.cc:464] REPLICATION: finished
0709 13:03:07.207174 (+ 47014us) write_op.cc:301] APPLY: starting
0709 13:03:07.207342 (+   168us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:07.454427 (+247085us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:07.454555 (+   128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:07.849105 (+394550us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:07.849738 (+   633us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=54,mrs_lookups=0
0709 13:03:07.849818 (+    80us) write_op.cc:312] APPLY: finished
0709 13:03:07.850587 (+   769us) log.cc:844] Serialized 236 byte log entry
0709 13:03:07.850821 (+   234us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:07.851364 (+   543us) write_op.cc:454] Released schema lock
0709 13:03:07.851445 (+    81us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:07.929659 (+ 78214us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:07.943160 (+ 13501us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":111060,"reported_tablets":40,"spinlock_wait_cycles":273605376,"child_traces":[["op",{"apply.queue_time_us":46641,"apply.run_cpu_time_us":43110,"apply.run_wall_time_us":680885,"cfile_cache_hit":110,"cfile_cache_hit_bytes":117149,"num_ops":27,"prepare.queue_time_us":114061,"prepare.run_cpu_time_us":39671,"prepare.run_wall_time_us":300371,"raft.queue_time_us":102986,"raft.run_cpu_time_us":13126,"raft.run_wall_time_us":62122,"replication_time_us":475517,"thread_start_us":504720,"threads_started":4,"wal-append.queue_time_us":177947,"wal-append.run_cpu_time_us":22559,"wal-append.run_wall_time_us":1941735}]]}
I20260709 13:03:09.088119  9561 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:09.094049  1692 raft_consensus.cc:2393] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 18: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:03:09.096226  1689 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:03:09.097025  1615 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 25.
I20260709 13:03:09.099047  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:09.100370  1612 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:03:09.102101  1692 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 25
I20260709 13:03:09.099138  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
W20260709 13:03:09.104125  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 97352.2us
I20260709 13:03:09.121258  1687 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:09.058534  9596 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 13:03:09.129745  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:09.125180  1658 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:09.137887  9559 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:09.138695  9559 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:09.139505  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:09.141861  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:09.145634  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81301f84b53f41349ac6bca78f1a77f4" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:09.151512  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:09.153157  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c601f0c0dfb54023907d2722495668cb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:09.154829  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:09.161324  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:09.162099  1689 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 18.
I20260709 13:03:09.169135  1686 raft_consensus.cc:2468] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 25.
I20260709 13:03:09.178114  1433 catalog_manager.cc:5697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 12 to 14, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:09.180029  1433 catalog_manager.cc:5697] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:09.182523  9561 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:09.183401  1507 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:09.184823  1507 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:09.133793  9428 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
W20260709 13:03:09.161145  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 94374.4us
I20260709 13:03:09.194512  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:09.202149  9607 raft_consensus.cc:2804] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:09.202944  9607 raft_consensus.cc:697] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:09.203172  1433 catalog_manager.cc:5697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 14 to 18, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:09.204052  9607 consensus_queue.cc:237] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:09.215726  1692 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 25.
W20260709 13:03:09.217293  1658 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:09.156243  9559 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:09.227993  1658 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.228463  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e984cd665a4768933ff09e8ba0e07b" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:09.207446  9596 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:09.230334  1690 raft_consensus.cc:2468] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 24.
W20260709 13:03:09.230805  1658 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.231263  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:09.215935  9561 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:09.221640  1509 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:03:09.223647  2236 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 20 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 13:03:09.222993  1433 catalog_manager.cc:5697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 12 to 18, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:09.224783  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:09.238451  9531 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:09.220721  9587 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:09.242091  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:09.244927  1615 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 17.
I20260709 13:03:09.242759  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:09.239650  1507 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:09.247083  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:09.250322  1692 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:09.247520  1690 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:03:09.250751  1507 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:09.229616  9379 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:09.257015  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:09.244776  9533 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:09.260816  9596 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:09.261452  2235 raft_consensus.cc:2393] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 25: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:09.262758  1508 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:09.264084  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:09.244247  9214 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:09.268209  2201 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:09.268333  1508 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:09.254428  9428 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:09.271039  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:09.271678  2234 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:09.271785  1612 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 23: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:09.275182  9588 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:09.267493  9596 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:09.276870  9596 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:09.277443  9379 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:09.238760  1433 catalog_manager.cc:5697] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } }
I20260709 13:03:09.282214  1433 catalog_manager.cc:5697] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 10 to 14, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 14 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:09.284252  9214 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:09.288751  9612 log.cc:927] Time spent T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:03:09.292124  9596 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:09.257256  9579 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:09.296604  9492 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:09.297151  1508 leader_election.cc:304] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:09.266705  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 207700us
I20260709 13:03:09.295063  9533 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 25 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:03:09.307696  2203 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.308171  2201 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:09.308657  2230 raft_consensus.cc:1275] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:03:09.317590  1692 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:03:09.324038  2234 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 23.
I20260709 13:03:09.324932  1508 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:09.314918  9582 raft_consensus.cc:2804] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:09.331560  2235 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 24 LEADER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:09.332733  1508 leader_election.cc:304] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:09.335354  9359 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:09.335510  9340 consensus_queue.cc:1059] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:03:09.328565  9579 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
I20260709 13:03:09.336380  9340 consensus_queue.cc:1243] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [LEADER]: Peer cecad4f86a114a7696e398b3e8cc511e log is divergent from this leader: its last log entry 24.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 13:03:09.337033  9447 raft_consensus.cc:3055] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:03:09.337638  9579 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:09.341079  9587 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.346699  1508 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:09.342101  9533 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:09.342979  2203 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:09.348373  1508 leader_election.cc:304] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:09.325902  9565 raft_consensus.cc:2749] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:09.363076  2201 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:09.333729  9444 raft_consensus.cc:2804] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:09.373769  9561 raft_consensus.cc:2749] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:09.372061  1582 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:09.382553  9214 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:09.383262  9214 raft_consensus.cc:2804] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:09.383767  9214 raft_consensus.cc:697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:09.384589  9363 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:09.363775  9596 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:09.386073  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:09.386816  1613 raft_consensus.cc:2368] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 24. Current term is 25.
I20260709 13:03:09.345691  9447 raft_consensus.cc:740] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:09.388540  9359 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 13:03:09.389246  9447 consensus_queue.cc:260] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:09.390489  9447 raft_consensus.cc:3060] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:09.367532  9582 raft_consensus.cc:697] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 25 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:09.361950  9587 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:09.357265  9588 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:09.344082  1433 catalog_manager.cc:5697] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 14 to 18, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:03:09.411362  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 48390.2us
W20260709 13:03:09.393136  1658 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.384738  9214 consensus_queue.cc:237] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:09.408959  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:09.415985  1692 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 23: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
W20260709 13:03:09.412933  1658 leader_election.cc:336] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.354712  8974 raft_consensus.cc:2749] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:09.417102  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:09.422152  9560 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:09.430367  1658 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.453825  2232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:09.456655  1613 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:09.457574  1613 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:03:09.418462  9444 raft_consensus.cc:697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 25 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:09.470063  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
W20260709 13:03:09.466655  1584 leader_election.cc:336] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.471998  2231 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 23: Already voted for candidate 83f6f5810950417195f707ff308a8125 in this term.
I20260709 13:03:09.471844  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:09.467743  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:09.479830  1684 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:09.442175  9428 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:09.487712  1613 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:03:09.465965  9400 consensus_queue.cc:1048] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 13:03:09.529997  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:09.534173  1582 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:09.515080  9568 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:09.538009  1508 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:09.540287  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 78743.4us
W20260709 13:03:09.540720  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 70375.5us
W20260709 13:03:09.538095  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 68753.9us
I20260709 13:03:09.541678  1658 leader_election.cc:400] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 24. Current term is 25.
I20260709 13:03:09.542029  1658 leader_election.cc:403] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:03:09.551043  1584 leader_election.cc:336] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:09.552421  1584 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.552911  1584 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:09.555290  1584 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:09.556520  1584 leader_election.cc:336] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:09.557580  1584 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.558175  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:09.564731  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet f03fb00780d54c5fbc0be2c8a1b8d4aa: Illegal state: Replica 984c9089d9804652846f3caeb609d56e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:09.539705  9492 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 13:03:09.546948  9428 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887) with higher term. Message: Invalid argument: T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e for earlier term 24. Current term is 25.
I20260709 13:03:09.547430  9562 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:03:09.548280  9588 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:09.566967  9379 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:09.529496  9582 consensus_queue.cc:237] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 23.20, Last appended by leader: 20, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:09.579953  9587 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:09.564250  9444 consensus_queue.cc:237] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:09.547151  1433 catalog_manager.cc:5697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 9 to 14, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 14 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:09.585245  1618 raft_consensus.cc:1240] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Rejecting Update request from peer 984c9089d9804652846f3caeb609d56e for earlier term 23. Current term is 24. Ops: []
I20260709 13:03:09.577330  9568 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:03:09.608435  1584 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:09.613992  1584 leader_election.cc:336] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.612232  1684 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83f6f5810950417195f707ff308a8125 in term 19.
I20260709 13:03:09.614445  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:09.625259  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 186197us
I20260709 13:03:09.626072  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:09.627290  2235 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:03:09.628587  1509 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 
I20260709 13:03:09.577409  9588 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:03:09.620196  9623 log.cc:927] Time spent T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 13:03:09.620051  9618 log.cc:927] Time spent T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 13:03:09.641537  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:09.643039  1614 raft_consensus.cc:2393] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83f6f5810950417195f707ff308a8125 in current term 19: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:03:09.639261  9587 raft_consensus.cc:493] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:09.644491  9587 raft_consensus.cc:515] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:09.646700  9587 leader_election.cc:290] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:09.648356  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:09.650743  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet 67b2c0e67e8f4d0c9f640a2ce4fb6041: Illegal state: Replica 984c9089d9804652846f3caeb609d56e is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:03:09.655265  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 106168us
I20260709 13:03:09.662017  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "674516b0482b47aaa9380469245f9d2e" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:09.629344  9562 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:09.664008  1689 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:09.667451  1507 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:09.637518  9579 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:03:09.678401  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet 978c5dd19ae34aa4b4b42b8c3ccfb52a: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:09.674863  8974 raft_consensus.cc:3060] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:09.677994  9579 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Becoming Leader. State: Replica: 83f6f5810950417195f707ff308a8125, State: Running, Role: LEADER
W20260709 13:03:09.735673  1509 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 92880.1us
W20260709 13:03:09.756074   431 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 1568)
W20260709 13:03:09.802011  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 132321us
I20260709 13:03:09.746601  9635 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:09.803056  9615 log.cc:927] Time spent T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
W20260709 13:03:09.828128  9614 log.cc:927] Time spent T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:03:09.871801  1507 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 57211.7us
I20260709 13:03:09.744094  1433 catalog_manager.cc:5697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 7 to 13. New cstate: current_term: 13 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:03:09.891700  9447 consensus_meta.cc:287] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Time spent flushing consensus metadata: real 0.501s	user 0.012s	sys 0.000s
I20260709 13:03:09.830834  9579 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:09.813452  9588 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:09.886832  9629 log.cc:927] Time spent T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: Append to log took a long time: real 0.272s	user 0.001s	sys 0.000s
I20260709 13:03:09.916901  1683 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 13:03:09.919536  1683 pending_rounds.cc:85] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Aborting all ops after (but not including) 16
I20260709 13:03:09.919910  1683 pending_rounds.cc:107] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 13:03:09.920181  1683 raft_consensus.cc:2889] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:09.855238  9635 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:09.930918  1690 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 13:03:09.949870  9642 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:03:09.950516  9533 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:09.950892  9635 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:09.918159  1433 catalog_manager.cc:5697] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 14 to 15, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 15 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:09.961783  1433 catalog_manager.cc:5697] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 10 to 13, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 13 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:03:09.963439  1658 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:09.967284  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:09.960805  9632 log.cc:927] Time spent T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
I20260709 13:03:10.023152  9654 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.023769  9654 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:10.026407  2236 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:03:10.042142  9654 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:10.055639  1508 leader_election.cc:304] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:10.057730  1686 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:03:10.058553  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:10.065724  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:10.090767  1508 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:10.085237  9589 raft_consensus.cc:2749] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:03:10.112186  1658 consensus_peers.cc:597] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:10.073364  9654 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:03:10.090571  9613 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 13:03:10.100287  9531 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:10.163208  9659 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 13:03:10.166421  8974 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:10.186568  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:10.212177  1609 raft_consensus.cc:1275] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:03:10.203934  9492 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.222803  9492 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:10.224153  9625 log.cc:927] Time spent T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: Append to log took a long time: real 0.585s	user 0.000s	sys 0.005s
W20260709 13:03:10.228317  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet 85ae4f3774324fcf9745a42f30ffe08b: Illegal state: Replica 8a6a5ae1b7f240408b3ddd30d8a18262 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 28 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } }
I20260709 13:03:10.233367  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "49db1a2fe0dd40eba2ac450e306a6937" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:10.235755  1615 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:03:10.242718  9596 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.243366  9596 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:10.246809  9492 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:10.247550  1612 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 13:03:10.255121  1614 raft_consensus.cc:1275] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:03:10.257957  9660 consensus_queue.cc:1048] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:10.265414  1684 raft_consensus.cc:1275] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 13:03:10.269135  1692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:10.270972  9596 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:10.282593  1509 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:10.286638  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
W20260709 13:03:10.303313  1584 consensus_peers.cc:597] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:10.296703  9587 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 13:03:10.309274  9649 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:03:10.311203  9628 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:10.329252  9363 consensus_queue.cc:1048] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:03:10.334884  9573 consensus_queue.cc:1048] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:03:10.330410  1584 consensus_peers.cc:597] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:10.359681  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6087b5252cf44609bcf98bb0925b1666" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:10.364065  1615 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:10.363981  1584 consensus_peers.cc:597] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:10.366465  1584 consensus_peers.cc:597] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:10.373795  1509 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:10.381727  1584 consensus_peers.cc:597] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:10.385740  1509 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:10.388514  9649 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:10.389508  9359 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:10.403532  2203 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:10.404420  2203 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:10.405647  9602 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.406279  9602 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:10.413168  1611 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:03:10.418236  9649 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 13:03:10.422410  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:10.423571  1584 consensus_peers.cc:597] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:10.429193  9602 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:10.429845  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:10.435227  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:10.442009  1615 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:03:10.437259  9602 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:10.463655  1612 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 13:03:10.464030  1612 pending_rounds.cc:85] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting all ops after (but not including) 11
I20260709 13:03:10.464243  1612 pending_rounds.cc:107] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:03:10.464449  1612 raft_consensus.cc:2889] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:03:10.470371  1683 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:03:10.468660  9602 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:03:10.479486  9573 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.480149  9573 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:10.482829  9533 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:03:10.483498  9573 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:10.485630  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:10.486326  1690 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 24.
I20260709 13:03:10.491658  9460 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:10.497622  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:10.498718  1612 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 24.
I20260709 13:03:10.501252  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:03:10.509132  2203 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:10.505402  9573 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:10.511610  9573 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:10.521727  9573 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:10.523602  9602 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:10.524505  9602 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:10.527505  9602 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:10.542876  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:10.546342  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88cbab659ea041148748114fae549ae0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:10.547920  9573 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:10.548658  1509 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:10.549847  9602 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:10.563535  1690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:10.564190  1690 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:10.570513  2230 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 13:03:10.571285  2203 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:10.571919  2203 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:10.572798  2203 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:10.574752  9587 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:03:10.576671  1683 raft_consensus.cc:1275] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:03:10.578373  9426 consensus_queue.cc:1048] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:03:10.562508  9573 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:10.562505  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:10.582355  1612 raft_consensus.cc:3060] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Advancing to term 25
W20260709 13:03:10.592635  1584 consensus_peers.cc:597] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:10.611304  1690 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 25.
I20260709 13:03:10.619999  1612 raft_consensus.cc:2468] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 25.
I20260709 13:03:10.621484  2203 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:10.631600  9573 raft_consensus.cc:2804] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:10.649156  1690 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Refusing update from remote peer 83f6f5810950417195f707ff308a8125: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:03:10.647580  9573 raft_consensus.cc:697] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 25 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:10.654717  1690 raft_consensus.cc:1275] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 23 index: 20. Preceding OpId from leader: term: 25 index: 21. (index mismatch)
W20260709 13:03:10.662405  1584 consensus_peers.cc:597] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:10.657670  9426 consensus_queue.cc:1048] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.015s
I20260709 13:03:10.678346  9573 consensus_queue.cc:237] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:10.652989  9602 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
W20260709 13:03:10.714919  2203 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:10.753171  1584 consensus_peers.cc:597] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:10.756748   431 thread.cc:527] Waited for 7000ms trying to join with heartbeat (tid 1568)
W20260709 13:03:10.828656  1584 consensus_peers.cc:597] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:10.837009  1434 catalog_manager.cc:5697] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:10.842821  9596 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:10.844086  9596 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:10.848522  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:10.850466  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:10.852630  9596 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:10.855000  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:10.856204  9596 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:10.854890  1434 catalog_manager.cc:5697] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 14 to 18, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 18 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:10.865336  1434 catalog_manager.cc:5697] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 17 to 19, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 19 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:03:10.866827  1433 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55374 (request call id 325) took 47345 ms (client timeout 15000 ms). Trace:
I20260709 13:03:10.867287  1434 catalog_manager.cc:5697] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 23 to 25, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 25 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:10.869702  1434 catalog_manager.cc:5697] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:03:10.867419  1433 rpcz_store.cc:269] 0709 13:02:23.521355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:23.522012 (+   657us) service_pool.cc:224] Handling call
0709 13:03:10.866612 (+47344600us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:10.217463 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:10.218150 (+   687us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:10.218200 (+    50us) write_op.cc:435] Acquired schema lock
0709 13:03:10.218219 (+    19us) tablet.cc:662] Decoding operations
0709 13:03:10.219556 (+  1337us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:10.219657 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:10.219682 (+    25us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:03:10.220849 (+  1167us) tablet.cc:701] Row locks acquired
0709 13:03:10.220906 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:03:10.221081 (+   175us) write_op.cc:270] Start()
0709 13:03:10.221225 (+   144us) write_op.cc:276] Timestamp: P: 1783602190221009 usec, L: 0
0709 13:03:10.221245 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:03:10.268416 (+ 47171us) log.cc:844] Serialized 5970 byte log entry
0709 13:03:10.481082 (+212666us) op_driver.cc:464] REPLICATION: finished
0709 13:03:10.566769 (+ 85687us) write_op.cc:301] APPLY: starting
0709 13:03:10.566861 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:10.706123 (+139262us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:10.711251 (+  5128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:10.790934 (+ 79683us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:10.791296 (+   362us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=30,mrs_lookups=0
0709 13:03:10.791356 (+    60us) write_op.cc:312] APPLY: finished
0709 13:03:10.791758 (+   402us) log.cc:844] Serialized 139 byte log entry
0709 13:03:10.791949 (+   191us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:10.792118 (+   169us) write_op.cc:454] Released schema lock
0709 13:03:10.792171 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:10.792664 (+   493us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:10.792704 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":935970,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":85431,"apply.run_cpu_time_us":24230,"apply.run_wall_time_us":225838,"cfile_cache_hit":62,"cfile_cache_hit_bytes":66917,"num_ops":15,"prepare.queue_time_us":85339,"prepare.run_cpu_time_us":16955,"prepare.run_wall_time_us":111303,"raft.queue_time_us":48449,"raft.run_cpu_time_us":47966,"raft.run_wall_time_us":78019,"replication_time_us":259673,"thread_start_us":227837,"threads_started":4,"wal-append.queue_time_us":119721}]]}
I20260709 13:03:10.871068  1434 catalog_manager.cc:5697] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 20 to 21, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 21 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:10.874080  1434 catalog_manager.cc:5697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 17 to 21, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 21 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:10.875653  1434 catalog_manager.cc:5697] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:10.877243  1434 catalog_manager.cc:5697] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 19 to 24. New cstate: current_term: 24 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:10.878558  1434 catalog_manager.cc:5697] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 19 to 23, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 23 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:03:10.885279  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:10.886735  1434 catalog_manager.cc:5697] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 17 to 22. New cstate: current_term: 22 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:10.889272  1434 catalog_manager.cc:5697] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 14 to 22, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 22 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:10.890937  1434 catalog_manager.cc:5697] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 17 to 21, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 21 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:10.898012  1434 catalog_manager.cc:5697] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 17 to 25. New cstate: current_term: 25 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:10.900913  1434 catalog_manager.cc:5697] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 19 to 20, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 20 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:10.908052  1434 catalog_manager.cc:5697] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 11 to 22. New cstate: current_term: 22 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:10.910140  1434 catalog_manager.cc:5697] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 22 to 24, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 24 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:10.917613  1434 catalog_manager.cc:5697] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 22 to 24, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 24 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:10.919497  1434 catalog_manager.cc:5697] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 22 to 27, leader changed from cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 27 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:10.966485  9587 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:10.971441  9587 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:10.976819  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:10.977015  9587 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:10.980329  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:10.982517  1508 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:10.983629  9587 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:11.059655  2203 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:11.062623  9587 raft_consensus.cc:493] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.063226  9587 raft_consensus.cc:515] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:11.065508  9587 leader_election.cc:290] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:11.066231  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:11.067233  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "474d3b57c33542aba459b97733122bf0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:11.068830  1508 leader_election.cc:304] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:11.070021  9587 raft_consensus.cc:2749] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:11.097764  1658 consensus_peers.cc:597] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:11.101979  2203 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:11.145581  1440 catalog_manager.cc:5697] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 14 to 19, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 19 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:11.161175  1440 catalog_manager.cc:5697] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:11.203858  1440 catalog_manager.cc:5697] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 15 to 21, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 21 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:11.205704  1440 catalog_manager.cc:5697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 18 to 22, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 22 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:11.208113  1440 catalog_manager.cc:5697] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 20 to 26, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 26 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } }
W20260709 13:03:11.211304  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:11.244984  1440 catalog_manager.cc:5697] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:11.248807  1684 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:03:11.256879  1440 catalog_manager.cc:5697] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 13 to 25, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 25 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:11.260308  1440 catalog_manager.cc:5697] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 18 to 23, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 23 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:11.263989  9630 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:11.267158  1440 catalog_manager.cc:5697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 14 to 20. New cstate: current_term: 20 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:03:11.269183  1658 consensus_peers.cc:597] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:11.274144  1440 catalog_manager.cc:5697] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 13 to 18, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 18 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:11.276250  1440 catalog_manager.cc:5697] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 13 to 17, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 17 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:11.290354  1614 raft_consensus.cc:1275] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:03:11.303025  9460 consensus_queue.cc:1048] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:11.355319  9587 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.355871  9587 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:11.358507  9587 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:11.359808  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:11.362182  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb1171fd6323498ea16dfea3e5356fe0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:11.363857  1509 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:11.366391  9596 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:11.411484  1440 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55374 (request call id 327) took 4815 ms. Trace:
I20260709 13:03:11.421281  1439 catalog_manager.cc:5697] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 14 to 16, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 16 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:11.423596  9596 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:11.424271  9596 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:11.432644  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:11.432788  9596 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:11.432687  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:11.440733  1507 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:11.442147  9596 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:03:11.443728  1439 catalog_manager.cc:5697] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:11.413595  1440 rpcz_store.cc:276] 0709 13:03:06.595715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:06.598117 (+  2402us) service_pool.cc:224] Handling call
0709 13:03:06.971179 (+373062us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:03:11.411419 (+4440240us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:11.309496 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:11.310099 (+   603us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:11.310138 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:03:11.310160 (+    22us) tablet.cc:662] Decoding operations
0709 13:03:11.311316 (+  1156us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:11.311391 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:11.311424 (+    33us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:03:11.312461 (+  1037us) tablet.cc:701] Row locks acquired
0709 13:03:11.312483 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:03:11.312653 (+   170us) write_op.cc:270] Start()
0709 13:03:11.312780 (+   127us) write_op.cc:276] Timestamp: P: 1783602191312599 usec, L: 0
0709 13:03:11.312803 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:03:11.313561 (+   758us) log.cc:844] Serialized 4418 byte log entry
0709 13:03:11.316166 (+  2605us) op_driver.cc:464] REPLICATION: finished
0709 13:03:11.316509 (+   343us) write_op.cc:301] APPLY: starting
0709 13:03:11.316591 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:11.327458 (+ 10867us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:11.327579 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:11.352479 (+ 24900us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:11.352704 (+   225us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 13:03:11.352756 (+    52us) write_op.cc:312] APPLY: finished
0709 13:03:11.353055 (+   299us) log.cc:844] Serialized 105 byte log entry
0709 13:03:11.361923 (+  8868us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:11.362073 (+   150us) write_op.cc:454] Released schema lock
0709 13:03:11.362132 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:11.362551 (+   419us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:11.362585 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13285,"reported_tablets":47,"spinlock_wait_cycles":605502720,"child_traces":[["op",{"apply.queue_time_us":120,"apply.run_cpu_time_us":14691,"apply.run_wall_time_us":45981,"cfile_cache_hit":44,"cfile_cache_hit_bytes":46046,"num_ops":11,"prepare.queue_time_us":125,"prepare.run_cpu_time_us":4400,"prepare.run_wall_time_us":7180,"replication_time_us":3218}]]}
I20260709 13:03:11.451973  1439 catalog_manager.cc:5697] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 14 to 17, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 17 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.480789  1439 catalog_manager.cc:5697] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 11 to 13, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 13 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:11.485311  1686 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:03:11.502717  1614 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 13:03:11.505085  1657 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293): Couldn't send request to peer cecad4f86a114a7696e398b3e8cc511e. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c53638cee15242e6a8f5dff3907d70c2. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:11.506533  1658 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:11.513617  9660 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:03:11.558095  1434 catalog_manager.cc:5697] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 14 to 25, leader changed from 984c9089d9804652846f3caeb609d56e (127.0.107.195) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 25 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:11.579571  1434 catalog_manager.cc:5697] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:11.581781  1434 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:03:11.595137  1439 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 344) took 57442 ms (client timeout 15000 ms). Trace:
W20260709 13:03:11.599210  1439 rpcz_store.cc:269] 0709 13:02:14.152658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:14.153037 (+   379us) service_pool.cc:224] Handling call
0709 13:02:14.834992 (+681955us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:03:11.594417 (+56759425us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:11.500868 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:11.501620 (+   752us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:11.501656 (+    36us) write_op.cc:435] Acquired schema lock
0709 13:03:11.501669 (+    13us) tablet.cc:662] Decoding operations
0709 13:03:11.502251 (+   582us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:11.502312 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:11.502336 (+    24us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:03:11.502722 (+   386us) tablet.cc:701] Row locks acquired
0709 13:03:11.502738 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:03:11.502864 (+   126us) write_op.cc:270] Start()
0709 13:03:11.502960 (+    96us) write_op.cc:276] Timestamp: P: 1783602191502821 usec, L: 0
0709 13:03:11.502979 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:03:11.503567 (+   588us) log.cc:844] Serialized 1764 byte log entry
0709 13:03:11.507442 (+  3875us) op_driver.cc:464] REPLICATION: finished
0709 13:03:11.507914 (+   472us) write_op.cc:301] APPLY: starting
0709 13:03:11.507991 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:11.510387 (+  2396us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:11.510473 (+    86us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:11.512970 (+  2497us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:11.513150 (+   180us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:03:11.513194 (+    44us) write_op.cc:312] APPLY: finished
0709 13:03:11.519167 (+  5973us) log.cc:844] Serialized 49 byte log entry
0709 13:03:11.519973 (+   806us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:11.520074 (+   101us) write_op.cc:454] Released schema lock
0709 13:03:11.520115 (+    41us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:11.520486 (+   371us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:11.520519 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":160,"reported_tablets":47,"spinlock_wait_cycles":56724736,"child_traces":[["op",{"apply.queue_time_us":258,"apply.run_cpu_time_us":6221,"apply.run_wall_time_us":12519,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16744,"num_ops":4,"prepare.queue_time_us":3316,"prepare.run_cpu_time_us":2776,"prepare.run_wall_time_us":2951,"replication_time_us":4317}]]}
I20260709 13:03:11.597497  9660 raft_consensus.cc:2955] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Committing config change with OpId 23.13: config changed from index -1 to 13, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:03:11.612141  1614 raft_consensus.cc:2955] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Committing config change with OpId 23.13: config changed from index -1 to 13, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
W20260709 13:03:11.643121  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet a35934c9d0d04fbeac6fa3a58fa8fd63: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:11.647878  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet ebc9ea99300141e883fcf2f1b0d7a744: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:11.652122  1489 auto_rebalancer.cc:629] Failed to schedule move for tablet 67b2c0e67e8f4d0c9f640a2ce4fb6041: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:11.665776  1436 catalog_manager.cc:5697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:03:11.757453   431 thread.cc:527] Waited for 8000ms trying to join with heartbeat (tid 1568)
I20260709 13:03:11.781461  9359 raft_consensus.cc:493] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.782104  9359 raft_consensus.cc:515] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:11.784998  9359 leader_election.cc:290] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:11.786002  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:11.786442  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "380b2dca2259476bbe1f22b2eb110fd0" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:11.796000  1437 catalog_manager.cc:5697] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 17 to 20, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.799913  1437 catalog_manager.cc:5697] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:11.815323  1437 catalog_manager.cc:5697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:11.801157  1436 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55368 (request call id 329) took 53929 ms (client timeout 15000 ms). Trace:
I20260709 13:03:11.816391  9596 raft_consensus.cc:493] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.825682  9596 raft_consensus.cc:515] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:11.819386  1508 leader_election.cc:304] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:11.827585  9359 raft_consensus.cc:2749] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:11.825502  1436 rpcz_store.cc:269] 0709 13:02:17.871673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:17.871970 (+   297us) service_pool.cc:224] Handling call
0709 13:03:11.801084 (+53929114us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:11.718178 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:11.718732 (+   554us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:11.718762 (+    30us) write_op.cc:435] Acquired schema lock
0709 13:03:11.718780 (+    18us) tablet.cc:662] Decoding operations
0709 13:03:11.719105 (+   325us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:11.719157 (+    52us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:11.719184 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:03:11.720975 (+  1791us) tablet.cc:701] Row locks acquired
0709 13:03:11.720993 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:03:11.721131 (+   138us) write_op.cc:270] Start()
0709 13:03:11.721259 (+   128us) write_op.cc:276] Timestamp: P: 1783602191721084 usec, L: 0
0709 13:03:11.721293 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:03:11.721960 (+   667us) log.cc:844] Serialized 541 byte log entry
0709 13:03:11.723760 (+  1800us) op_driver.cc:464] REPLICATION: finished
0709 13:03:11.724215 (+   455us) write_op.cc:301] APPLY: starting
0709 13:03:11.724291 (+    76us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:11.725066 (+   775us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:11.725159 (+    93us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:11.726376 (+  1217us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:11.727912 (+  1536us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:03:11.727954 (+    42us) write_op.cc:312] APPLY: finished
0709 13:03:11.729501 (+  1547us) log.cc:844] Serialized 25 byte log entry
0709 13:03:11.730559 (+  1058us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:11.730656 (+    97us) write_op.cc:454] Released schema lock
0709 13:03:11.730701 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:11.731116 (+   415us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:11.731145 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":26051,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":241,"apply.run_cpu_time_us":2885,"apply.run_wall_time_us":6846,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4186,"num_ops":1,"prepare.queue_time_us":1588,"prepare.run_cpu_time_us":2516,"prepare.run_wall_time_us":4053,"replication_time_us":2339}]]}
I20260709 13:03:11.830071  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:11.836767  9596 leader_election.cc:290] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:11.837878  1437 catalog_manager.cc:5697] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 14 to 15, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 15 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:11.841301  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0695e4119591411b8adab45b5925e8eb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 26 candidate_status { last_received { term: 23 index: 20 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:11.850478  1508 leader_election.cc:304] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:11.851421  9359 raft_consensus.cc:2749] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:11.876978  9587 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.877640  9587 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:11.881029  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:11.881497  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:11.883332  1508 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:11.884593  9359 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:11.888620  9587 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:11.902614  1584 consensus_peers.cc:597] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:11.945626  9587 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:11.946166  9587 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:11.948324  9587 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:11.950515  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:11.952275  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f03fb00780d54c5fbc0be2c8a1b8d4aa" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:11.954006  1507 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:11.955201  9587 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:11.957366  1437 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55368 (request call id 330) took 28842 ms (client timeout 15000 ms). Trace:
W20260709 13:03:11.958006  1437 rpcz_store.cc:269] 0709 13:02:43.114815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:43.115172 (+   357us) service_pool.cc:224] Handling call
0709 13:03:11.957271 (+28842099us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:11.865781 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:11.866378 (+   597us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:11.866432 (+    54us) write_op.cc:435] Acquired schema lock
0709 13:03:11.866449 (+    17us) tablet.cc:662] Decoding operations
0709 13:03:11.867112 (+   663us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:11.867174 (+    62us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:11.867207 (+    33us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:03:11.867656 (+   449us) tablet.cc:701] Row locks acquired
0709 13:03:11.867677 (+    21us) write_op.cc:260] PREPARE: finished
0709 13:03:11.867838 (+   161us) write_op.cc:270] Start()
0709 13:03:11.867975 (+   137us) write_op.cc:276] Timestamp: P: 1783602191867786 usec, L: 0
0709 13:03:11.867997 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:03:11.868740 (+   743us) log.cc:844] Serialized 1706 byte log entry
0709 13:03:11.871631 (+  2891us) op_driver.cc:464] REPLICATION: finished
0709 13:03:11.871959 (+   328us) write_op.cc:301] APPLY: starting
0709 13:03:11.872053 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:11.876516 (+  4463us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:11.876606 (+    90us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:11.879159 (+  2553us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:11.879335 (+   176us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:03:11.879380 (+    45us) write_op.cc:312] APPLY: finished
0709 13:03:11.879603 (+   223us) log.cc:844] Serialized 49 byte log entry
0709 13:03:11.880860 (+  1257us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:11.880965 (+   105us) write_op.cc:454] Released schema lock
0709 13:03:11.881005 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:11.881338 (+   333us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:11.881366 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":299213,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":108,"apply.run_cpu_time_us":6768,"apply.run_wall_time_us":9348,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16744,"num_ops":4,"prepare.queue_time_us":5221,"prepare.run_cpu_time_us":3308,"prepare.run_wall_time_us":5453,"replication_time_us":3494}]]}
I20260709 13:03:11.964178  9704 ts_tablet_manager.cc:933] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e: Initiating tablet copy from peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:11.986450  9704 tablet_copy_client.cc:323] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Beginning tablet copy session from remote peer at address 127.0.107.195:33859
I20260709 13:03:11.990397  1702 tablet_copy_service.cc:140] P 984c9089d9804652846f3caeb609d56e: Received BeginTabletCopySession request for tablet c53638cee15242e6a8f5dff3907d70c2 from peer cecad4f86a114a7696e398b3e8cc511e ({username='slave'} at 127.0.0.1:40014)
I20260709 13:03:11.991174  1702 tablet_copy_service.cc:161] P 984c9089d9804652846f3caeb609d56e: Beginning new tablet copy session on tablet c53638cee15242e6a8f5dff3907d70c2 from peer cecad4f86a114a7696e398b3e8cc511e at {username='slave'} at 127.0.0.1:40014: session id = cecad4f86a114a7696e398b3e8cc511e-c53638cee15242e6a8f5dff3907d70c2
I20260709 13:03:12.000236  1702 tablet_copy_source_session.cc:215] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:03:12.018764  9704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c53638cee15242e6a8f5dff3907d70c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:12.052306  9596 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.052968  9596 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:12.056358  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:12.071357  1435 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 345) took 28015 ms (client timeout 15000 ms). Trace:
I20260709 13:03:12.064206  1432 catalog_manager.cc:5697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
W20260709 13:03:12.089588  2203 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:12.081105  9587 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.084947  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:12.085981  9596 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:12.090301  9587 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:12.091334  1507 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:12.090322  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:12.093115  9587 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:12.093492  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:12.093653  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8377b7234e4472ea27005f16dede669" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:12.095639  1509 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:12.093137  9596 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:12.097586  9587 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:12.109330  1435 rpcz_store.cc:269] 0709 13:02:44.055439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:44.059049 (+  3610us) service_pool.cc:224] Handling call
0709 13:03:12.071152 (+28012103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":86794,"reported_tablets":47}
I20260709 13:03:12.153070  9704 tablet_copy_client.cc:806] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 0 data blocks...
I20260709 13:03:12.170625  9704 tablet_copy_client.cc:670] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Starting download of 1 WAL segments...
I20260709 13:03:12.189517  9704 tablet_copy_client.cc:538] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:03:12.226253  1432 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55356 (request call id 346) took 6630 ms. Trace:
W20260709 13:03:12.230863  2203 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:12.241568  1441 catalog_manager.cc:5697] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.227924  1432 rpcz_store.cc:276] 0709 13:03:05.595531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.595754 (+   223us) service_pool.cc:224] Handling call
0709 13:03:12.226169 (+6630415us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:12.123222 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:12.123771 (+   549us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:12.123805 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:03:12.123825 (+    20us) tablet.cc:662] Decoding operations
0709 13:03:12.124151 (+   326us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:12.124205 (+    54us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:12.124255 (+    50us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:03:12.124445 (+   190us) tablet.cc:701] Row locks acquired
0709 13:03:12.124459 (+    14us) write_op.cc:260] PREPARE: finished
0709 13:03:12.124591 (+   132us) write_op.cc:270] Start()
0709 13:03:12.124709 (+   118us) write_op.cc:276] Timestamp: P: 1783602192124548 usec, L: 0
0709 13:03:12.124749 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:03:12.142296 (+ 17547us) log.cc:844] Serialized 541 byte log entry
0709 13:03:12.143434 (+  1138us) spinlock_profiling.cc:243] Waited 16.7 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa55571 00007fe08fae7e28 00007fe093522581 00007fe093521d58 00007fe0935269f1 00007fe0935269a9 00007fe093526939 00007fe093526901 00007fe093525b2d 000000000038f354 000000000038f199
0709 13:03:12.145245 (+  1811us) op_driver.cc:464] REPLICATION: finished
0709 13:03:12.145599 (+   354us) write_op.cc:301] APPLY: starting
0709 13:03:12.145683 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:12.146448 (+   765us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:12.146471 (+    23us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:12.147128 (+   657us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:12.147302 (+   174us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:03:12.147346 (+    44us) write_op.cc:312] APPLY: finished
0709 13:03:12.147524 (+   178us) log.cc:844] Serialized 25 byte log entry
0709 13:03:12.147682 (+   158us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:12.147765 (+    83us) write_op.cc:454] Released schema lock
0709 13:03:12.147803 (+    38us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:12.148158 (+   355us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:12.148186 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":146410,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":137,"apply.run_cpu_time_us":2510,"apply.run_wall_time_us":2510,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4186,"num_ops":1,"prepare.queue_time_us":13833,"prepare.run_cpu_time_us":3854,"prepare.run_wall_time_us":20359,"replication_time_us":20365,"spinlock_wait_cycles":38470016}]]}
I20260709 13:03:12.250574   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:12.252231   431 tablet_replica.cc:333] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.253280  9596 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:12.253933  9596 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:12.255336   431 raft_consensus.cc:2243] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.256289  9596 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:12.256554   431 raft_consensus.cc:2272] T afa53a120a654b379c23e60544c86b5e P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.258787  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:12.259941   431 tablet_replica.cc:333] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.260951   431 raft_consensus.cc:2243] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.262249   431 raft_consensus.cc:2272] T b7e984cd665a4768933ff09e8ba0e07b P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.262984  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "12adfb7855f340329f110249c207f409" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:12.264545  1509 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:12.273950   431 tablet_replica.cc:333] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.274111  9359 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:12.265245  1441 catalog_manager.cc:5697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.276165  9704 tablet_bootstrap.cc:492] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap starting.
I20260709 13:03:12.285667   431 raft_consensus.cc:2243] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.288095  1441 catalog_manager.cc:5697] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 15 to 17, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 17 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.289400   431 raft_consensus.cc:2272] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.291028  9706 raft_consensus.cc:493] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:12.291805  9706 raft_consensus.cc:515] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:12.295653  9706 leader_election.cc:290] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:12.295751  1658 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:12.297746  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:12.302388  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ae2bf844acee4ce5b28b92217256b084" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:12.307423  1507 leader_election.cc:304] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:12.308655  9706 raft_consensus.cc:2749] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:12.311839   431 tablet_replica.cc:333] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.314323   431 raft_consensus.cc:2243] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.315687   431 raft_consensus.cc:2272] T c65847e774e445909d83c69bf486b7fc P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.312350  1441 catalog_manager.cc:5697] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 15 to 19, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 19 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:12.331338   431 tablet_replica.cc:333] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.332953   431 raft_consensus.cc:2243] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.334117   431 raft_consensus.cc:2272] T 2f07ede48ffe4a5eb0b162da083502c2 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.336812   431 tablet_replica.cc:333] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.338147   431 raft_consensus.cc:2243] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.339318   431 raft_consensus.cc:2272] T 960d36351aeb45a08c60a7d839c40e6d P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.342257   431 tablet_replica.cc:333] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
W20260709 13:03:12.365634  1584 consensus_peers.cc:597] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:12.416697   431 raft_consensus.cc:2243] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.417888   431 raft_consensus.cc:2272] T ebc9ea99300141e883fcf2f1b0d7a744 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:12.429369  1658 consensus_peers.cc:597] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:12.432497  1441 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55368 (request call id 331) took 6867 ms. Trace:
I20260709 13:03:12.433007  1441 rpcz_store.cc:276] 0709 13:03:05.565328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:05.565617 (+   289us) service_pool.cc:224] Handling call
0709 13:03:12.432410 (+6866793us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:12.328989 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:12.329576 (+   587us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:12.329633 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:03:12.329650 (+    17us) tablet.cc:662] Decoding operations
0709 13:03:12.330231 (+   581us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:12.330289 (+    58us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:12.330315 (+    26us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:03:12.330730 (+   415us) tablet.cc:701] Row locks acquired
0709 13:03:12.330755 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:03:12.330894 (+   139us) write_op.cc:270] Start()
0709 13:03:12.331021 (+   127us) write_op.cc:276] Timestamp: P: 1783602192330843 usec, L: 0
0709 13:03:12.331039 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 13:03:12.331667 (+   628us) log.cc:844] Serialized 1706 byte log entry
0709 13:03:12.333794 (+  2127us) op_driver.cc:464] REPLICATION: finished
0709 13:03:12.334177 (+   383us) write_op.cc:301] APPLY: starting
0709 13:03:12.334260 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:12.336850 (+  2590us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:12.336951 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:12.339536 (+  2585us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:12.339737 (+   201us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 13:03:12.339793 (+    56us) write_op.cc:312] APPLY: finished
0709 13:03:12.342088 (+  2295us) log.cc:844] Serialized 49 byte log entry
0709 13:03:12.342904 (+   816us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:12.343557 (+   653us) write_op.cc:454] Released schema lock
0709 13:03:12.343604 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:12.343936 (+   332us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:12.343969 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":187680,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":186,"apply.run_cpu_time_us":6597,"apply.run_wall_time_us":9733,"cfile_cache_hit":16,"cfile_cache_hit_bytes":16744,"num_ops":4,"prepare.queue_time_us":105,"prepare.run_cpu_time_us":2940,"prepare.run_wall_time_us":2949,"replication_time_us":2647}]]}
I20260709 13:03:12.444252   431 tablet_replica.cc:333] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.449855   431 raft_consensus.cc:2243] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.449549  1440 catalog_manager.cc:5697] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:12.451473  1440 catalog_manager.cc:5697] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 13 to 24. New cstate: current_term: 24 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:12.453689  1440 catalog_manager.cc:5697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 12 to 20. New cstate: current_term: 20 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:12.455092  1440 catalog_manager.cc:5697] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 16 to 26, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 83f6f5810950417195f707ff308a8125 (127.0.107.193). New cstate: current_term: 26 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:12.456290  1440 catalog_manager.cc:5697] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:12.457594  1440 catalog_manager.cc:5697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
W20260709 13:03:12.462193  1584 consensus_peers.cc:597] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:12.464329   431 raft_consensus.cc:2272] T 81301f84b53f41349ac6bca78f1a77f4 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.469815  1440 catalog_manager.cc:5697] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } }
I20260709 13:03:12.471222  1440 catalog_manager.cc:5697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 20 to 23, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 23 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:12.472916  1440 catalog_manager.cc:5697] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:12.475103  1440 catalog_manager.cc:5697] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:12.477660  1440 catalog_manager.cc:5697] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "83f6f5810950417195f707ff308a8125" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } }
I20260709 13:03:12.482833   431 tablet_replica.cc:333] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.484006   431 raft_consensus.cc:2243] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:12.495000  1438 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55374 (request call id 326) took 25589 ms (client timeout 15000 ms). Trace:
W20260709 13:03:12.495453  1438 rpcz_store.cc:269] 0709 13:02:46.904192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:02:46.905624 (+  1432us) service_pool.cc:224] Handling call
0709 13:02:47.035633 (+130009us) spinlock_profiling.cc:243] Waited 9.55 ms on lock 0x7b5400353754. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe08fa56090 00007fe08faf4f3a 00007fe0958ab1cc 00007fe0959a937e 00007fe093e6a2a4 00007fe093e6a232 00007fe093e6a161 00007fe093e6a0dc 00007fe093e69392 00007fe08de2748c 00007fe08de268c6
0709 13:03:12.492913 (+25457280us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":207,"reported_tablets":47,"spinlock_wait_cycles":21972736}
I20260709 13:03:12.506357   431 raft_consensus.cc:2272] T da6820413cb240fd9a4bd8655e6762f7 P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.513630  9596 raft_consensus.cc:493] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:12.514514  9596 raft_consensus.cc:515] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:12.516423   431 tablet_replica.cc:333] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.526192   431 raft_consensus.cc:2243] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.526787   431 raft_consensus.cc:2272] T c601f0c0dfb54023907d2722495668cb P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.528337  9596 leader_election.cc:290] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:12.531747   431 tablet_replica.cc:333] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.532487  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:12.541468  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59d952214bc24204a149e6f1a9af3184" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
W20260709 13:03:12.543843  1428 catalog_manager.cc:4755] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): ChangeConfig:REMOVE_PEER RPC failed for tablet c65847e774e445909d83c69bf486b7fc: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:12.559808  1508 leader_election.cc:304] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:12.563648  9596 raft_consensus.cc:2749] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:12.532655   431 raft_consensus.cc:2243] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.572398   431 raft_consensus.cc:2272] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.575294   431 tablet_replica.cc:333] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.576313   431 raft_consensus.cc:2243] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.577472   431 raft_consensus.cc:2272] T 13adc029302444abaaffd4bed11760a3 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.581403   431 tablet_replica.cc:333] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.582479   431 raft_consensus.cc:2243] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.583545   431 raft_consensus.cc:2272] T 3b24dad799cc469c9d10c4833dffe30c P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.589211   431 tablet_replica.cc:333] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.590334   431 raft_consensus.cc:2243] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.591003   431 raft_consensus.cc:2272] T 0695e4119591411b8adab45b5925e8eb P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.594094   431 tablet_replica.cc:333] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.595044   431 raft_consensus.cc:2243] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.601157   431 raft_consensus.cc:2272] T e0058494c1f14737a2e0e2b8f4697969 P 83f6f5810950417195f707ff308a8125 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.604477   431 tablet_replica.cc:333] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.615003  9704 tablet_bootstrap.cc:492] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=3 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:03:12.616118  9704 tablet_bootstrap.cc:492] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e: Bootstrap complete.
I20260709 13:03:12.616894  9704 ts_tablet_manager.cc:1403] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e: Time spent bootstrapping tablet: real 0.341s	user 0.052s	sys 0.020s
I20260709 13:03:12.605391   431 raft_consensus.cc:2243] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.620529   431 pending_rounds.cc:70] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Trying to abort 1 pending ops.
I20260709 13:03:12.620826   431 pending_rounds.cc:77] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125: Aborting op as it isn't in flight: id { term: 18 index: 8 } timestamp: 7305634504737562624 op_type: NO_OP noop_request { }
I20260709 13:03:12.621249   431 raft_consensus.cc:2889] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:12.621628   431 raft_consensus.cc:2272] T 7ba0620abe844f5bb71ba9dd947c3b9c P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.625351   431 tablet_replica.cc:333] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.629125  9704 raft_consensus.cc:359] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:03:12.630059  9704 raft_consensus.cc:740] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Initialized, Role: LEARNER
I20260709 13:03:12.630734  9704 consensus_queue.cc:260] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } }
I20260709 13:03:12.633351  9704 ts_tablet_manager.cc:1434] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:03:12.647261  1702 tablet_copy_service.cc:342] P 984c9089d9804652846f3caeb609d56e: Request end of tablet copy session cecad4f86a114a7696e398b3e8cc511e-c53638cee15242e6a8f5dff3907d70c2 received from {username='slave'} at 127.0.0.1:40014
I20260709 13:03:12.647794  1702 tablet_copy_service.cc:434] P 984c9089d9804652846f3caeb609d56e: ending tablet copy session cecad4f86a114a7696e398b3e8cc511e-c53638cee15242e6a8f5dff3907d70c2 on tablet c53638cee15242e6a8f5dff3907d70c2 with peer cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:12.674428  1584 consensus_peers.cc:597] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:12.701615  1441 catalog_manager.cc:5697] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: config changed from index -1 to 13, NON_VOTER cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) added. New cstate: current_term: 23 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } }
I20260709 13:03:12.725076  1440 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:55374 (request call id 328) took 1178 ms. Trace:
W20260709 13:03:12.727577  1584 consensus_peers.cc:597] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:12.725545  1440 rpcz_store.cc:276] 0709 13:03:11.546633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:11.546776 (+   143us) service_pool.cc:224] Handling call
0709 13:03:12.724979 (+1178203us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:03:12.507738 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:03:12.508311 (+   573us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:03:12.508353 (+    42us) write_op.cc:435] Acquired schema lock
0709 13:03:12.508365 (+    12us) tablet.cc:662] Decoding operations
0709 13:03:12.509375 (+  1010us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:03:12.509456 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 13:03:12.509485 (+    29us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:03:12.514427 (+  4942us) tablet.cc:701] Row locks acquired
0709 13:03:12.514446 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:03:12.514606 (+   160us) write_op.cc:270] Start()
0709 13:03:12.514739 (+   133us) write_op.cc:276] Timestamp: P: 1783602192514555 usec, L: 0
0709 13:03:12.514763 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:03:12.515557 (+   794us) log.cc:844] Serialized 4418 byte log entry
0709 13:03:12.519470 (+  3913us) op_driver.cc:464] REPLICATION: finished
0709 13:03:12.520578 (+  1108us) write_op.cc:301] APPLY: starting
0709 13:03:12.520666 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:03:12.536934 (+ 16268us) spinlock_profiling.cc:243] Waited 971 us on lock 0x7b18002ff0e0. stack: 00007fe08c9cd343 00007fe08c9cd240 00007fe08c35c1d4 00007fe09609a304 00007fe09609a299 00007fe0958c7de1 00007fe0958d45b1 00007fe08f82b0ec 00007fe08f877c0a 00007fe08f8774f7 00007fe08f877490 00007fe08f82ef48 00007fe0935431bb 00007fe093543532 00007fe09358ab93 00007fe093437de3
0709 13:03:12.581969 (+ 45035us) tablet.cc:1370] finished BulkCheckPresence
0709 13:03:12.582006 (+    37us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:03:12.686726 (+104720us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:03:12.686951 (+   225us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 13:03:12.686997 (+    46us) write_op.cc:312] APPLY: finished
0709 13:03:12.687310 (+   313us) log.cc:844] Serialized 105 byte log entry
0709 13:03:12.687510 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:12.687649 (+   139us) write_op.cc:454] Released schema lock
0709 13:03:12.687698 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:03:12.688059 (+   361us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:03:12.688087 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8099,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":871,"apply.run_cpu_time_us":20361,"apply.run_wall_time_us":167456,"cfile_cache_hit":44,"cfile_cache_hit_bytes":46046,"num_ops":11,"prepare.queue_time_us":662,"prepare.run_cpu_time_us":4371,"prepare.run_wall_time_us":14583,"replication_time_us":4574,"spinlock_wait_cycles":2233088}]]}
W20260709 13:03:12.737979  2203 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:12.742758   431 raft_consensus.cc:2243] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.754441   431 pending_rounds.cc:70] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Trying to abort 1 pending ops.
I20260709 13:03:12.754765   431 pending_rounds.cc:77] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125: Aborting op as it isn't in flight: id { term: 18 index: 12 } timestamp: 7305634525978980352 op_type: NO_OP noop_request { }
I20260709 13:03:12.755287   431 raft_consensus.cc:2889] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:12.755677   431 raft_consensus.cc:2272] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:12.767535  1584 consensus_peers.cc:597] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:12.796814   431 tablet_replica.cc:333] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.797971   431 raft_consensus.cc:2243] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.807019   431 raft_consensus.cc:2272] T ab5b971190d342d2b2568c30f3c0c81f P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:12.824806  1584 consensus_peers.cc:597] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:12.829146   431 tablet_replica.cc:333] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.830186   431 raft_consensus.cc:2243] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.859674   431 raft_consensus.cc:2272] T 1a69ce55a73e464a962e15e73fd1db2c P 83f6f5810950417195f707ff308a8125 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.862923   431 tablet_replica.cc:333] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.863843   431 raft_consensus.cc:2243] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.869979   431 raft_consensus.cc:2272] T e1ef0a9c8bca48b6898df70834e97c4c P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.872779   431 tablet_replica.cc:333] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.877645   431 raft_consensus.cc:2243] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.879015   431 raft_consensus.cc:2272] T 59d952214bc24204a149e6f1a9af3184 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.893216   431 tablet_replica.cc:333] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.894134   431 raft_consensus.cc:2243] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:12.894829   431 raft_consensus.cc:2272] T 6087b5252cf44609bcf98bb0925b1666 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:12.898509   431 tablet_replica.cc:333] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:12.901017   431 raft_consensus.cc:2243] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:03:12.902141   431 raft_consensus.cc:2272] T 22357a3949344a1792e5bae4a0291987 P 83f6f5810950417195f707ff308a8125 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.031427   431 tablet_replica.cc:333] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.032305   431 raft_consensus.cc:2243] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.033058   431 raft_consensus.cc:2272] T 380b2dca2259476bbe1f22b2eb110fd0 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.041940  9359 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.042495  9359 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.044587  9359 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:13.045961  9359 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.046435  9359 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.048883  9359 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:13.055106  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:13.060207  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:13.063036   431 tablet_replica.cc:333] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.078063   431 raft_consensus.cc:2243] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:13.104684   431 raft_consensus.cc:2272] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.111718   431 tablet_replica.cc:333] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
W20260709 13:03:13.122961  2203 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:13.124014   431 raft_consensus.cc:2243] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.124922   431 raft_consensus.cc:2272] T 034eef09c5fd47c9b010158a104b4028 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:13.142236  2203 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:13.145973   431 tablet_replica.cc:333] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.156792   431 raft_consensus.cc:2243] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.162578  2235 raft_consensus.cc:1217] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 LEARNER]: Deduplicated request from leader. Original: 23.12->[23.13-23.13]   Dedup: 23.13->[]
W20260709 13:03:13.177773  1584 consensus_peers.cc:597] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:13.180025  2203 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:13.183843  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 76621.5us
W20260709 13:03:13.192788  1584 consensus_peers.cc:597] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:13.195384   431 raft_consensus.cc:2272] T bb1171fd6323498ea16dfea3e5356fe0 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.203050   431 tablet_replica.cc:333] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.204176   431 raft_consensus.cc:2243] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 28 LEADER]: Raft consensus shutting down.
I20260709 13:03:13.205461   431 raft_consensus.cc:2272] T 85ae4f3774324fcf9745a42f30ffe08b P 83f6f5810950417195f707ff308a8125 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:13.206804  1584 consensus_peers.cc:597] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:13.210906  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fa01a3d0bb094d0d80ca50f0287dbb27" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:13.208360   431 tablet_replica.cc:333] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.214736  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34b2ba71a23e4ba3becae445ee7598c5" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:13.223453   431 raft_consensus.cc:2243] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.231628  1507 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:13.233011  1507 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:13.233891  9713 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:13.234608  9713 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:13.244494   431 raft_consensus.cc:2272] T 12adfb7855f340329f110249c207f409 P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:13.248143  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:13.259742  1658 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 257 similar messages]
I20260709 13:03:13.249331   431 tablet_replica.cc:333] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.262316   431 raft_consensus.cc:2243] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.263109   431 raft_consensus.cc:2272] T fa01a3d0bb094d0d80ca50f0287dbb27 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:13.258675  1657 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 97220.4us
I20260709 13:03:13.296439   431 tablet_replica.cc:333] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.297325   431 raft_consensus.cc:2243] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:03:13.298220   431 raft_consensus.cc:2272] T ad1245d9a7bf4a21ac3d2291898dd976 P 83f6f5810950417195f707ff308a8125 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:13.304715  2203 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:13.322625   431 tablet_replica.cc:333] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.323688   431 raft_consensus.cc:2243] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.324374   431 raft_consensus.cc:2272] T 34b2ba71a23e4ba3becae445ee7598c5 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.354049  9714 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.354686  9714 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.356900  9714 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:13.357940  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:13.363235  1658 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.363730  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:13.365319  9718 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:13.372167  2203 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:13.372781  1584 consensus_peers.cc:597] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:13.374624  9693 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.375200  9693 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.395154  9706 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.395812  9706 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.396646   431 tablet_replica.cc:333] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.397557   431 raft_consensus.cc:2243] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:13.397811  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:13.399659  9693 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:13.401539  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:13.402026  9706 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:13.401630  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:13.404335  1507 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:13.405468  9706 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:13.417879  9693 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.418620  9693 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:13.428445  1584 leader_election.cc:336] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.429018  1584 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:13.429991  9692 raft_consensus.cc:2749] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:13.433411   431 raft_consensus.cc:2272] T 969ec621630549bb9b288fdb50e46fbe P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.438032   431 tablet_replica.cc:333] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.441546  9693 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:13.445454  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:13.446280  1684 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:03:13.449775  1582 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:13.450642  9692 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:13.451174  9692 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.451668  9692 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:13.455103  1584 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.464355  9692 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.469782  9573 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.470448  9573 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.472563  9573 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:13.475452  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:13.476313  1612 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:03:13.477546  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:13.480165   431 raft_consensus.cc:2243] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.480913   431 raft_consensus.cc:2272] T c8377b7234e4472ea27005f16dede669 P 83f6f5810950417195f707ff308a8125 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.481575  9693 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:03:13.482038  2203 leader_election.cc:336] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.483444  9712 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:13.483896  9712 raft_consensus.cc:493] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.484282  9712 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:13.482143  9693 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.494635  9596 raft_consensus.cc:493] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.497797  9596 raft_consensus.cc:515] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:13.499888  1584 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.500012  9596 leader_election.cc:290] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:13.501540  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:13.501757  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "960d36351aeb45a08c60a7d839c40e6d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:13.503645  9693 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:13.504060  1683 raft_consensus.cc:3060] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:13.502238  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ce476a5b0534bc5b6915606f6d37f30" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:13.511188  1508 leader_election.cc:304] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:13.512722  9596 raft_consensus.cc:2749] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:03:13.496686  9692 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:13.511253  9573 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.510674  9712 raft_consensus.cc:515] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:13.517518  1584 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.514535  9573 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.518172  1683 raft_consensus.cc:2468] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:03:13.524504  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:13.525514  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:13.521840   431 tablet_replica.cc:333] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.526573  1582 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:13.527972  9573 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:13.529711  9720 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.530278  9720 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.532842  9720 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:13.533702  1583 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:13.534597  9720 raft_consensus.cc:2804] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:13.535216  9720 raft_consensus.cc:697] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:13.536237  9720 consensus_queue.cc:237] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.539021  9714 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:03:13.539825  1584 leader_election.cc:336] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.539562  9714 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.540576  9692 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:13.541854  9714 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:13.545979  1658 leader_election.cc:336] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.547043  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:13.547706  1612 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:03:13.549647  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:13.551744  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:13.552436  9714 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:13.552862  9714 raft_consensus.cc:493] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.553249  9714 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:13.557780  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:13.560633  9596 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:13.561230  9596 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.566322  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:13.568460  9596 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:13.570554  2203 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.571127  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:13.571970  9573 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:13.576313  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:13.578576  1509 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 83f6f5810950417195f707ff308a8125; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:13.579614  9596 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:13.590703  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:13.576395  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "83f6f5810950417195f707ff308a8125" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:13.597469  9712 leader_election.cc:290] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:13.599555  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afa53a120a654b379c23e60544c86b5e" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:13.600169  1610 raft_consensus.cc:3060] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:13.604215   431 raft_consensus.cc:2243] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.604970   431 raft_consensus.cc:2272] T 3ce476a5b0534bc5b6915606f6d37f30 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.606349  9718 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.606945  9718 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.607735   431 tablet_replica.cc:333] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.607829  1610 raft_consensus.cc:2468] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:03:13.608711   431 raft_consensus.cc:2243] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:03:13.611465  9718 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:13.614820   431 raft_consensus.cc:2272] T ed741b21bc0447a0896904a04e535f8a P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.614671  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:13.615706  1610 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
W20260709 13:03:13.616109  1658 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.611285  9714 raft_consensus.cc:515] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.621949  2203 leader_election.cc:304] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:13.626881  9712 raft_consensus.cc:2804] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:13.627140  1438 catalog_manager.cc:5697] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 19 to 20, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:13.629165  9712 raft_consensus.cc:697] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:13.640882  9573 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.641515  9573 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.644465  9573 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:13.645676  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:13.646349  1683 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:03:13.648346  9712 consensus_queue.cc:237] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.654459  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:13.655169  2236 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:03:13.660120  9720 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.660806  9720 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:13.661571  1658 consensus_peers.cc:597] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:13.664500  1684 raft_consensus.cc:2393] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 20: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
W20260709 13:03:13.668813  1584 leader_election.cc:336] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:13.671239  1658 leader_election.cc:336] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.671355  9720 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:13.672716  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:13.674535  1582 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:13.677145  9719 raft_consensus.cc:2749] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:13.677211  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ebc9ea99300141e883fcf2f1b0d7a744" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:13.678076  1610 raft_consensus.cc:3060] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:13.678417   431 tablet_replica.cc:333] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.680305   431 raft_consensus.cc:2243] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.676654  9714 leader_election.cc:290] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:13.689528  1582 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:13.697892   431 raft_consensus.cc:2272] T ae2bf844acee4ce5b28b92217256b084 P 83f6f5810950417195f707ff308a8125 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.701449   431 tablet_replica.cc:333] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.702433   431 raft_consensus.cc:2243] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:13.662313  2203 leader_election.cc:336] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.690124  1610 raft_consensus.cc:2468] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:03:13.696056  9692 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.711606  9692 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.713553  9719 raft_consensus.cc:2749] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:13.703151   431 raft_consensus.cc:2272] T caae9b2a426d42fb8c2353e1c978d0fb P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.715930  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:13.718214  1658 leader_election.cc:304] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:13.720786  9714 raft_consensus.cc:2804] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:13.721488  9714 raft_consensus.cc:697] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 20 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:13.723409  9718 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.723390  9714 consensus_queue.cc:237] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.724385  9718 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.726517  9718 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:13.714699  9692 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:13.716068  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 53789us
W20260709 13:03:13.734123  1658 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.735425  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:13.736357  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:13.737079  1614 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:03:13.739209  9719 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.739995  9719 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.720643   431 tablet_replica.cc:333] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.742303   431 raft_consensus.cc:2243] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.743032   431 raft_consensus.cc:2272] T 3ab2e02ed3c849e9bda035a57add301d P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.743251  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:13.743952  2236 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 22.
I20260709 13:03:13.745187  1583 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:13.745563  1657 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:13.743052  9719 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:13.746516   431 tablet_replica.cc:333] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.747030  9718 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:13.747530  9718 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.747646   431 raft_consensus.cc:2243] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.748080  9718 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:13.748390   431 pending_rounds.cc:70] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Trying to abort 2 pending ops.
I20260709 13:03:13.748739   431 pending_rounds.cc:77] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Aborting op as it isn't in flight: id { term: 22 index: 17 } timestamp: 7305634494429474816 op_type: NO_OP noop_request { }
I20260709 13:03:13.746567  9719 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:13.753656  9719 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.761999  9719 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:13.760281  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:13.770118  9701 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:13.777629  9701 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.778175  9701 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.780335  9701 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:13.781193  9712 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.781873  9712 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.783998  9712 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:13.793156  1584 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.793769  1584 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:13.749104   431 raft_consensus.cc:2889] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:13.757772  1440 catalog_manager.cc:5697] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 22 to 23, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 23 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:13.799309   431 pending_rounds.cc:77] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125: Aborting op as it isn't in flight: id { term: 20 index: 16 } timestamp: 7305634441710903296 op_type: NO_OP noop_request { }
I20260709 13:03:13.799924   431 raft_consensus.cc:2889] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:13.800423   431 raft_consensus.cc:2272] T 474d3b57c33542aba459b97733122bf0 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.802537  9720 raft_consensus.cc:2749] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:13.818742  9711 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.819389  9711 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:13.822664  1658 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.826404  9719 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:13.827376  1658 consensus_peers.cc:597] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:13.829015  9719 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:13.829883  9711 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:13.831730   431 tablet_replica.cc:333] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.834072  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f07ede48ffe4a5eb0b162da083502c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:13.834718  2236 raft_consensus.cc:3060] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:13.842620  2236 raft_consensus.cc:2468] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 23.
I20260709 13:03:13.846043   431 raft_consensus.cc:2243] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.847939   431 raft_consensus.cc:2272] T 21b84a6a3d5b4e8b95867a87260513a1 P 83f6f5810950417195f707ff308a8125 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.829923  9718 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.849001  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:13.850014  1614 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:03:13.851197  9719 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.851728  9719 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.846212  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:13.859331  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:13.862571  9711 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:13.863160  9711 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.863610  9711 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:13.869585  9693 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.870263  9693 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.872406  9693 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:13.872987  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:13.874126  9573 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:03:13.874536  9573 raft_consensus.cc:493] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:03:13.874837  1658 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.875233  9573 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:13.876510  2203 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:13.879920   431 tablet_replica.cc:333] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.880894   431 raft_consensus.cc:2243] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:13.882022   431 raft_consensus.cc:2272] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.882573  9718 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:13.883184  9719 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:13.884578  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:13.885344  1684 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 25.
I20260709 13:03:13.889794  1438 catalog_manager.cc:5697] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 19 to 20, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 20 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:13.893687  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:13.897007  1583 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:13.897555   431 tablet_replica.cc:333] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.896205  1614 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
I20260709 13:03:13.890431  9711 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.899467  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:13.900640   431 raft_consensus.cc:2243] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.922129   431 raft_consensus.cc:2272] T 88cbab659ea041148748114fae549ae0 P 83f6f5810950417195f707ff308a8125 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.899434  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:13.904841  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:13.906539  9719 raft_consensus.cc:2804] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:13.923974  9719 raft_consensus.cc:697] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:13.924185  9711 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:13.925240   431 tablet_replica.cc:333] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.924973  9719 consensus_queue.cc:237] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.926087   431 raft_consensus.cc:2243] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.926743   431 raft_consensus.cc:2272] T 674516b0482b47aaa9380469245f9d2e P 83f6f5810950417195f707ff308a8125 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.929921   431 tablet_replica.cc:333] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.930953  9718 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:13.931442  9718 raft_consensus.cc:493] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.931846  9718 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:13.934017  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:13.934401  9716 raft_consensus.cc:1064] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: attempting to promote NON_VOTER cecad4f86a114a7696e398b3e8cc511e to VOTER
I20260709 13:03:13.900805  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:13.935261  1614 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:03:13.937224  9716 consensus_queue.cc:237] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 11, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.940094  9727 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.940707  9727 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.907588  9573 raft_consensus.cc:515] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:13.942108  9718 raft_consensus.cc:515] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.942927  9727 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:13.943648  2235 raft_consensus.cc:2393] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in current term 24: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:13.947047  9573 leader_election.cc:290] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:13.947285  1658 leader_election.cc:336] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.948513   431 raft_consensus.cc:2243] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:13.949708   431 raft_consensus.cc:2272] T 214bc9e9284a4502844dbdc3395843fa P 83f6f5810950417195f707ff308a8125 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.952437   431 tablet_replica.cc:333] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.953128   431 raft_consensus.cc:2243] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.954172  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:13.955041  2235 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 24: Already voted for candidate cecad4f86a114a7696e398b3e8cc511e in this term.
I20260709 13:03:13.955598  9718 leader_election.cc:290] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:13.956243  1428 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 76890.7us
W20260709 13:03:13.957892  1428 catalog_manager.cc:4755] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): ChangeConfig:REMOVE_PEER RPC failed for tablet c65847e774e445909d83c69bf486b7fc: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.957484   431 raft_consensus.cc:2272] T c53638cee15242e6a8f5dff3907d70c2 P 83f6f5810950417195f707ff308a8125 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.960163  1440 catalog_manager.cc:5697] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 22 to 23, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 23 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:13.961440   431 tablet_replica.cc:333] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125: stopping tablet replica
I20260709 13:03:13.962190   431 raft_consensus.cc:2243] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:13.962852   431 raft_consensus.cc:2272] T 49db1a2fe0dd40eba2ac450e306a6937 P 83f6f5810950417195f707ff308a8125 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:13.972074  1657 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:13.974197  9701 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.974807  9701 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:13.977183  9701 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:13.909003  1584 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.934866  1684 raft_consensus.cc:2393] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cecad4f86a114a7696e398b3e8cc511e in current term 24: Already voted for candidate 984c9089d9804652846f3caeb609d56e in this term.
I20260709 13:03:13.978684  9718 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 13:03:13.979259  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 70299.1us
W20260709 13:03:13.972162  2203 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:13.982885  2203 leader_election.cc:336] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:13.983659  1658 leader_election.cc:336] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:13.983955  2203 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:13.984403  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:13.985440  9728 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:03:13.986202  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 86842.6us
I20260709 13:03:13.986742  9719 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:13.993357  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:13.993677  9719 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:13.994664  9719 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Advancing to term 26
I20260709 13:03:14.009893  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:14.010270  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13adc029302444abaaffd4bed11760a3" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:14.010802  1610 raft_consensus.cc:3060] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:03:14.011813  2203 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:14.013242  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:14.017822  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a35934c9d0d04fbeac6fa3a58fa8fd63" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:14.018740  1612 raft_consensus.cc:3060] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:03:14.019265  1614 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 24.
I20260709 13:03:14.020407  1610 raft_consensus.cc:2468] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 24.
I20260709 13:03:14.025669  1612 raft_consensus.cc:2468] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 20.
I20260709 13:03:14.027016  9573 raft_consensus.cc:2749] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 13:03:14.029547  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:14.030231  1617 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 22.
W20260709 13:03:14.027060  2203 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:14.031468  2203 leader_election.cc:336] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:14.033931  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:14.035334  2203 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:14.036810  9711 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:14.037624  9711 raft_consensus.cc:2804] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:03:14.038097  9711 raft_consensus.cc:697] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 24 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:14.040277  9711 consensus_queue.cc:237] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:14.057822  1658 leader_election.cc:304] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:14.059912  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:03:14.062222  1584 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:14.063779  1584 leader_election.cc:336] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:14.064214  1584 leader_election.cc:304] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:14.065006  9693 raft_consensus.cc:2749] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:14.065653  1584 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:14.061729  9714 raft_consensus.cc:2804] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:14.070823  9714 raft_consensus.cc:697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 20 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:14.071832  9714 consensus_queue.cc:237] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:14.073807  9718 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:14.074275  9718 raft_consensus.cc:493] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.075918  9718 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:14.076576  1584 consensus_peers.cc:597] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:14.079492  1684 raft_consensus.cc:1275] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:03:14.088114  9701 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.088747  9701 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:14.085659  9693 consensus_queue.cc:1048] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:03:14.091358  9701 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:14.093345  1440 catalog_manager.cc:5697] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 23 to 24, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 24 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:14.101707  1614 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:14.107196  1612 raft_consensus.cc:1275] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:03:14.119789  9719 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:14.120781  1617 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 13:03:14.122646  1658 leader_election.cc:336] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:14.123135  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:14.123042  9736 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:14.126691  9701 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:14.134569  9573 consensus_queue.cc:1048] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:14.140391  2235 raft_consensus.cc:1275] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 LEARNER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:03:14.145767  9736 consensus_queue.cc:1048] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:14.159276  9737 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.161371  9719 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:14.167565  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24dad799cc469c9d10c4833dffe30c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:14.168315  1683 raft_consensus.cc:3060] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:03:14.172806  1584 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:14.175448  1683 raft_consensus.cc:2468] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 26.
I20260709 13:03:14.180824  1582 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:03:14.181887  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 59258.2us
I20260709 13:03:14.179801  1440 catalog_manager.cc:5697] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 19 to 20, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 20 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:14.182935  9692 raft_consensus.cc:2804] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:03:14.183557  9692 raft_consensus.cc:697] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:14.184675  9718 raft_consensus.cc:515] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:14.160241  9737 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:14.224915  9701 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.225585  9701 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:03:14.222358  9720 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.227607  9701 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:14.227392  9720 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:14.233911  9692 consensus_queue.cc:237] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:14.235978  9720 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:14.237905  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:14.238559  1686 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 26.
I20260709 13:03:14.244006  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:14.245548  9719 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:03:14.246045  9719 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.246447  9719 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Advancing to term 27
W20260709 13:03:14.249874  2203 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:14.251446  9693 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.252041  9693 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:14.254299  9693 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:14.254645  9737 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:14.249852  9731 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.255580  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:14.255430  9731 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:14.256264  1686 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:03:14.259219  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:14.263019  9720 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:14.263443  9720 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.263823  9720 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:14.265746  9731 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:14.269203  1584 consensus_peers.cc:597] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:14.239095  9718 leader_election.cc:290] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:14.268484  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:14.270653  2235 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 26.
W20260709 13:03:14.273011  1584 leader_election.cc:336] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:14.297937  1584 leader_election.cc:336] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:14.298609  1584 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:14.307300  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:14.287701  9719 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:14.311033  9692 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 13:03:14.311542  9731 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.309096  1438 catalog_manager.cc:5697] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 25 to 26, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 26 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:14.316236  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 57073us
I20260709 13:03:14.312862  9720 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:14.320571  9731 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:14.327498  9731 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:14.314909  9692 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.328665  9692 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:03:14.334391  9720 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:14.337688  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:14.338397  2235 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 27.
W20260709 13:03:14.336858  1584 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:14.349524  9719 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:14.351468  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:03:14.367563  1584 leader_election.cc:336] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:03:14.368355  1584 leader_election.cc:336] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:14.380209  1612 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67b2c0e67e8f4d0c9f640a2ce4fb6041" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:14.380951  1612 raft_consensus.cc:3060] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:14.390046  1618 raft_consensus.cc:1275] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:03:14.390486  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:14.391088  1686 raft_consensus.cc:3060] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:14.397012  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22357a3949344a1792e5bae4a0291987" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:14.397697  1684 raft_consensus.cc:3060] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:03:14.397838  1686 raft_consensus.cc:2468] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:03:14.402030  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978c5dd19ae34aa4b4b42b8c3ccfb52a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:14.402395  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:14.402839  1610 raft_consensus.cc:2393] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 984c9089d9804652846f3caeb609d56e in current term 21: Already voted for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in this term.
I20260709 13:03:14.369647  9720 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 13:03:14.409940  1684 raft_consensus.cc:2468] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 27.
I20260709 13:03:14.403882  9720 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.413751  9720 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Advancing to term 28
I20260709 13:03:14.414930  2230 raft_consensus.cc:1275] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 23 index: 20. (index mismatch)
I20260709 13:03:14.416963  9693 consensus_queue.cc:1048] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 13:03:14.426095  9746 raft_consensus.cc:2955] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Committing config change with OpId 23.14: config changed from index 13 to 14, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:14.439327  1612 raft_consensus.cc:2468] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 23.
I20260709 13:03:14.445246  1582 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:14.436494  9720 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:14.447803  9719 raft_consensus.cc:2804] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:03:14.450579  9719 raft_consensus.cc:697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:14.451520  9719 consensus_queue.cc:237] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:14.453841  1584 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:14.456149  9692 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:14.457677  9720 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 28 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:14.457870  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a69ce55a73e464a962e15e73fd1db2c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:14.458427  2235 raft_consensus.cc:3060] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Advancing to term 28
W20260709 13:03:14.461992  1584 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:14.463567  9748 consensus_queue.cc:1048] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:03:14.466718  2235 raft_consensus.cc:2468] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 28.
I20260709 13:03:14.471730  1583 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:14.470628  1658 leader_election.cc:304] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
W20260709 13:03:14.477633  2203 connection.cc:573] client connection to 127.0.107.193:43459 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 13:03:14.486639  1612 raft_consensus.cc:2955] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Committing config change with OpId 23.14: config changed from index 13 to 14, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:14.484227  9737 raft_consensus.cc:2804] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:14.484602  9693 raft_consensus.cc:2804] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 13:03:14.492646  9693 raft_consensus.cc:697] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 28 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:14.493584  9737 raft_consensus.cc:697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:14.494573  9737 consensus_queue.cc:237] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:14.477941  9692 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:14.483448  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ef0a9c8bca48b6898df70834e97c4c" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:14.504683  2235 raft_consensus.cc:3060] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Advancing to term 27
I20260709 13:03:14.490106  1582 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:14.495229  9693 consensus_queue.cc:237] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:14.501101  1658 connection.cc:769] client connection to 127.0.107.193:43459 send error: Network error: sendmsg error: Connection reset by peer (error 104)
W20260709 13:03:14.513792  1658 consensus_peers.cc:597] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: sendmsg error: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:14.514467  1658 leader_election.cc:336] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: sendmsg error: Connection reset by peer (error 104)
I20260709 13:03:14.511071  1440 catalog_manager.cc:5697] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 13 to 14, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.516587  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:14.513609  2236 raft_consensus.cc:2955] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Committing config change with OpId 23.14: config changed from index 13 to 14, cecad4f86a114a7696e398b3e8cc511e (127.0.107.196) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:14.515731  9744 raft_consensus.cc:2804] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:03:14.525460  9744 raft_consensus.cc:697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:14.526458  9744 consensus_queue.cc:237] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:14.537225  1610 raft_consensus.cc:1275] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 13:03:14.550318  2235 raft_consensus.cc:2468] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 27.
I20260709 13:03:14.553354  1583 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:03:14.554404  1658 leader_election.cc:336] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: sendmsg error: Connection reset by peer (error 104)
I20260709 13:03:14.554586  9739 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:14.562355  9749 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.562960  9731 raft_consensus.cc:2804] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 13:03:14.570129  9749 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:03:14.572409  9749 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:14.573652  9731 raft_consensus.cc:697] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:14.574781  9731 consensus_queue.cc:237] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:14.580933  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 49739.5us
W20260709 13:03:14.596374  1658 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: sendmsg error: Connection reset by peer (error 104). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:14.597011  1658 leader_election.cc:336] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: sendmsg error: Connection reset by peer (error 104)
I20260709 13:03:14.597435  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:14.610591  9746 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:03:14.610857  1438 catalog_manager.cc:5697] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 22 to 23, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 23 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:14.624454  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:14.624758  9573 consensus_queue.cc:1048] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 13:03:14.625176  1684 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 28.
I20260709 13:03:14.651062  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:14.656342  9744 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 13:03:14.656805  9744 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.657188  9744 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 28 FOLLOWER]: Advancing to term 29
W20260709 13:03:14.688431  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:03:14.699741  9729 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.700340  9729 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:14.702613  9729 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:14.712397  1610 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
W20260709 13:03:14.717456  2203 consensus_peers.cc:597] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:14.721166  2203 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:14.721771  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:14.722602  9729 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:14.733594  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 69198.7us
W20260709 13:03:14.737602  1584 consensus_peers.cc:597] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:14.739494  1584 leader_election.cc:336] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:14.740046  9749 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.747823  9749 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:14.732577  9729 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.750881  9729 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:14.753185  2203 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:03:14.755853  9749 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:14.756175  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:14.756837  2230 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 19.
I20260709 13:03:14.756875  9729 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:14.757822  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:14.759528  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:14.760812  9749 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:03:14.761247  9749 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.761652  9749 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:14.772192  2203 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:14.772722  2203 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:14.774832  1610 raft_consensus.cc:1275] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:03:14.776714  9728 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:14.807598  1584 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:14.788864  9759 log.cc:927] Time spent T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: Append to log took a long time: real 0.138s	user 0.002s	sys 0.000s
I20260709 13:03:14.798923  1440 catalog_manager.cc:5697] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 20 to 21, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 21 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.792445  9749 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:14.829212  9749 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:14.829502  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "969ec621630549bb9b288fdb50e46fbe" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:14.798561  9746 consensus_queue.cc:1048] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 13:03:14.830181  2230 raft_consensus.cc:3060] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:03:14.833209  1658 consensus_peers.cc:597] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:14.833586  9744 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:03:14.807340  9739 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.835120  9739 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:14.837316  2230 raft_consensus.cc:2468] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:03:14.838644  1583 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:14.840061  9749 raft_consensus.cc:2804] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:03:14.840621  9749 raft_consensus.cc:697] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:14.841583  9749 consensus_queue.cc:237] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:14.846184  1584 consensus_peers.cc:597] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:14.846686  1584 consensus_peers.cc:597] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:14.849869  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85ae4f3774324fcf9745a42f30ffe08b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:14.850425  1686 raft_consensus.cc:3060] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 28 FOLLOWER]: Advancing to term 29
I20260709 13:03:14.854133  9744 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 29 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:14.874358  1684 raft_consensus.cc:1275] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 13:03:14.876806  9739 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:14.881084  9744 consensus_queue.cc:1048] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:03:14.887259  1686 raft_consensus.cc:2468] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 29.
W20260709 13:03:14.888005  1584 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:14.888281  9728 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
W20260709 13:03:14.895160  1584 leader_election.cc:336] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:14.894858  9728 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:14.898430  9728 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:14.901207  1582 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:14.902484  9744 raft_consensus.cc:2804] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 13:03:14.903251  9744 raft_consensus.cc:697] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 29 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:14.903760  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:14.904469  9744 consensus_queue.cc:237] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
W20260709 13:03:14.896441  1658 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:14.918960  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:14.919763  2230 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 18.
W20260709 13:03:14.933864  1584 consensus_peers.cc:597] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:14.935894  1658 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:14.948328  1584 consensus_peers.cc:597] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:14.950280  9746 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.950877  9746 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:14.951692  1684 raft_consensus.cc:1275] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:03:14.953030  9746 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:14.954504  9749 consensus_queue.cc:1048] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:03:14.975528  1438 catalog_manager.cc:5697] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 27 to 28, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 28 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.978348  1657 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:14.977337  1438 catalog_manager.cc:5697] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 26 to 27, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 27 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.980679  1438 catalog_manager.cc:5697] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 26 to 27, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 27 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.982331  1617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:14.984195  1438 catalog_manager.cc:5697] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 19 to 20, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 20 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:14.986989  9737 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:03:14.993079  9737 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:14.993559  9737 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:03:15.000067  2203 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:15.000638  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:14.987535  9749 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:15.005967  9749 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:15.011698  2235 raft_consensus.cc:1275] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 28 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 13:03:15.001998  9712 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:15.002081  9731 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:15.017760  9731 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:15.020534  9731 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:15.023169  9755 consensus_queue.cc:1048] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:15.025790  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:15.026750  1684 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 20.
I20260709 13:03:15.027776  1582 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 
I20260709 13:03:15.043632  9749 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:15.027837  9746 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:15.045029  9746 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:15.046593  1584 consensus_peers.cc:597] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.043692  9693 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:03:15.048193  9693 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:15.048661  9693 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:03:15.049722  1584 leader_election.cc:336] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:15.050110  2203 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:15.050513  1584 leader_election.cc:336] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:15.062896  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 49047.4us
I20260709 13:03:15.074513  9746 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:15.079542  9737 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:15.085350  2230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:15.090591  2236 raft_consensus.cc:1275] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 13:03:15.102291  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:15.102959  2236 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 25.
I20260709 13:03:15.105783  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:15.108155  1583 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:15.108901  1617 raft_consensus.cc:1275] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 13:03:15.128008  9737 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:15.148748  9748 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 13:03:15.149469  9748 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:15.149930  9748 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:03:15.222035  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 66967.4us
I20260709 13:03:15.238029  9720 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:15.248173  1658 leader_election.cc:336] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:15.248843  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:15.231832  1440 catalog_manager.cc:5697] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 28 to 29, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 29 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:15.276391  9772 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:15.278533  9772 consensus_queue.cc:1048] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:03:15.280104  1658 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:15.280989  1658 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:15.283252  9748 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:15.285034  1658 consensus_peers.cc:597] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.285766  2236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad1245d9a7bf4a21ac3d2291898dd976" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:15.286370  2236 raft_consensus.cc:3060] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:03:15.264133  9749 consensus_queue.cc:1048] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.025s
I20260709 13:03:15.296769  2236 raft_consensus.cc:2468] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 19.
W20260709 13:03:15.297883  1428 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 57136.8us
I20260709 13:03:15.259938  9693 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:15.323844  9693 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:15.352327  1658 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:15.353034  1428 catalog_manager.cc:4755] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): ChangeConfig:REMOVE_PEER RPC failed for tablet c65847e774e445909d83c69bf486b7fc: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:15.339087  9748 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:15.362753  1583 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:39293, user_credentials={real_user=slave}} blocked reactor thread for 120967us
I20260709 13:03:15.362334  1657 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:15.378481  9772 raft_consensus.cc:2804] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:03:15.379145  9772 raft_consensus.cc:697] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:15.380160  9772 consensus_queue.cc:237] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:15.390828  1584 consensus_peers.cc:597] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.391824  1584 leader_election.cc:336] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:15.392381  1584 consensus_peers.cc:597] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:15.392709  1658 consensus_peers.cc:597] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:15.392781  1584 consensus_peers.cc:597] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.404997  1686 raft_consensus.cc:1275] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 27 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 13:03:15.405304  1584 consensus_peers.cc:597] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:15.405905  1584 consensus_peers.cc:597] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:15.406635  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63835fd2de1c4dd4b0cea71fe7f09ed5" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:15.407294  1686 raft_consensus.cc:3060] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:03:15.418565  2235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed741b21bc0447a0896904a04e535f8a" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e"
I20260709 13:03:15.419265  2235 raft_consensus.cc:3060] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Advancing to term 26
W20260709 13:03:15.411954  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 159407us
I20260709 13:03:15.436941  1686 raft_consensus.cc:2468] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a6a5ae1b7f240408b3ddd30d8a18262 in term 21.
I20260709 13:03:15.463999  2229 raft_consensus.cc:1275] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:03:15.468106  9775 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:15.471575  9775 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:15.479027  2235 raft_consensus.cc:2468] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 984c9089d9804652846f3caeb609d56e in term 26.
I20260709 13:03:15.481148  9693 consensus_queue.cc:1048] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:03:15.499804  9775 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:15.505187  1584 consensus_peers.cc:597] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.523391  1657 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:15.524288  9737 raft_consensus.cc:2804] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Leader election won for term 26
I20260709 13:03:15.524823  9737 raft_consensus.cc:697] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 26 LEADER]: Becoming Leader. State: Replica: 984c9089d9804652846f3caeb609d56e, State: Running, Role: LEADER
I20260709 13:03:15.525750  9737 consensus_queue.cc:237] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:15.529433  9784 consensus_queue.cc:1048] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 13:03:15.534070  1584 consensus_peers.cc:597] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:15.560345  1582 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:15.563635  9782 raft_consensus.cc:2804] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:03:15.566953  2203 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:15.568986   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 1564)
I20260709 13:03:15.581964  1690 raft_consensus.cc:1275] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 29 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 13:03:15.581130  9782 raft_consensus.cc:697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Becoming Leader. State: Replica: 8a6a5ae1b7f240408b3ddd30d8a18262, State: Running, Role: LEADER
I20260709 13:03:15.618783  9719 consensus_queue.cc:1048] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.008s
W20260709 13:03:15.624562  2203 leader_election.cc:336] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:15.632560  9782 consensus_queue.cc:237] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:15.657960  1584 consensus_peers.cc:597] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:15.655830  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 54950.6us
I20260709 13:03:15.666513  9729 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:15.667147  9729 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:15.672698  9729 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:15.676353  2203 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:15.685204  1683 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:15.685930  1683 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 24.
I20260709 13:03:15.727659  1438 catalog_manager.cc:5697] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 18 to 19, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 19 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:03:15.731213  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 124331us
I20260709 13:03:15.733700  1438 catalog_manager.cc:5697] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e reported cstate change: term changed from 25 to 26, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 984c9089d9804652846f3caeb609d56e (127.0.107.195). New cstate: current_term: 26 leader_uuid: "984c9089d9804652846f3caeb609d56e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:15.734112  1441 catalog_manager.cc:5697] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 reported cstate change: term changed from 20 to 21, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194). New cstate: current_term: 21 leader_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:15.750085  1618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:15.750726  1618 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 21.
I20260709 13:03:15.762708  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" is_pre_election: true
I20260709 13:03:15.763362  1611 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 24.
W20260709 13:03:15.766881  1584 consensus_peers.cc:597] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:15.771268  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:15.772253  9786 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:03:15.772671  9786 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting leader election (detected failure of leader 83f6f5810950417195f707ff308a8125)
I20260709 13:03:15.773121  9786 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:15.785063  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
W20260709 13:03:15.827286  1658 consensus_peers.cc:597] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:15.828241  9775 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:03:15.828809  9775 raft_consensus.cc:493] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:15.829244  9775 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:15.835532  9786 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:15.839928  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55417.7us
I20260709 13:03:15.852514  2234 raft_consensus.cc:1275] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:03:15.860952  9786 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:15.866828  1584 consensus_peers.cc:597] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:15.871731  9737 consensus_queue.cc:1048] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:15.880014  9775 raft_consensus.cc:515] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:15.888625  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:15.899215  1611 raft_consensus.cc:3060] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:03:15.906868  1611 raft_consensus.cc:2468] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
I20260709 13:03:15.911108  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:15.911738  1686 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:15.924889  1686 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 25.
W20260709 13:03:15.935298  2203 leader_election.cc:336] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:15.939585  2203 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:15.943964  2203 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:03:15.946261  9775 leader_election.cc:290] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 election: Requested vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:15.976089  1611 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c65847e774e445909d83c69bf486b7fc" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262"
I20260709 13:03:15.979296  1611 raft_consensus.cc:3060] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:03:15.987139  1611 raft_consensus.cc:2468] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 25.
I20260709 13:03:16.012055  2203 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125
I20260709 13:03:16.012938  9775 raft_consensus.cc:2804] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:03:16.016834  2203 leader_election.cc:304] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
I20260709 13:03:16.018010  9727 raft_consensus.cc:2804] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:03:16.018520  9727 raft_consensus.cc:697] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 25 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:16.019378  9727 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:16.035137  1584 consensus_peers.cc:597] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:03:16.045240  9775 raft_consensus.cc:697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
W20260709 13:03:16.046836  1658 consensus_peers.cc:597] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:16.047883  9775 consensus_queue.cc:237] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:16.060796  2203 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:16.061558  2203 leader_election.cc:336] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:16.063525  2203 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:16.080382  2203 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:03:16.087167  1684 raft_consensus.cc:1275] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Refusing update from remote peer 8a6a5ae1b7f240408b3ddd30d8a18262: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:03:16.088703  1440 catalog_manager.cc:5697] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 23 to 25, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 25 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:16.091864  9787 consensus_queue.cc:1048] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:03:16.123241  1584 consensus_peers.cc:597] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:16.113225  1438 catalog_manager.cc:5697] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 21 to 22, leader changed from 83f6f5810950417195f707ff308a8125 (127.0.107.193) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 22 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:03:16.186574  2235 raft_consensus.cc:1275] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Refusing update from remote peer 984c9089d9804652846f3caeb609d56e: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 13:03:16.200664  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:03:16.213385  9739 consensus_queue.cc:1048] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [LEADER]: Connected to new peer: Peer: permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 13:03:16.227325  1584 consensus_peers.cc:597] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:16.245796  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:16.333580  2203 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:16.339178  1658 consensus_peers.cc:597] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:16.386299  2203 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:16.461369  2203 consensus_peers.cc:597] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:16.505038  1686 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:03:16.516438  9786 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:03:16.536197  1658 consensus_peers.cc:597] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:16.539057  2203 consensus_peers.cc:597] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:16.575349   431 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 1564)
I20260709 13:03:16.578027  1618 raft_consensus.cc:1275] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 13:03:16.581501  9712 consensus_queue.cc:1048] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:03:16.632987  1611 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 13:03:16.635211  9786 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:03:16.745862  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 55431.2us
W20260709 13:03:16.790096  1658 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:16.867274  1584 consensus_peers.cc:597] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:16.914721  1658 consensus_peers.cc:597] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.001730  1658 consensus_peers.cc:597] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.064157  1584 consensus_peers.cc:597] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.180032  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:17.304935  1658 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:17.325987  1584 consensus_peers.cc:597] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.362505  1584 consensus_peers.cc:597] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:17.363180  1584 consensus_peers.cc:597] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:03:17.448454  2230 consensus_queue.cc:237] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:17.469266  1686 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:03:17.477629  9786 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:03:17.481550  2203 consensus_peers.cc:597] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.482229  2203 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 13:03:17.493247  1617 raft_consensus.cc:1275] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Refusing update from remote peer cecad4f86a114a7696e398b3e8cc511e: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 13:03:17.498994  9819 consensus_queue.cc:1048] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:03:17.505456  9786 raft_consensus.cc:2955] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 25 LEADER]: Committing config change with OpId 25.14: config changed from index 4 to 14, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:03:17.521099  1658 consensus_peers.cc:597] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:17.519164  1617 raft_consensus.cc:2955] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Committing config change with OpId 25.14: config changed from index 4 to 14, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:17.530982  1686 raft_consensus.cc:2955] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Committing config change with OpId 25.14: config changed from index 4 to 14, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
I20260709 13:03:17.564173  1427 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c65847e774e445909d83c69bf486b7fc with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
W20260709 13:03:17.576067   431 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 1564)
I20260709 13:03:17.579275  1438 catalog_manager.cc:5697] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e reported cstate change: config changed from index 4 to 14, VOTER 83f6f5810950417195f707ff308a8125 (127.0.107.193) evicted. New cstate: current_term: 25 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } } }
W20260709 13:03:17.589541  1584 consensus_peers.cc:597] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.599501  1584 consensus_peers.cc:597] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:17.627022  1584 consensus_peers.cc:597] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.816195  1584 consensus_peers.cc:597] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.819654  2203 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:17.829263  1584 consensus_peers.cc:597] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.840045  1428 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 65031.9us
W20260709 13:03:17.843439  1428 catalog_manager.cc:4755] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c65847e774e445909d83c69bf486b7fc: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:17.837598  1658 consensus_peers.cc:597] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:17.869887  2203 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:17.878803  1584 consensus_peers.cc:597] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:17.959292  1584 consensus_peers.cc:597] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:17.959843  1584 consensus_peers.cc:597] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.010365  1584 consensus_peers.cc:597] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:18.109102  2203 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:18.144485  1658 consensus_peers.cc:597] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.189008  1584 consensus_peers.cc:597] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:18.242507  1584 consensus_peers.cc:597] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:18.262717  1658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111) [suppressed 457 similar messages]
W20260709 13:03:18.330895  1658 consensus_peers.cc:597] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.352304  1584 consensus_peers.cc:597] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:18.456676  2203 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:18.479786  1584 consensus_peers.cc:597] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:18.576426  2203 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:18.576929  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:18.577277   431 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 1564)
W20260709 13:03:18.579444  1584 consensus_peers.cc:597] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:18.579958  1584 consensus_peers.cc:597] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:18.621255  2203 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:18.644690  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:18.728058  2203 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:18.790861  2203 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:18.838935  1658 consensus_peers.cc:597] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:19.006974  1584 consensus_peers.cc:597] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:19.044147  2203 consensus_peers.cc:597] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:19.096721  1428 catalog_manager.cc:4755] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c65847e774e445909d83c69bf486b7fc: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:19.221696  1658 consensus_peers.cc:597] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:19.284966  1658 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:19.302145  2203 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:19.493846  1584 consensus_peers.cc:597] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:19.548300  1658 consensus_peers.cc:597] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:19.556558  1584 consensus_peers.cc:597] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:19.577862   431 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 1564)
W20260709 13:03:19.682392  1584 consensus_peers.cc:597] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:19.721088  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:19.721705  2203 consensus_peers.cc:597] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:19.750694  1658 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:19.753278  1584 consensus_peers.cc:597] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:19.755424  1658 consensus_peers.cc:597] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:19.957830  1584 consensus_peers.cc:597] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.008870  2203 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:20.036921  1584 consensus_peers.cc:597] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:20.173806  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 85714.2us
W20260709 13:03:20.216150  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 93545.2us
W20260709 13:03:20.270061  1584 consensus_peers.cc:597] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.264021  2203 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:20.302737  1584 consensus_peers.cc:597] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.334458  1584 consensus_peers.cc:597] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:20.335052  1584 consensus_peers.cc:597] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.353271  1658 consensus_peers.cc:597] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:20.356320  1658 consensus_peers.cc:597] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.405304  1584 consensus_peers.cc:597] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.405977  1584 consensus_peers.cc:597] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.419924  2203 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:20.421573  1428 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 115796us
W20260709 13:03:20.438084  1428 catalog_manager.cc:4755] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c65847e774e445909d83c69bf486b7fc: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:20.459609  1658 consensus_peers.cc:597] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.493245  1584 consensus_peers.cc:597] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:20.572451  1584 consensus_peers.cc:597] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.578382   431 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 1564)
W20260709 13:03:20.586622  1584 consensus_peers.cc:597] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:20.775303  2203 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:20.832018  1584 consensus_peers.cc:597] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:20.856841  1658 consensus_peers.cc:597] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:20.901582  2203 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:20.913120  1584 consensus_peers.cc:597] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:20.913961  1584 consensus_peers.cc:597] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:20.950627  2203 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:20.955739  2203 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:21.019541  1584 consensus_peers.cc:597] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:21.127365  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:21.209370  2203 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:21.293793  1584 consensus_peers.cc:597] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:21.310041  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:21.340438  2203 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:21.372164  2203 consensus_peers.cc:597] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:21.467013  1658 consensus_peers.cc:597] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:21.508488  1584 consensus_peers.cc:597] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:21.578919   431 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 1564)
W20260709 13:03:21.702608  1584 consensus_peers.cc:597] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:21.902015  1658 consensus_peers.cc:597] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:22.039721  2203 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:22.111316  1658 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.145787  2203 consensus_peers.cc:597] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.146625  1658 consensus_peers.cc:597] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.166474  1658 consensus_peers.cc:597] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.278838  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:22.301000  1584 consensus_peers.cc:597] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:22.406462  1584 consensus_peers.cc:597] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.422367  1584 consensus_peers.cc:597] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:22.475006  1658 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:22.505559  1584 consensus_peers.cc:597] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.541435  2203 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:22.571764  1584 consensus_peers.cc:597] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.579449   431 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 1564)
W20260709 13:03:22.592175  2203 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:22.612413  1428 catalog_manager.cc:4755] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c65847e774e445909d83c69bf486b7fc: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:22.631304  1658 consensus_peers.cc:597] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:22.633991  1584 consensus_peers.cc:597] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.656103  1658 consensus_peers.cc:597] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.659139  1584 consensus_peers.cc:597] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.873270  1584 consensus_peers.cc:597] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:22.885850  1584 consensus_peers.cc:597] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:22.893047  1584 consensus_peers.cc:597] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.917120  1658 consensus_peers.cc:597] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.950388  1584 consensus_peers.cc:597] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:22.968889  2203 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:23.007723  1584 consensus_peers.cc:597] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:23.129128  1584 consensus_peers.cc:597] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:23.249295  1584 consensus_peers.cc:597] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:23.255807  2203 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:23.260154  1584 consensus_peers.cc:597] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:23.270839  2203 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111) [suppressed 460 similar messages]
W20260709 13:03:23.273743  1584 consensus_peers.cc:597] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:23.321136  2203 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:23.434525  1584 consensus_peers.cc:597] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:23.445047  2203 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:23.498996  1584 consensus_peers.cc:597] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:23.517593  1658 consensus_peers.cc:597] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:23.579960   431 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 1564)
W20260709 13:03:23.664520  2203 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:23.665671  1584 consensus_peers.cc:597] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:23.666810  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 53013us
W20260709 13:03:23.724370  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:23.850272  2203 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:23.884191  2203 consensus_peers.cc:597] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:03:23.897492  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:23.898181  2203 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:24.025535  1584 consensus_peers.cc:597] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:24.051025  1658 consensus_peers.cc:597] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:24.262866  1584 consensus_peers.cc:597] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:24.489912  1658 consensus_peers.cc:597] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:24.580468   431 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 1564)
W20260709 13:03:24.613615  2203 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:24.627190  1658 consensus_peers.cc:597] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:24.694162  1584 consensus_peers.cc:597] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:24.715438  2203 consensus_peers.cc:597] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:24.748773  1658 consensus_peers.cc:597] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:24.755300  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:24.839514  1658 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:24.916950  1658 consensus_peers.cc:597] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:25.000624  1658 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:25.114938  2203 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:25.210266  1658 consensus_peers.cc:597] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:25.228220  2203 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:25.414186  1658 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:43887, user_credentials={real_user=slave}} blocked reactor thread for 74534.2us
W20260709 13:03:25.580991   431 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 1564)
W20260709 13:03:25.601805  1658 consensus_peers.cc:597] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:25.736876  2203 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:25.807492  1584 consensus_peers.cc:597] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:25.808511  1584 consensus_peers.cc:597] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:25.809043  1584 consensus_peers.cc:597] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:25.821051  1584 consensus_peers.cc:597] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:25.826366  1584 consensus_peers.cc:597] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:25.847545  1584 consensus_peers.cc:597] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:25.849560  2203 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:25.876515  1584 consensus_peers.cc:597] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:25.958861  2203 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:26.061048  2203 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:26.137689  1658 consensus_peers.cc:597] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:26.141297  1584 consensus_peers.cc:597] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:26.197985  1584 consensus_peers.cc:597] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:26.283862  1584 consensus_peers.cc:597] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:26.288535  1584 consensus_peers.cc:597] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:26.304922  2203 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:26.341225  1658 consensus_peers.cc:597] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:26.465950  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 54165.4us
W20260709 13:03:26.467053  1584 consensus_peers.cc:597] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:26.474066  1582 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 45431.3us
W20260709 13:03:26.441149  2203 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:26.581518   431 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 1564)
W20260709 13:03:26.628758  1584 consensus_peers.cc:597] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:26.665647  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:26.666229  2203 consensus_peers.cc:597] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:26.721493  1584 consensus_peers.cc:597] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:26.722656  1584 consensus_peers.cc:597] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:26.737232  1584 consensus_peers.cc:597] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:26.746649  1428 catalog_manager.cc:4755] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c65847e774e445909d83c69bf486b7fc: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:26.822551  1584 consensus_peers.cc:597] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:26.831712  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:26.835783  2203 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:27.053195  1584 consensus_peers.cc:597] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:03:27.123584  2203 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:27.162201  1658 consensus_peers.cc:597] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:27.211762  1584 consensus_peers.cc:597] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:27.225889  1658 consensus_peers.cc:597] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:27.277582  1658 consensus_peers.cc:597] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:27.425796  1658 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:27.426837  1658 consensus_peers.cc:597] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:03:27.461756  1430 maintenance_manager.cc:419] P 1ca716594c6348a883b88d48068b8416: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033374
W20260709 13:03:27.504841  1658 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:27.515277  1639 debug-util.cc:398] Leaking SignalData structure 0x7b08002c58e0 after lost signal to thread 434
W20260709 13:03:27.517619  1639 debug-util.cc:398] Leaking SignalData structure 0x7b08003715c0 after lost signal to thread 1475
W20260709 13:03:27.520346  1639 debug-util.cc:398] Leaking SignalData structure 0x7b0800058c60 after lost signal to thread 1642
W20260709 13:03:27.522460  1639 debug-util.cc:398] Leaking SignalData structure 0x7b08000594e0 after lost signal to thread 1716
W20260709 13:03:27.524916  1639 debug-util.cc:398] Leaking SignalData structure 0x7b0800329840 after lost signal to thread 2261
W20260709 13:03:27.563565  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:27.564380  2203 consensus_peers.cc:597] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:27.582386   431 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 1564)
W20260709 13:03:27.809803  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 62169.7us
W20260709 13:03:27.815002  1658 consensus_peers.cc:597] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:27.822459  2203 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:27.861526  1429 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.243s	user 0.005s	sys 0.001s
W20260709 13:03:27.870038  1584 consensus_peers.cc:597] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:27.880692  2203 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:27.905680  1584 consensus_peers.cc:597] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:03:27.946002  1429 maintenance_manager.cc:643] P 1ca716594c6348a883b88d48068b8416: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.483s	user 0.044s	sys 0.009s Metrics: {"bytes_written":7107,"delete_count":0,"lbm_write_time_us":1296,"lbm_writes_lt_1ms":6,"reinsert_count":0,"thread_start_us":242487,"threads_started":1,"update_count":288,"wal-append.queue_time_us":239379}
I20260709 13:03:27.954674  1430 maintenance_manager.cc:419] P 1ca716594c6348a883b88d48068b8416: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.347798
W20260709 13:03:27.973541  1584 consensus_peers.cc:597] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:28.068002  2203 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:03:28.116309  1429 maintenance_manager.cc:643] P 1ca716594c6348a883b88d48068b8416: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.161s	user 0.099s	sys 0.039s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":28339,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":29747,"dirs.run_cpu_time_us":580,"dirs.run_wall_time_us":1546,"lbm_read_time_us":2083,"lbm_reads_lt_1ms":25,"lbm_write_time_us":1714,"lbm_writes_lt_1ms":12,"peak_mem_usage":308495,"reinsert_count":0,"thread_start_us":36632,"threads_started":2,"update_count":303}
I20260709 13:03:28.119032  1430 maintenance_manager.cc:419] P 1ca716594c6348a883b88d48068b8416: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 13:03:28.119880  1429 maintenance_manager.cc:643] P 1ca716594c6348a883b88d48068b8416: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
W20260709 13:03:28.143445  1584 consensus_peers.cc:597] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:28.194074  1584 consensus_peers.cc:597] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:28.285861  2203 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:28.287420  1584 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111) [suppressed 419 similar messages]
W20260709 13:03:28.289753  1584 consensus_peers.cc:597] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:28.358018  1584 consensus_peers.cc:597] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:28.370028  2203 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:28.380743  1658 consensus_peers.cc:597] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:28.393602  2203 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:28.570876  1584 consensus_peers.cc:597] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:28.582897   431 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 1564)
W20260709 13:03:28.613302  1658 consensus_peers.cc:597] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:28.629118  1584 consensus_peers.cc:597] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:28.715829  1584 consensus_peers.cc:597] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:28.767076  1584 consensus_peers.cc:597] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:28.767621  1584 consensus_peers.cc:597] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:28.776427  2203 consensus_peers.cc:597] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:28.858465  1584 consensus_peers.cc:597] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:28.884640  1584 consensus_peers.cc:597] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:28.885257  1584 consensus_peers.cc:597] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:28.892791  2203 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:28.987287  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:28.990797  2203 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:29.042273  1584 consensus_peers.cc:597] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:29.046583  1658 consensus_peers.cc:597] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:29.090106  1584 consensus_peers.cc:597] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:29.109629  1584 consensus_peers.cc:597] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:29.323670  1584 consensus_peers.cc:597] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:29.436406  2203 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:29.568651  1584 consensus_peers.cc:597] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:03:29.571113  2203 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:29.634963  1584 consensus_peers.cc:597] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:29.682070  1658 consensus_peers.cc:597] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:29.686421  1658 consensus_peers.cc:597] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:03:29.719301   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:03:29.732040  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:29.739003  1658 consensus_peers.cc:597] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:29.830209  1658 consensus_peers.cc:597] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:29.982668  1658 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:30.058553  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:30.118474  2203 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:03:30.140884   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:03:30.152063  1658 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:30.208720  2203 consensus_peers.cc:597] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:30.221858  1658 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.227805  1658 consensus_peers.cc:597] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.232892  1658 consensus_peers.cc:597] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.265964  2203 consensus_peers.cc:597] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.266880  2203 consensus_peers.cc:597] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.285876  2203 consensus_peers.cc:597] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.292608  2203 consensus_peers.cc:597] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.330310  1658 consensus_peers.cc:597] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:03:30.343451  1584 consensus_peers.cc:597] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:30.348810  1658 consensus_peers.cc:597] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.349164  2203 consensus_peers.cc:597] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.350688  1658 consensus_peers.cc:597] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.356300  1658 consensus_peers.cc:597] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.357227  1658 consensus_peers.cc:597] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:30.357646   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:30.360380   431 tablet_replica.cc:333] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.362514   431 raft_consensus.cc:2243] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:30.363636   431 raft_consensus.cc:2272] T afa53a120a654b379c23e60544c86b5e P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.367228   431 tablet_replica.cc:333] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.370198   431 raft_consensus.cc:2243] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:30.428516   431 raft_consensus.cc:2272] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.431974   431 tablet_replica.cc:333] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
W20260709 13:03:30.432294  1584 consensus_peers.cc:597] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:30.438858  1658 consensus_peers.cc:597] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:30.440415   431 raft_consensus.cc:2243] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:30.442126   431 raft_consensus.cc:2272] T 2f07ede48ffe4a5eb0b162da083502c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.457136   431 tablet_replica.cc:333] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.458063   431 raft_consensus.cc:2243] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:30.458813   431 raft_consensus.cc:2272] T c65847e774e445909d83c69bf486b7fc P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:30.484903  1584 consensus_peers.cc:597] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:03:30.491379   431 tablet_replica.cc:333] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.495201   431 raft_consensus.cc:2243] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:30.496598   431 raft_consensus.cc:2272] T 960d36351aeb45a08c60a7d839c40e6d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.512532   431 tablet_replica.cc:333] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.518340   431 raft_consensus.cc:2243] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:30.532610   431 raft_consensus.cc:2272] T ebc9ea99300141e883fcf2f1b0d7a744 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.544437   431 tablet_replica.cc:333] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
W20260709 13:03:30.535240  1584 consensus_peers.cc:597] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:30.540274  2203 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:30.547128  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.547772  2203 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.548312  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.549577  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.550117  2203 consensus_peers.cc:597] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.550621  2203 consensus_peers.cc:597] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.551152  2203 consensus_peers.cc:597] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.562916  2203 consensus_peers.cc:597] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.563076  1584 consensus_peers.cc:597] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:03:30.566277   431 raft_consensus.cc:2243] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:30.566710  2203 consensus_peers.cc:597] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.567487  1658 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.568362  2203 consensus_peers.cc:597] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:30.569587   431 raft_consensus.cc:2272] T fb767ced66f24d579d3a9c116606ef2f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.580154   431 tablet_replica.cc:333] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.581009   431 raft_consensus.cc:2243] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:30.581097  1658 consensus_peers.cc:597] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:03:30.581796   431 raft_consensus.cc:2272] T b833a9bd5950418885dc620b52d4558f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.609247   431 tablet_replica.cc:333] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.616801   431 raft_consensus.cc:2243] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:30.634758  2203 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:30.635442   431 raft_consensus.cc:2272] T c601f0c0dfb54023907d2722495668cb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.639176   431 tablet_replica.cc:333] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.653347   431 raft_consensus.cc:2243] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:30.654331   431 raft_consensus.cc:2272] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.659927   431 tablet_replica.cc:333] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.660930   431 raft_consensus.cc:2243] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:30.661767   431 raft_consensus.cc:2272] T 13adc029302444abaaffd4bed11760a3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.664659   431 tablet_replica.cc:333] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.693583   431 raft_consensus.cc:2243] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:03:30.695490   431 raft_consensus.cc:2272] T da6820413cb240fd9a4bd8655e6762f7 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:30.697774  2203 consensus_peers.cc:597] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:30.698515  1584 consensus_peers.cc:597] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:03:30.702821   431 tablet_replica.cc:333] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
W20260709 13:03:30.727252  2203 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:30.729820   431 raft_consensus.cc:2243] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:30.734181   431 raft_consensus.cc:2272] T e4d8a1326e854e51b5539e7addc91fe5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.753038   431 tablet_replica.cc:333] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.754046   431 raft_consensus.cc:2243] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:30.754773   431 raft_consensus.cc:2272] T 8343e6d9eb5348b99ed8b5dc38a6087d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.785009   431 tablet_replica.cc:333] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
W20260709 13:03:30.810838  2203 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:30.823808   431 raft_consensus.cc:2243] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 28 LEADER]: Raft consensus shutting down.
W20260709 13:03:30.834625  1658 consensus_peers.cc:597] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:03:30.840090   431 raft_consensus.cc:2272] T 1a69ce55a73e464a962e15e73fd1db2c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.884611   431 tablet_replica.cc:333] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.888777   431 raft_consensus.cc:2243] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:03:30.923779   431 raft_consensus.cc:2272] T e1ef0a9c8bca48b6898df70834e97c4c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:30.939232  2203 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:30.941365   431 tablet_replica.cc:333] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.947026   431 raft_consensus.cc:2243] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:03:30.948341   431 raft_consensus.cc:2272] T 6087b5252cf44609bcf98bb0925b1666 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.980545   431 tablet_replica.cc:333] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:30.986133   431 raft_consensus.cc:2243] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:30.989717   431 raft_consensus.cc:2272] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:30.996155   431 tablet_replica.cc:333] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.012914   431 raft_consensus.cc:2243] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.024451   431 raft_consensus.cc:2272] T 3b24dad799cc469c9d10c4833dffe30c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.103858  9901 raft_consensus.cc:493] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:31.104796  9901 raft_consensus.cc:515] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:31.111399  1685 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21b84a6a3d5b4e8b95867a87260513a1" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:31.114830  9901 leader_election.cc:290] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:31.062151   431 tablet_replica.cc:333] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.116370   431 raft_consensus.cc:2243] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.122781   431 raft_consensus.cc:2272] T fa01a3d0bb094d0d80ca50f0287dbb27 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:31.124233  2203 consensus_peers.cc:597] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:03:31.184453   431 tablet_replica.cc:333] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.185626   431 raft_consensus.cc:2243] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.196393   431 raft_consensus.cc:2272] T e0058494c1f14737a2e0e2b8f4697969 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:31.171051  1584 leader_election.cc:336] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:31.203918  9901 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:31.204634  9901 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:31.209332  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:31.209843  9901 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:31.210526  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c53638cee15242e6a8f5dff3907d70c2" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:31.211155  1584 leader_election.cc:304] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:31.212179  1583 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:31.215014  9901 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:31.216832  9895 raft_consensus.cc:2749] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:31.219431  1584 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 48418.9us
W20260709 13:03:31.223867  1584 consensus_peers.cc:597] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:03:31.243608   431 tablet_replica.cc:333] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.252866   431 raft_consensus.cc:2243] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.255631  9895 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:31.266731  9895 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:31.262110  9897 raft_consensus.cc:493] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:31.270277  9897 raft_consensus.cc:515] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:31.272725  9897 leader_election.cc:290] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:31.273408  9895 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:31.275223  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:31.276304  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c52c22aef94ec6b8226f64dc4baca4" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:31.276577  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "214bc9e9284a4502844dbdc3395843fa" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:31.278399  1582 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:31.280436  9895 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:31.283736   431 raft_consensus.cc:2272] T 7ba0620abe844f5bb71ba9dd947c3b9c P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.296021  9895 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:31.296734  9895 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:31.298590   431 tablet_replica.cc:333] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.299574   431 raft_consensus.cc:2243] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.299927  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:31.300235   431 raft_consensus.cc:2272] T 4d7cdb06308e4f21a67e6a685abdc179 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.301081  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7154d65b3794a0e87f1ae971e939fec" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:31.302596  1583 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:31.303586  9897 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:31.300901  9895 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
W20260709 13:03:31.315018  1584 consensus_peers.cc:597] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:03:31.316273   431 tablet_replica.cc:333] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
W20260709 13:03:31.327769  1584 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:31.328477   431 raft_consensus.cc:2243] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.343672   431 raft_consensus.cc:2272] T bb1171fd6323498ea16dfea3e5356fe0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.348537   431 tablet_replica.cc:333] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.357826   431 raft_consensus.cc:2243] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 29 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.370879   431 raft_consensus.cc:2272] T 85ae4f3774324fcf9745a42f30ffe08b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:31.380982  2203 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:31.383591  9895 raft_consensus.cc:493] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:31.388919  9895 raft_consensus.cc:515] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:31.383699  9897 raft_consensus.cc:493] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:31.392445  9897 raft_consensus.cc:515] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:31.394371  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:31.395658  9895 leader_election.cc:290] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:31.398586  9897 leader_election.cc:290] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:31.400467  2234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ab2e02ed3c849e9bda035a57add301d" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:31.401233  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "532fefba367345759887da9fcc9872e3" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:31.402784  1582 leader_election.cc:304] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:31.403985  9897 raft_consensus.cc:2749] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:31.383386  1584 leader_election.cc:336] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:31.411046  1584 leader_election.cc:304] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:31.417971  9897 raft_consensus.cc:2749] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:03:31.421362   431 tablet_replica.cc:333] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.426772   431 raft_consensus.cc:2243] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:03:31.425340  1658 consensus_peers.cc:597] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:31.441601  9897 raft_consensus.cc:493] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:31.442252  9897 raft_consensus.cc:515] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:31.444922  9897 leader_election.cc:290] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:31.449080  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:31.451663   431 raft_consensus.cc:2272] T 12adfb7855f340329f110249c207f409 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.457471  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27181dd872d647748dc1fe8c2ac6c12b" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:31.461918  9897 raft_consensus.cc:493] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:31.462661  9897 raft_consensus.cc:515] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:31.462080  1583 leader_election.cc:304] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:31.464298  9895 raft_consensus.cc:2749] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:31.466518  1584 leader_election.cc:336] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:31.467280  1584 leader_election.cc:304] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:31.469578  9895 raft_consensus.cc:2749] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:31.472788  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:03:31.475310  9897 leader_election.cc:290] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:31.475850  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034eef09c5fd47c9b010158a104b4028" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:31.481681   431 tablet_replica.cc:333] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.486685  9895 raft_consensus.cc:493] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:31.487823  9895 raft_consensus.cc:515] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:31.490118  9895 leader_election.cc:290] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:31.487157  9897 raft_consensus.cc:493] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cecad4f86a114a7696e398b3e8cc511e)
I20260709 13:03:31.491932  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:31.491029  9897 raft_consensus.cc:515] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:31.495553  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eed65fd1add844e7821095e2215fb393" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:31.499105  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:31.501138  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caae9b2a426d42fb8c2353e1c978d0fb" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:31.501453  1582 leader_election.cc:304] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:31.502393  9897 leader_election.cc:290] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:31.503954  9901 raft_consensus.cc:2749] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:03:31.504118  1582 leader_election.cc:304] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:31.507444   431 raft_consensus.cc:2243] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.512871  9895 raft_consensus.cc:2749] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:03:31.515239  2203 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:31.519272  1584 leader_election.cc:336] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:31.522028  1584 leader_election.cc:304] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:31.526270  9901 raft_consensus.cc:2749] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:31.537346  1584 leader_election.cc:336] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:31.518956   431 raft_consensus.cc:2272] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.538820  9913 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:31.539674  9913 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:31.541734  9913 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:31.545653   431 tablet_replica.cc:333] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.549928   431 raft_consensus.cc:2243] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.557277   431 raft_consensus.cc:2272] T 034eef09c5fd47c9b010158a104b4028 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.567512   431 tablet_replica.cc:333] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.572760  9897 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:31.580487  9897 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:31.585440  9897 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:31.587663  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:31.592612  1684 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab5b971190d342d2b2568c30f3c0c81f" candidate_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:31.592656   431 raft_consensus.cc:2243] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.595271   431 raft_consensus.cc:2272] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.594285  1582 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 8a6a5ae1b7f240408b3ddd30d8a18262; no voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e
W20260709 13:03:31.598626  1582 raft_consensus.cc:2686] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 13:03:31.576830  2203 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:31.606583  1584 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P 8a6a5ae1b7f240408b3ddd30d8a18262 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:31.612463  2203 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:31.614601  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:31.613279   431 tablet_replica.cc:333] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.616859   431 raft_consensus.cc:2243] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.617808   431 raft_consensus.cc:2272] T 0a1244fbe0514ba59a0164883a315bbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:31.615321  1584 consensus_peers.cc:597] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:03:31.621371  9898 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:31.624195   431 tablet_replica.cc:333] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.686555   431 raft_consensus.cc:2243] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.687693   431 raft_consensus.cc:2272] T 22357a3949344a1792e5bae4a0291987 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.696208   431 tablet_replica.cc:333] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.697474   431 raft_consensus.cc:2243] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.698415   431 raft_consensus.cc:2272] T eed65fd1add844e7821095e2215fb393 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.701781   431 tablet_replica.cc:333] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.702677   431 raft_consensus.cc:2243] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.703330   431 raft_consensus.cc:2272] T e2c52c22aef94ec6b8226f64dc4baca4 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.706216   431 tablet_replica.cc:333] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.707002   431 raft_consensus.cc:2243] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.708321   431 raft_consensus.cc:2272] T c8377b7234e4472ea27005f16dede669 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.711691   431 tablet_replica.cc:333] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.712631   431 raft_consensus.cc:2243] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.713999   431 raft_consensus.cc:2272] T 969ec621630549bb9b288fdb50e46fbe P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.720788   431 tablet_replica.cc:333] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.721704   431 raft_consensus.cc:2243] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.722431   431 raft_consensus.cc:2272] T c7154d65b3794a0e87f1ae971e939fec P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.725520   431 tablet_replica.cc:333] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.726415   431 raft_consensus.cc:2243] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.727041   431 raft_consensus.cc:2272] T 27181dd872d647748dc1fe8c2ac6c12b P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.730062   431 tablet_replica.cc:333] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.730958   431 raft_consensus.cc:2243] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.731657   431 raft_consensus.cc:2272] T 214bc9e9284a4502844dbdc3395843fa P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.738739   431 tablet_replica.cc:333] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.745780   431 raft_consensus.cc:2243] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.754474   431 raft_consensus.cc:2272] T 532fefba367345759887da9fcc9872e3 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.757539   431 tablet_replica.cc:333] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.758425   431 raft_consensus.cc:2243] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.759610   431 raft_consensus.cc:2272] T c53638cee15242e6a8f5dff3907d70c2 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.763025   431 tablet_replica.cc:333] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.764237   431 raft_consensus.cc:2243] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.764952   431 raft_consensus.cc:2272] T 21b84a6a3d5b4e8b95867a87260513a1 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.767899   431 tablet_replica.cc:333] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.768760   431 raft_consensus.cc:2243] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.770211   431 raft_consensus.cc:2272] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.773277   431 tablet_replica.cc:333] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.774318   431 raft_consensus.cc:2243] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.782696   431 raft_consensus.cc:2272] T 88cbab659ea041148748114fae549ae0 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.796172   431 tablet_replica.cc:333] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.804067   431 raft_consensus.cc:2243] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.805824   431 raft_consensus.cc:2272] T 49db1a2fe0dd40eba2ac450e306a6937 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.809006   431 tablet_replica.cc:333] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.810153   431 raft_consensus.cc:2243] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.811133   431 raft_consensus.cc:2272] T caae9b2a426d42fb8c2353e1c978d0fb P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.814051   431 tablet_replica.cc:333] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.814904   431 raft_consensus.cc:2243] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:31.815622   431 raft_consensus.cc:2272] T 3ab2e02ed3c849e9bda035a57add301d P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.818856   431 tablet_replica.cc:333] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262: stopping tablet replica
I20260709 13:03:31.819671   431 raft_consensus.cc:2243] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:31.821090   431 raft_consensus.cc:2272] T 488823792cd54090a36c758519a28875 P 8a6a5ae1b7f240408b3ddd30d8a18262 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:31.837604  9914 raft_consensus.cc:493] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:31.838238  9914 raft_consensus.cc:515] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:31.840751  9914 leader_election.cc:290] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:31.859362  1658 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:31.860385  1658 leader_election.cc:336] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:31.860801  1658 leader_election.cc:304] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:31.861711  9914 raft_consensus.cc:2749] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:31.912992  2203 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:32.001616  9904 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.002215  9904 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:32.007372  9904 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:32.013177  2203 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:32.016397  9914 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.017026  9914 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:32.021948  9914 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:32.025537  1658 leader_election.cc:336] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:32.029316  1658 leader_election.cc:336] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:32.029871  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.031250  9914 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:32.044440  2203 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:03:32.045114  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.045989  9904 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 13:03:32.071084  2203 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:32.124939  9914 raft_consensus.cc:493] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.125581  9914 raft_consensus.cc:515] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:32.127915  9914 leader_election.cc:290] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:32.135912  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:32.156946  1658 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
W20260709 13:03:32.157939  1658 leader_election.cc:336] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:32.159529  1658 leader_election.cc:304] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.182767  9914 raft_consensus.cc:2749] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:32.189357  9914 raft_consensus.cc:493] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.189944  9914 raft_consensus.cc:515] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:32.192914  9914 leader_election.cc:290] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:32.223328  9898 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.223896  9898 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:32.226014  9898 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:32.227855  1658 leader_election.cc:336] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
W20260709 13:03:32.228634  1658 consensus_peers.cc:597] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:03:32.229048  1658 consensus_peers.cc:597] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:32.229454  1658 leader_election.cc:336] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:32.230175  1658 leader_election.cc:304] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.230904  9917 raft_consensus.cc:2749] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 13:03:32.262710  2203 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:32.263851  2203 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:32.264356  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.265116  9898 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:03:32.302083  9917 raft_consensus.cc:493] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.302717  9917 raft_consensus.cc:515] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:32.305011  9917 leader_election.cc:290] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:32.315025  1658 leader_election.cc:336] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:32.315683  1658 consensus_peers.cc:597] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:32.316349  1658 leader_election.cc:336] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:32.316754  1658 leader_election.cc:304] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.317540  9917 raft_consensus.cc:2749] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:32.333946  9917 raft_consensus.cc:493] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.334522  9917 raft_consensus.cc:515] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:32.336719  9917 leader_election.cc:290] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:32.346405  1658 leader_election.cc:336] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:32.347132  1658 leader_election.cc:336] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:32.347560  1658 leader_election.cc:304] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.349321  9917 raft_consensus.cc:2749] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:32.397199  2203 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:32.417650  9914 raft_consensus.cc:493] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.418308  9914 raft_consensus.cc:515] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:32.420603  9914 leader_election.cc:290] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:32.429603  1658 leader_election.cc:336] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:32.430923  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:32.431229  1658 leader_election.cc:336] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:32.431695  1658 leader_election.cc:304] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.433097  9914 raft_consensus.cc:2749] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:32.454962  1658 consensus_peers.cc:597] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:32.531073   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:03:32.550849  9898 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.551545  9898 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:32.553972  9898 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:32.558832  2203 leader_election.cc:336] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:32.559597  2203 leader_election.cc:336] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:32.559988  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.561071  9916 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:32.567322  9914 raft_consensus.cc:493] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.567899  9914 raft_consensus.cc:515] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:32.570261  9914 leader_election.cc:290] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:32.577232  2203 consensus_peers.cc:597] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.580163  1658 leader_election.cc:336] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:32.581027  1658 leader_election.cc:336] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:32.581543  1658 leader_election.cc:304] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.582412  9914 raft_consensus.cc:2749] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:03:32.605587  1658 consensus_peers.cc:597] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.619851  1658 consensus_peers.cc:597] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.676064  1658 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:03:32.681218  9917 raft_consensus.cc:493] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.681784  9917 raft_consensus.cc:515] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:32.683961  9917 leader_election.cc:290] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:32.688944  9906 raft_consensus.cc:493] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.689637  9906 raft_consensus.cc:515] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } }
I20260709 13:03:32.692118  9906 leader_election.cc:290] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:32.704073  1658 leader_election.cc:336] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:32.705338  1658 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.705858  1658 leader_election.cc:336] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:32.706326  1658 leader_election.cc:304] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.714854  9917 raft_consensus.cc:2749] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:32.721232  1658 leader_election.cc:336] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:32.722218  1658 leader_election.cc:336] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:32.722748  1658 leader_election.cc:304] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.723510  9917 raft_consensus.cc:2749] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 13:03:32.734325  2203 consensus_peers.cc:597] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.734879  2203 consensus_peers.cc:597] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.755293  2203 consensus_peers.cc:597] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:32.758137  9916 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.758733  9916 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:32.760911  9916 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:32.768579  1658 consensus_peers.cc:597] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:32.770277  2203 leader_election.cc:336] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
W20260709 13:03:32.772516  2203 leader_election.cc:336] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:32.772943  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.774387  9916 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:32.779564  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.787580  1658 consensus_peers.cc:597] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.802012  1658 consensus_peers.cc:597] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:32.806147  2203 consensus_peers.cc:597] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.813906  2203 consensus_peers.cc:597] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.842603  1658 consensus_peers.cc:597] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:32.844655  9917 raft_consensus.cc:493] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.845180  9917 raft_consensus.cc:515] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:32.847859  9917 leader_election.cc:290] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:32.882187  2203 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43459, user_credentials={real_user=slave}} blocked reactor thread for 50663.7us
W20260709 13:03:32.888083  1658 leader_election.cc:336] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:32.892591  2203 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.895181  1658 leader_election.cc:336] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:32.895818  1658 leader_election.cc:304] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.900764  9917 raft_consensus.cc:2749] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:32.904665  2203 consensus_peers.cc:597] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:32.906015  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.922624  1658 consensus_peers.cc:597] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:32.926229  2203 consensus_peers.cc:597] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:32.952695  9917 raft_consensus.cc:493] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:32.953291  9917 raft_consensus.cc:515] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:32.955789  9917 leader_election.cc:290] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:32.971383  1658 leader_election.cc:336] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:32.972416  1658 leader_election.cc:336] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:32.974206  1658 leader_election.cc:304] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:32.975852  9917 raft_consensus.cc:2749] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:32.978983  2203 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:32.995464  2203 consensus_peers.cc:597] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:33.005146  2203 consensus_peers.cc:597] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:33.018785  9917 raft_consensus.cc:493] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:33.019454  9917 raft_consensus.cc:515] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
W20260709 13:03:33.009145  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:33.024467  9906 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:33.025215  9906 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:33.025641  9917 leader_election.cc:290] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:33.027884  9906 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), cecad4f86a114a7696e398b3e8cc511e (127.0.107.196:39293)
I20260709 13:03:33.031836  9916 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:33.032061  2231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "984c9089d9804652846f3caeb609d56e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cecad4f86a114a7696e398b3e8cc511e" is_pre_election: true
I20260709 13:03:33.032424  9916 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:33.034740  9916 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:33.039371  1658 consensus_peers.cc:597] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:33.047842  9906 raft_consensus.cc:493] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:33.048439  9906 raft_consensus.cc:515] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:33.050527  9906 leader_election.cc:290] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:33.049204  2203 leader_election.cc:336] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:33.056385  2203 leader_election.cc:336] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:33.056898  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
W20260709 13:03:33.059732  1658 leader_election.cc:336] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:33.060395  9916 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:33.070399  1658 consensus_peers.cc:597] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:33.070945  1658 leader_election.cc:336] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:33.071364  1658 leader_election.cc:304] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:33.073264  9906 raft_consensus.cc:2749] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:03:33.073508  1658 leader_election.cc:336] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
W20260709 13:03:33.072379  2203 consensus_peers.cc:597] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:33.073939  1658 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, cecad4f86a114a7696e398b3e8cc511e
I20260709 13:03:33.074664  9906 raft_consensus.cc:2749] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:03:33.104290  2203 consensus_peers.cc:597] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:33.110622  1658 leader_election.cc:336] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:33.112624  1658 leader_election.cc:336] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:33.113188  1658 leader_election.cc:304] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:33.114089  9906 raft_consensus.cc:2749] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:03:33.124861  1658 consensus_peers.cc:597] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:33.134179  9919 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:33.134827  9919 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:33.137355  9919 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:33.145300  9919 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:33.145816  9919 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:33.147805  9919 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:33.152083  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e" is_pre_election: true
I20260709 13:03:33.152825  1686 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 22.
W20260709 13:03:33.155179  2203 leader_election.cc:336] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
W20260709 13:03:33.158330  2203 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:33.160172  2201 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:33.163487  9919 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:03:33.163898  9919 raft_consensus.cc:493] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:33.164281  9919 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:03:33.173759  2203 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:33.174332  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:33.175638  9916 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:33.188097  9919 raft_consensus.cc:515] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:33.191560  1686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "488823792cd54090a36c758519a28875" candidate_uuid: "cecad4f86a114a7696e398b3e8cc511e" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "984c9089d9804652846f3caeb609d56e"
I20260709 13:03:33.192236  1686 raft_consensus.cc:3060] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:03:33.219197  9919 leader_election.cc:290] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Requested vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:33.220068  1686 raft_consensus.cc:2468] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cecad4f86a114a7696e398b3e8cc511e in term 23.
I20260709 13:03:33.221959  2201 leader_election.cc:304] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 984c9089d9804652846f3caeb609d56e, cecad4f86a114a7696e398b3e8cc511e; no voters: 
W20260709 13:03:33.226307  2203 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:03:33.227974  9916 raft_consensus.cc:2804] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:03:33.228572  9916 raft_consensus.cc:697] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Becoming Leader. State: Replica: cecad4f86a114a7696e398b3e8cc511e, State: Running, Role: LEADER
I20260709 13:03:33.229614  9916 consensus_queue.cc:237] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:33.251917  1658 consensus_peers.cc:597] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:33.257368  1438 catalog_manager.cc:5697] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e reported cstate change: term changed from 22 to 23, leader changed from 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194) to cecad4f86a114a7696e398b3e8cc511e (127.0.107.196). New cstate: current_term: 23 leader_uuid: "cecad4f86a114a7696e398b3e8cc511e" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:03:33.277925  2203 leader_election.cc:336] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
W20260709 13:03:33.279072  2203 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:33.286195  2203 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111) [suppressed 578 similar messages]
W20260709 13:03:33.292788  2203 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:03:33.310009   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:03:33.326500  2201 consensus_peers.cc:597] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.330456  9920 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:33.331049  9920 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:33.333801  9920 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:33.353870  2203 consensus_peers.cc:597] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:33.354372  2203 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
W20260709 13:03:33.355158  2203 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:33.355630  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:33.360422  9920 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:33.371105  2201 consensus_peers.cc:597] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:33.409106  2201 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:33.426690  2201 consensus_peers.cc:597] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:33.436064  2201 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:33.447573  2201 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.451332   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:33.456342   431 tablet_replica.cc:333] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.457588   431 raft_consensus.cc:2243] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.461980   431 raft_consensus.cc:2272] T b7e984cd665a4768933ff09e8ba0e07b P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.465082   431 tablet_replica.cc:333] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.470160   431 raft_consensus.cc:2243] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.470979   431 raft_consensus.cc:2272] T c65847e774e445909d83c69bf486b7fc P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.483939   431 tablet_replica.cc:333] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.485910   431 raft_consensus.cc:2243] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.487241   431 raft_consensus.cc:2272] T a35934c9d0d04fbeac6fa3a58fa8fd63 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.490171   431 tablet_replica.cc:333] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.491055   431 raft_consensus.cc:2243] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.492015   431 raft_consensus.cc:2272] T 960d36351aeb45a08c60a7d839c40e6d P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.505846   431 tablet_replica.cc:333] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.506834   431 raft_consensus.cc:2243] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.508174   431 raft_consensus.cc:2272] T ebc9ea99300141e883fcf2f1b0d7a744 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.511152   431 tablet_replica.cc:333] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
W20260709 13:03:33.512926  2203 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:03:33.513125   431 raft_consensus.cc:2243] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.514034   431 raft_consensus.cc:2272] T 59d952214bc24204a149e6f1a9af3184 P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.516877   431 tablet_replica.cc:333] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
W20260709 13:03:33.520936  1658 consensus_peers.cc:597] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:03:33.527415   431 raft_consensus.cc:2243] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.528417   431 raft_consensus.cc:2272] T 81301f84b53f41349ac6bca78f1a77f4 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.531713   431 tablet_replica.cc:333] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
W20260709 13:03:33.531947  2201 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.532757   431 raft_consensus.cc:2243] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.544895   431 raft_consensus.cc:2272] T 0695e4119591411b8adab45b5925e8eb P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.548221   431 tablet_replica.cc:333] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.549065   431 raft_consensus.cc:2243] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.549691   431 raft_consensus.cc:2272] T da6820413cb240fd9a4bd8655e6762f7 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.552317   431 tablet_replica.cc:333] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.553094   431 raft_consensus.cc:2243] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus shutting down.
W20260709 13:03:33.560742  2201 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.566006   431 raft_consensus.cc:2272] T e4d8a1326e854e51b5539e7addc91fe5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.569082   431 tablet_replica.cc:333] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.569979   431 raft_consensus.cc:2243] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.570621   431 raft_consensus.cc:2272] T fb767ced66f24d579d3a9c116606ef2f P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:33.572497  2201 consensus_peers.cc:597] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.581722   431 tablet_replica.cc:333] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.582693   431 raft_consensus.cc:2243] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.583608   431 raft_consensus.cc:2272] T b833a9bd5950418885dc620b52d4558f P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.603662   431 tablet_replica.cc:333] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.604491   431 raft_consensus.cc:2243] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.605785   431 raft_consensus.cc:2272] T 67b2c0e67e8f4d0c9f640a2ce4fb6041 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.608585   431 tablet_replica.cc:333] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.609373   431 raft_consensus.cc:2243] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.618784   431 raft_consensus.cc:2272] T f03fb00780d54c5fbc0be2c8a1b8d4aa P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.621658   431 tablet_replica.cc:333] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
W20260709 13:03:33.622777  2201 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.623039   431 raft_consensus.cc:2243] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.623852   431 raft_consensus.cc:2272] T 3b24dad799cc469c9d10c4833dffe30c P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.626683   431 tablet_replica.cc:333] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.627503   431 raft_consensus.cc:2243] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.628155   431 raft_consensus.cc:2272] T fa01a3d0bb094d0d80ca50f0287dbb27 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:33.633069  2201 consensus_peers.cc:597] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:33.633780  2201 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:33.634351  2201 consensus_peers.cc:597] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:33.637229  2201 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.649724   431 tablet_replica.cc:333] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.650638   431 raft_consensus.cc:2243] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.652016   431 raft_consensus.cc:2272] T ad1245d9a7bf4a21ac3d2291898dd976 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.654749   431 tablet_replica.cc:333] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.655535   431 raft_consensus.cc:2243] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.656116   431 raft_consensus.cc:2272] T e0058494c1f14737a2e0e2b8f4697969 P 984c9089d9804652846f3caeb609d56e [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.658885   431 tablet_replica.cc:333] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.659663   431 raft_consensus.cc:2243] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.660382   431 raft_consensus.cc:2272] T 7ba0620abe844f5bb71ba9dd947c3b9c P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:33.662866  2201 consensus_peers.cc:597] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.671314   431 tablet_replica.cc:333] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.674054   431 raft_consensus.cc:2243] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.674926   431 raft_consensus.cc:2272] T 4d7cdb06308e4f21a67e6a685abdc179 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.677785   431 tablet_replica.cc:333] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.678519   431 raft_consensus.cc:2243] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.679199   431 raft_consensus.cc:2272] T 8343e6d9eb5348b99ed8b5dc38a6087d P 984c9089d9804652846f3caeb609d56e [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:33.679618  2201 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.681910   431 tablet_replica.cc:333] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.682744   431 raft_consensus.cc:2243] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.684418   431 raft_consensus.cc:2272] T ab5b971190d342d2b2568c30f3c0c81f P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.685582  9906 raft_consensus.cc:493] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:33.686105  9906 raft_consensus.cc:515] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } }
I20260709 13:03:33.688323  9906 leader_election.cc:290] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:33.694900  2203 consensus_peers.cc:597] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:03:33.696966  1658 leader_election.cc:336] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:33.698536   431 tablet_replica.cc:333] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
W20260709 13:03:33.699154  1658 leader_election.cc:336] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:33.699440   431 raft_consensus.cc:2243] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.699638  1658 leader_election.cc:304] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 984c9089d9804652846f3caeb609d56e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:33.700424   431 raft_consensus.cc:2272] T 0a1244fbe0514ba59a0164883a315bbe P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.700876  9906 raft_consensus.cc:2749] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:03:33.702781  2203 consensus_peers.cc:597] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.707929   431 tablet_replica.cc:333] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.708877   431 raft_consensus.cc:2243] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.709626   431 raft_consensus.cc:2272] T 6087b5252cf44609bcf98bb0925b1666 P 984c9089d9804652846f3caeb609d56e [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.712435   431 tablet_replica.cc:333] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.720440   431 raft_consensus.cc:2243] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.721133   431 raft_consensus.cc:2272] T 85ae4f3774324fcf9745a42f30ffe08b P 984c9089d9804652846f3caeb609d56e [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.724335   431 tablet_replica.cc:333] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.725203   431 raft_consensus.cc:2243] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.725950   431 raft_consensus.cc:2272] T 978c5dd19ae34aa4b4b42b8c3ccfb52a P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.728809   431 tablet_replica.cc:333] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
W20260709 13:03:33.738291  2201 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:03:33.754771  2201 consensus_peers.cc:597] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.761333   431 raft_consensus.cc:2243] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.762739   431 raft_consensus.cc:2272] T 034eef09c5fd47c9b010158a104b4028 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.765669   431 tablet_replica.cc:333] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.766469   431 raft_consensus.cc:2243] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.767158   431 raft_consensus.cc:2272] T 22357a3949344a1792e5bae4a0291987 P 984c9089d9804652846f3caeb609d56e [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:33.776871  2201 consensus_peers.cc:597] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.780763   431 tablet_replica.cc:333] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.781736   431 raft_consensus.cc:2243] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.782405   431 raft_consensus.cc:2272] T 380b2dca2259476bbe1f22b2eb110fd0 P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.785234   431 tablet_replica.cc:333] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.786072   431 raft_consensus.cc:2243] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.786685   431 raft_consensus.cc:2272] T eed65fd1add844e7821095e2215fb393 P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.789409   431 tablet_replica.cc:333] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.803493   431 raft_consensus.cc:2243] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.804430   431 raft_consensus.cc:2272] T 49db1a2fe0dd40eba2ac450e306a6937 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:33.805847  2201 consensus_peers.cc:597] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.817819   431 tablet_replica.cc:333] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.818782   431 raft_consensus.cc:2243] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.826660   431 raft_consensus.cc:2272] T c7154d65b3794a0e87f1ae971e939fec P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.829747   431 tablet_replica.cc:333] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.830600   431 raft_consensus.cc:2243] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.831919   431 raft_consensus.cc:2272] T 34b2ba71a23e4ba3becae445ee7598c5 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.838135   431 tablet_replica.cc:333] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.838912   431 raft_consensus.cc:2243] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.840415   431 raft_consensus.cc:2272] T 27181dd872d647748dc1fe8c2ac6c12b P 984c9089d9804652846f3caeb609d56e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.843286   431 tablet_replica.cc:333] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.844025   431 raft_consensus.cc:2243] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.844671   431 raft_consensus.cc:2272] T 474d3b57c33542aba459b97733122bf0 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.847355   431 tablet_replica.cc:333] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.848157   431 raft_consensus.cc:2243] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.849686   431 raft_consensus.cc:2272] T 488823792cd54090a36c758519a28875 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:33.853250  2201 consensus_peers.cc:597] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:33.854171   431 tablet_replica.cc:333] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.855050   431 raft_consensus.cc:2243] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.855913   431 raft_consensus.cc:2272] T caae9b2a426d42fb8c2353e1c978d0fb P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.858907   431 tablet_replica.cc:333] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.859628  9927 raft_consensus.cc:493] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:33.860389  9927 raft_consensus.cc:515] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:33.862697  9927 leader_election.cc:290] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:33.859728   431 raft_consensus.cc:2243] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.866462   431 raft_consensus.cc:2272] T c8377b7234e4472ea27005f16dede669 P 984c9089d9804652846f3caeb609d56e [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:33.868324  2203 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:33.877786  2203 leader_election.cc:336] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:33.878264  2203 leader_election.cc:304] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:33.879426  9927 raft_consensus.cc:2749] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 13:03:33.883332   431 tablet_replica.cc:333] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.883966   431 raft_consensus.cc:2243] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.884665   431 raft_consensus.cc:2272] T 3ce476a5b0534bc5b6915606f6d37f30 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.887264   431 tablet_replica.cc:333] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.887966   431 raft_consensus.cc:2243] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.889067   431 raft_consensus.cc:2272] T e2c52c22aef94ec6b8226f64dc4baca4 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.891850   431 tablet_replica.cc:333] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.892546   431 raft_consensus.cc:2243] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.893363   431 raft_consensus.cc:2272] T ae2bf844acee4ce5b28b92217256b084 P 984c9089d9804652846f3caeb609d56e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.896193   431 tablet_replica.cc:333] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.896874   431 raft_consensus.cc:2243] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 26 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.898164   431 raft_consensus.cc:2272] T ed741b21bc0447a0896904a04e535f8a P 984c9089d9804652846f3caeb609d56e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.900955   431 tablet_replica.cc:333] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.901664   431 raft_consensus.cc:2243] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.902390   431 raft_consensus.cc:2272] T 532fefba367345759887da9fcc9872e3 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.902544  9927 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:33.903129  9927 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:33.905021   431 tablet_replica.cc:333] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.905372  9927 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
I20260709 13:03:33.905805   431 raft_consensus.cc:2243] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.907460   431 raft_consensus.cc:2272] T c53638cee15242e6a8f5dff3907d70c2 P 984c9089d9804652846f3caeb609d56e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:33.910295   431 tablet_replica.cc:333] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.911026   431 raft_consensus.cc:2243] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:03:33.912262   431 raft_consensus.cc:2272] T 21b84a6a3d5b4e8b95867a87260513a1 P 984c9089d9804652846f3caeb609d56e [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:33.913373  2203 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:33.915146   431 tablet_replica.cc:333] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.915874   431 raft_consensus.cc:2243] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.916566   431 raft_consensus.cc:2272] T 63835fd2de1c4dd4b0cea71fe7f09ed5 P 984c9089d9804652846f3caeb609d56e [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:33.915086  2203 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:33.917613  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:33.918393  9927 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 13:03:33.921471   431 tablet_replica.cc:333] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e: stopping tablet replica
I20260709 13:03:33.922200   431 raft_consensus.cc:2243] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:33.923034   431 raft_consensus.cc:2272] T 674516b0482b47aaa9380469245f9d2e P 984c9089d9804652846f3caeb609d56e [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:33.953219  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:33.958989  2203 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:34.147949  2203 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:34.206874  2203 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:03:34.438172  9920 raft_consensus.cc:493] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:34.438735  9920 raft_consensus.cc:515] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:34.440876  9920 leader_election.cc:290] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:34.455606  2203 leader_election.cc:336] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:34.456894  2203 leader_election.cc:336] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:34.457299  2203 leader_election.cc:304] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:34.458127  9920 raft_consensus.cc:2749] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:03:34.487519  9928 raft_consensus.cc:493] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:34.488178  9928 raft_consensus.cc:515] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:34.490540  9928 leader_election.cc:290] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:34.498328  2203 leader_election.cc:336] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:34.502892  2203 leader_election.cc:336] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:34.503492  2203 leader_election.cc:304] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:34.504362  9928 raft_consensus.cc:2749] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:03:34.583319  2203 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:34.625523  2203 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:03:34.832682  9928 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:34.833307  9928 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:34.835638  9928 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:34.847704  2203 consensus_peers.cc:597] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:34.867007  2203 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
W20260709 13:03:34.871277  2203 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:34.871790  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:34.872871  9920 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:34.897588  9920 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:34.898195  9920 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:34.900485  9920 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:34.913444  9928 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:34.914072  9928 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:34.916477  9928 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:34.935400  2201 leader_election.cc:336] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111)
W20260709 13:03:34.939949  2201 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111)
I20260709 13:03:34.947290  9920 raft_consensus.cc:493] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:34.948052  9920 raft_consensus.cc:515] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:34.950768  9920 leader_election.cc:290] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:34.938184  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:03:34.954339  2203 leader_election.cc:336] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:34.954932  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:34.955799  9920 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:34.982635  2203 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:34.989675  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:34.993721  9920 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 13:03:35.009181  2203 leader_election.cc:336] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:35.012981  9920 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
W20260709 13:03:35.013085  1428 catalog_manager.cc:4755] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c65847e774e445909d83c69bf486b7fc: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:35.013638  9920 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:35.015810  9920 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:35.019594  2201 leader_election.cc:336] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111)
W20260709 13:03:35.023650  2203 leader_election.cc:336] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:35.024263  2203 leader_election.cc:304] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:35.025112  9920 raft_consensus.cc:2749] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:35.027820  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:35.042471  2203 leader_election.cc:336] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:35.043085  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:35.043924  9920 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:35.062425  2203 consensus_peers.cc:597] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.066351   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 1713)
I20260709 13:03:35.066778  9920 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:35.067348  9920 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:35.074644  2201 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111)
I20260709 13:03:35.075516  9920 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:35.081678  9920 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:35.082258  9920 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:35.090668  2201 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111)
I20260709 13:03:35.091612  9920 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:35.100214  2203 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:35.100840  2203 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:35.101686  9920 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:35.110522  2203 consensus_peers.cc:597] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:35.118422  9928 raft_consensus.cc:493] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:35.119060  9928 raft_consensus.cc:515] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:35.129871  2201 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111)
I20260709 13:03:35.132933  9928 leader_election.cc:290] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:35.144011  2203 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:35.144688  2203 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:35.145620  9928 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:35.156920  2203 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
W20260709 13:03:35.162202  2203 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:35.162802  2203 leader_election.cc:304] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:35.163661  9928 raft_consensus.cc:2749] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:03:35.178623  2203 leader_election.cc:336] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:35.189955  2203 consensus_peers.cc:597] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:35.222244  9928 raft_consensus.cc:493] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:35.222848  9928 raft_consensus.cc:515] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
W20260709 13:03:35.242833  2203 leader_election.cc:336] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:35.251106  9928 leader_election.cc:290] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:35.257692  2203 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.258859  2201 leader_election.cc:336] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111)
I20260709 13:03:35.259449  2201 leader_election.cc:304] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:35.262645  9928 raft_consensus.cc:2749] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 13:03:35.304325  2203 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:35.324790  2203 consensus_peers.cc:597] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.347430  2203 consensus_peers.cc:597] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.358556  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.455317  2201 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:33859, user_credentials={real_user=slave}} blocked reactor thread for 62680.4us
W20260709 13:03:35.491734  2203 consensus_peers.cc:597] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.529870  2203 consensus_peers.cc:597] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.536013  2203 consensus_peers.cc:597] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.553087  2203 consensus_peers.cc:597] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.580919  2203 consensus_peers.cc:597] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:35.615411  2203 consensus_peers.cc:597] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.621994  2203 consensus_peers.cc:597] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.658042  2203 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:35.667001  2203 consensus_peers.cc:597] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:03:35.739862  2203 consensus_peers.cc:597] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:03:35.818194  9920 raft_consensus.cc:493] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:35.818742  9920 raft_consensus.cc:515] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:35.821161  9920 leader_election.cc:290] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:35.838310  2201 consensus_peers.cc:597] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:35.844802  2201 consensus_peers.cc:597] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:35.857234  2203 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:35.858932  2203 leader_election.cc:336] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:35.859439  2203 leader_election.cc:304] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:35.860212  9920 raft_consensus.cc:2749] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:35.916962  2201 consensus_peers.cc:597] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:35.934023  2201 consensus_peers.cc:597] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:35.954885  2203 consensus_peers.cc:597] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:03:35.980676  2201 consensus_peers.cc:597] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.003279  2201 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.045596  2201 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.062675  2201 consensus_peers.cc:597] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.099289  2201 consensus_peers.cc:597] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.108172  2201 consensus_peers.cc:597] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.134357  2203 consensus_peers.cc:597] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:36.144544  2201 consensus_peers.cc:597] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.153744  2201 consensus_peers.cc:597] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.162741  2201 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.174798  2203 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:36.178261  2203 consensus_peers.cc:597] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:03:36.237614  2201 consensus_peers.cc:597] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:36.261178  9920 raft_consensus.cc:493] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:36.261842  9920 raft_consensus.cc:515] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:36.263764  9920 leader_election.cc:290] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:36.272694  2203 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:36.273487  2203 leader_election.cc:336] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:36.273918  2203 leader_election.cc:304] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:36.274673  9920 raft_consensus.cc:2749] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 13:03:36.287582  2201 consensus_peers.cc:597] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:36.287760   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
W20260709 13:03:36.332155  2203 consensus_peers.cc:597] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e -> Peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Couldn't send request to peer 8a6a5ae1b7f240408b3ddd30d8a18262. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.344094  2201 consensus_peers.cc:597] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.379626  2201 consensus_peers.cc:597] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.405807  1472 debug-util.cc:398] Leaking SignalData structure 0x7b080000b6c0 after lost signal to thread 434
W20260709 13:03:36.407099  1472 debug-util.cc:398] Leaking SignalData structure 0x7b08001beec0 after lost signal to thread 1475
W20260709 13:03:36.408753  1472 debug-util.cc:398] Leaking SignalData structure 0x7b08001667e0 after lost signal to thread 2261
W20260709 13:03:36.439005  2201 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.445279  2201 consensus_peers.cc:597] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.473033  2201 consensus_peers.cc:597] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:03:36.478446  2203 consensus_peers.cc:597] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20260709 13:03:36.543031  2201 consensus_peers.cc:597] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e -> Peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Couldn't send request to peer 984c9089d9804652846f3caeb609d56e. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:03:36.553063   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
W20260709 13:03:36.568200  2203 consensus_peers.cc:597] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:36.652233  9928 raft_consensus.cc:493] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:36.652856  9928 raft_consensus.cc:515] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:36.656589  9928 leader_election.cc:290] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:36.659588  9920 raft_consensus.cc:493] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:36.660141  9920 raft_consensus.cc:515] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:36.662215  9920 leader_election.cc:290] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:36.669616  2201 leader_election.cc:336] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111)
W20260709 13:03:36.675838  2203 leader_election.cc:336] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:36.676368  2203 leader_election.cc:304] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:36.677191  9928 raft_consensus.cc:2749] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:03:36.677604  2201 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111)
W20260709 13:03:36.687134  2203 leader_election.cc:336] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:36.687629  2203 leader_election.cc:304] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:36.688426  9928 raft_consensus.cc:2749] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 13:03:36.699402   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:03:36.702634   431 tablet_replica.cc:333] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.703989   431 raft_consensus.cc:2243] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:36.704736  9920 raft_consensus.cc:493] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 8a6a5ae1b7f240408b3ddd30d8a18262)
I20260709 13:03:36.705263  9920 raft_consensus.cc:515] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:36.707372  9920 leader_election.cc:290] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887)
W20260709 13:03:36.724261  2203 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
W20260709 13:03:36.725006  2203 leader_election.cc:336] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:36.725440  2203 leader_election.cc:304] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 8a6a5ae1b7f240408b3ddd30d8a18262
I20260709 13:03:36.726197  9920 raft_consensus.cc:2749] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:03:36.727871   431 raft_consensus.cc:2272] T ab5b971190d342d2b2568c30f3c0c81f P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.731407   431 tablet_replica.cc:333] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.732421   431 raft_consensus.cc:2243] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:36.733171   431 raft_consensus.cc:2272] T 88cbab659ea041148748114fae549ae0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.736380   431 tablet_replica.cc:333] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.737303   431 raft_consensus.cc:2243] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e [term 17 LEARNER]: Raft consensus shutting down.
I20260709 13:03:36.737869   431 pending_rounds.cc:70] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: Trying to abort 1 pending ops.
I20260709 13:03:36.738147   431 pending_rounds.cc:77] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e: Aborting op as it isn't in flight: id { term: 16 index: 8 } timestamp: 7305634469497479168 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "7ba0620abe844f5bb71ba9dd947c3b9c" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } } new_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } attrs { replace: true } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: NON_VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: true } } } }
I20260709 13:03:36.739596   431 raft_consensus.cc:2917] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e [term 17 LEARNER]: Aborting config change with OpId 16.8: Aborted: Op aborted
I20260709 13:03:36.740031   431 raft_consensus.cc:2889] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e [term 17 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:03:36.740396   431 raft_consensus.cc:2272] T 7ba0620abe844f5bb71ba9dd947c3b9c P cecad4f86a114a7696e398b3e8cc511e [term 17 NON_PARTICIPANT]: Raft consensus is shut down!
I20260709 13:03:36.743848   431 tablet_replica.cc:333] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.744786   431 raft_consensus.cc:2243] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.746066   431 raft_consensus.cc:2272] T 674516b0482b47aaa9380469245f9d2e P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.759583  9928 raft_consensus.cc:493] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:36.760281  9928 raft_consensus.cc:515] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } attrs { replace: true } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:36.763274  9928 leader_election.cc:290] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
W20260709 13:03:36.780699  2201 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111)
W20260709 13:03:36.781594  2203 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:36.782063  2203 leader_election.cc:304] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
W20260709 13:03:36.782955  2203 leader_election.cc:336] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:36.784103  9928 raft_consensus.cc:2749] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:03:36.756233   431 tablet_replica.cc:333] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.786038   431 raft_consensus.cc:2243] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:36.786878   431 raft_consensus.cc:2272] T 2f07ede48ffe4a5eb0b162da083502c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:36.805022  2203 consensus_peers.cc:597] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e -> Peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Couldn't send request to peer 83f6f5810950417195f707ff308a8125. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:03:36.812100   431 tablet_replica.cc:333] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.813156   431 raft_consensus.cc:2243] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.814464   431 raft_consensus.cc:2272] T b833a9bd5950418885dc620b52d4558f P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.817718   431 tablet_replica.cc:333] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.818691   431 raft_consensus.cc:2243] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.819952   431 raft_consensus.cc:2272] T 13adc029302444abaaffd4bed11760a3 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.844112   431 tablet_replica.cc:333] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.845037   431 raft_consensus.cc:2243] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.846524   431 pending_rounds.cc:70] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Trying to abort 1 pending ops.
I20260709 13:03:36.846803   431 pending_rounds.cc:77] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e: Aborting op as it isn't in flight: id { term: 23 index: 17 } timestamp: 7305634665398456320 op_type: NO_OP noop_request { }
I20260709 13:03:36.847187   431 raft_consensus.cc:2889] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:03:36.847566   431 raft_consensus.cc:2272] T 488823792cd54090a36c758519a28875 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.850998   431 tablet_replica.cc:333] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.851908   431 raft_consensus.cc:2243] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.853282   431 raft_consensus.cc:2272] T 474d3b57c33542aba459b97733122bf0 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.884244   431 tablet_replica.cc:333] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.885191   431 raft_consensus.cc:2243] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.886433   431 raft_consensus.cc:2272] T e4d8a1326e854e51b5539e7addc91fe5 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.889622   431 tablet_replica.cc:333] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.890478   431 raft_consensus.cc:2243] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:36.891148   431 raft_consensus.cc:2272] T 1a69ce55a73e464a962e15e73fd1db2c P cecad4f86a114a7696e398b3e8cc511e [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.895345   431 tablet_replica.cc:333] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.896247   431 raft_consensus.cc:2243] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.897778   431 raft_consensus.cc:2272] T caae9b2a426d42fb8c2353e1c978d0fb P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.901191   431 tablet_replica.cc:333] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.902097   431 raft_consensus.cc:2243] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.903589   431 raft_consensus.cc:2272] T 3ab2e02ed3c849e9bda035a57add301d P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.906677   431 tablet_replica.cc:333] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.907589   431 raft_consensus.cc:2243] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.908769   431 raft_consensus.cc:2272] T ae2bf844acee4ce5b28b92217256b084 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.909585  9920 raft_consensus.cc:493] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:36.910213  9920 raft_consensus.cc:515] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "83f6f5810950417195f707ff308a8125" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43459 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:36.912389  9920 leader_election.cc:290] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:36.912356   431 tablet_replica.cc:333] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.913751   431 raft_consensus.cc:2243] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.914732   431 raft_consensus.cc:2272] T b7e984cd665a4768933ff09e8ba0e07b P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.917717   431 tablet_replica.cc:333] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.918565   431 raft_consensus.cc:2243] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:36.919257   431 raft_consensus.cc:2272] T 34b2ba71a23e4ba3becae445ee7598c5 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.922376   431 tablet_replica.cc:333] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.923205   431 raft_consensus.cc:2243] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.924408   431 raft_consensus.cc:2272] T 3ce476a5b0534bc5b6915606f6d37f30 P cecad4f86a114a7696e398b3e8cc511e [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:36.927518  2203 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:36.929649  2201 leader_election.cc:336] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111)
I20260709 13:03:36.930126  2201 leader_election.cc:304] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 83f6f5810950417195f707ff308a8125, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:36.930861  9920 raft_consensus.cc:2749] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:03:36.934617   431 tablet_replica.cc:333] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.935612   431 raft_consensus.cc:2243] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:36.936272   431 raft_consensus.cc:2272] T e2c52c22aef94ec6b8226f64dc4baca4 P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.939456   431 tablet_replica.cc:333] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.940377   431 raft_consensus.cc:2243] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:36.941282   431 raft_consensus.cc:2272] T 969ec621630549bb9b288fdb50e46fbe P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.953967   431 tablet_replica.cc:333] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.959515   431 raft_consensus.cc:2243] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:36.960254   431 raft_consensus.cc:2272] T ed741b21bc0447a0896904a04e535f8a P cecad4f86a114a7696e398b3e8cc511e [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.963742   431 tablet_replica.cc:333] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.964677   431 raft_consensus.cc:2243] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.965920   431 raft_consensus.cc:2272] T c65847e774e445909d83c69bf486b7fc P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.968986   431 tablet_replica.cc:333] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.969898   431 raft_consensus.cc:2243] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:36.971866   431 raft_consensus.cc:2272] T bb1171fd6323498ea16dfea3e5356fe0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.978025   431 tablet_replica.cc:333] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.978917   431 raft_consensus.cc:2243] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:36.979720   431 raft_consensus.cc:2272] T 12adfb7855f340329f110249c207f409 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.982681   431 tablet_replica.cc:333] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.983495   431 raft_consensus.cc:2243] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.984794   431 raft_consensus.cc:2272] T fb767ced66f24d579d3a9c116606ef2f P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.988080   431 tablet_replica.cc:333] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.988938   431 raft_consensus.cc:2243] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:36.990473   431 raft_consensus.cc:2272] T 59d952214bc24204a149e6f1a9af3184 P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:36.996869   431 tablet_replica.cc:333] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:36.998915   431 raft_consensus.cc:2243] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:03:37.000277   431 raft_consensus.cc:2272] T afa53a120a654b379c23e60544c86b5e P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.003424   431 tablet_replica.cc:333] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.004369   431 raft_consensus.cc:2243] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:37.005131   431 raft_consensus.cc:2272] T c53638cee15242e6a8f5dff3907d70c2 P cecad4f86a114a7696e398b3e8cc511e [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.008724   431 tablet_replica.cc:333] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.009599   431 raft_consensus.cc:2243] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:37.013207   431 raft_consensus.cc:2272] T 532fefba367345759887da9fcc9872e3 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.016255   431 tablet_replica.cc:333] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.017089   431 raft_consensus.cc:2243] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:37.018393   431 raft_consensus.cc:2272] T 214bc9e9284a4502844dbdc3395843fa P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.021512   431 tablet_replica.cc:333] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.022310   431 raft_consensus.cc:2243] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:37.023444   431 raft_consensus.cc:2272] T 0a1244fbe0514ba59a0164883a315bbe P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.026678   431 tablet_replica.cc:333] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.027478   431 raft_consensus.cc:2243] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:37.028613   431 raft_consensus.cc:2272] T 4d7cdb06308e4f21a67e6a685abdc179 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.031611   431 tablet_replica.cc:333] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.032454   431 raft_consensus.cc:2243] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:37.033699   431 raft_consensus.cc:2272] T 380b2dca2259476bbe1f22b2eb110fd0 P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.036935   431 tablet_replica.cc:333] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.037781   431 raft_consensus.cc:2243] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:03:37.038893   431 raft_consensus.cc:2272] T eed65fd1add844e7821095e2215fb393 P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.041787   431 tablet_replica.cc:333] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.042518   431 raft_consensus.cc:2243] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:03:37.043658   431 raft_consensus.cc:2272] T 0695e4119591411b8adab45b5925e8eb P cecad4f86a114a7696e398b3e8cc511e [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.046383   431 tablet_replica.cc:333] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.047060   431 raft_consensus.cc:2243] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:03:37.048139   431 raft_consensus.cc:2272] T 81301f84b53f41349ac6bca78f1a77f4 P cecad4f86a114a7696e398b3e8cc511e [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.050920   431 tablet_replica.cc:333] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.051667   431 raft_consensus.cc:2243] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:37.052241   431 raft_consensus.cc:2272] T e1ef0a9c8bca48b6898df70834e97c4c P cecad4f86a114a7696e398b3e8cc511e [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.054875   431 tablet_replica.cc:333] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.055773  9920 raft_consensus.cc:493] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 984c9089d9804652846f3caeb609d56e)
I20260709 13:03:37.056318  9920 raft_consensus.cc:515] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "8a6a5ae1b7f240408b3ddd30d8a18262" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 43887 } } peers { permanent_uuid: "984c9089d9804652846f3caeb609d56e" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 33859 } } peers { permanent_uuid: "cecad4f86a114a7696e398b3e8cc511e" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 39293 } attrs { promote: false } }
I20260709 13:03:37.058487  9920 leader_election.cc:290] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887), 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859)
I20260709 13:03:37.055640   431 raft_consensus.cc:2243] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:37.064193   431 raft_consensus.cc:2272] T 27181dd872d647748dc1fe8c2ac6c12b P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:03:37.064443  2201 leader_election.cc:336] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111)
W20260709 13:03:37.066694  2203 leader_election.cc:336] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 8a6a5ae1b7f240408b3ddd30d8a18262 (127.0.107.194:43887): Network error: Client connection negotiation failed: client connection to 127.0.107.194:43887: connect: Connection refused (error 111)
I20260709 13:03:37.067186  2203 leader_election.cc:304] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cecad4f86a114a7696e398b3e8cc511e; no voters: 8a6a5ae1b7f240408b3ddd30d8a18262, 984c9089d9804652846f3caeb609d56e
I20260709 13:03:37.068401  9920 raft_consensus.cc:2749] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:03:37.067144   431 tablet_replica.cc:333] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.070173   431 raft_consensus.cc:2243] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:37.071056   431 raft_consensus.cc:2272] T c7154d65b3794a0e87f1ae971e939fec P cecad4f86a114a7696e398b3e8cc511e [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.073907   431 tablet_replica.cc:333] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.074577   431 raft_consensus.cc:2243] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:03:37.075631   431 raft_consensus.cc:2272] T c601f0c0dfb54023907d2722495668cb P cecad4f86a114a7696e398b3e8cc511e [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.078316   431 tablet_replica.cc:333] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.078938   431 raft_consensus.cc:2243] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:03:37.080076   431 raft_consensus.cc:2272] T 8343e6d9eb5348b99ed8b5dc38a6087d P cecad4f86a114a7696e398b3e8cc511e [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:37.082903   431 tablet_replica.cc:333] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e: stopping tablet replica
I20260709 13:03:37.083559   431 raft_consensus.cc:2243] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:03:37.084175   431 raft_consensus.cc:2272] T ad1245d9a7bf4a21ac3d2291898dd976 P cecad4f86a114a7696e398b3e8cc511e [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:38.376820   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 13:03:38.478379   431 master.cc:562] Master@127.0.107.254:36375 shutting down...
W20260709 13:03:38.534183  1489 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:03:38.534540  1489 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:03:38.534788  1489 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:03:41.788966  1426 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111) [suppressed 451 similar messages]
W20260709 13:03:41.792637  1426 catalog_manager.cc:4755] TS 984c9089d9804652846f3caeb609d56e (127.0.107.195:33859): ChangeConfig:REMOVE_PEER RPC failed for tablet ab5b971190d342d2b2568c30f3c0c81f: Network error: Client connection negotiation failed: client connection to 127.0.107.195:33859: connect: Connection refused (error 111)
W20260709 13:03:51.437274  1428 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
W20260709 13:03:51.441088  1428 catalog_manager.cc:4755] TS 83f6f5810950417195f707ff308a8125 (127.0.107.193:43459): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet c65847e774e445909d83c69bf486b7fc: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43459: connect: Connection refused (error 111)
I20260709 13:03:52.155467   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:03:52.156610   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:03:52.157029   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1ca716594c6348a883b88d48068b8416: stopping tablet replica
I20260709 13:03:52.358840   431 master.cc:584] Master@127.0.107.254:36375 shutdown complete.
I20260709 13:03:52.410149   431 test_util.cc:182] -----------------------------------------------
I20260709 13:03:52.410432   431 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783601935286233-431-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (273835 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:03:52.414280   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:44803
I20260709 13:03:52.415498   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:52.421739  9938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:52.422106  9941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:52.422464  9939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:52.423782   431 server_base.cc:1061] running on GCE node
I20260709 13:03:52.427105   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:52.427317   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:52.427439   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602232427428 us; error 0 us; skew 500 ppm
I20260709 13:03:52.427898   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:52.430466   431 webserver.cc:533] Webserver started at http://127.0.107.254:45451/ using document root <none> and password file <none>
I20260709 13:03:52.430936   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:52.431087   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:52.431308   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:52.432590   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/master-0-root/instance:
uuid: "51ac909067f74d7e8fa4b33a1a8523c7"
format_stamp: "Formatted at 2026-07-09 13:03:52 on dist-test-slave-jk05"
I20260709 13:03:52.437192   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:03:52.440723  9946 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:52.441571   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:03:52.441882   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/master-0-root
uuid: "51ac909067f74d7e8fa4b33a1a8523c7"
format_stamp: "Formatted at 2026-07-09 13:03:52 on dist-test-slave-jk05"
I20260709 13:03:52.442188   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:52.457715   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:52.459002   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:52.499257   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:44803
I20260709 13:03:52.499372  9997 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:44803 every 8 connection(s)
I20260709 13:03:52.503799  9998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:52.516541  9998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7: Bootstrap starting.
I20260709 13:03:52.522027  9998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:52.529325  9998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7: No bootstrap required, opened a new log
I20260709 13:03:52.532248  9998 raft_consensus.cc:359] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ac909067f74d7e8fa4b33a1a8523c7" member_type: VOTER }
I20260709 13:03:52.532698  9998 raft_consensus.cc:385] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:52.532882  9998 raft_consensus.cc:740] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ac909067f74d7e8fa4b33a1a8523c7, State: Initialized, Role: FOLLOWER
I20260709 13:03:52.533527  9998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ac909067f74d7e8fa4b33a1a8523c7" member_type: VOTER }
I20260709 13:03:52.533980  9998 raft_consensus.cc:399] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:03:52.534180  9998 raft_consensus.cc:493] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:03:52.534406  9998 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:52.539402  9998 raft_consensus.cc:515] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ac909067f74d7e8fa4b33a1a8523c7" member_type: VOTER }
I20260709 13:03:52.539943  9998 leader_election.cc:304] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51ac909067f74d7e8fa4b33a1a8523c7; no voters: 
I20260709 13:03:52.541101  9998 leader_election.cc:290] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:03:52.541626 10001 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:52.543066 10001 raft_consensus.cc:697] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [term 1 LEADER]: Becoming Leader. State: Replica: 51ac909067f74d7e8fa4b33a1a8523c7, State: Running, Role: LEADER
I20260709 13:03:52.543838 10001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ac909067f74d7e8fa4b33a1a8523c7" member_type: VOTER }
I20260709 13:03:52.544466  9998 sys_catalog.cc:565] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:03:52.550814 10002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51ac909067f74d7e8fa4b33a1a8523c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ac909067f74d7e8fa4b33a1a8523c7" member_type: VOTER } }
I20260709 13:03:52.550877 10003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51ac909067f74d7e8fa4b33a1a8523c7. Latest consensus state: current_term: 1 leader_uuid: "51ac909067f74d7e8fa4b33a1a8523c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ac909067f74d7e8fa4b33a1a8523c7" member_type: VOTER } }
I20260709 13:03:52.551719 10002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:52.551770 10003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [sys.catalog]: This master's current role is: LEADER
I20260709 13:03:52.558004 10009 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:03:52.563807 10009 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:03:52.565021   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:03:52.573630 10009 catalog_manager.cc:1383] Generated new cluster ID: 31a03cdce7664106a5a487a4399079e2
I20260709 13:03:52.573975 10009 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:03:52.602479 10009 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:03:52.603972 10009 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:03:52.620880 10009 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7: Generated new TSK 0
I20260709 13:03:52.621536 10009 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:03:52.633092   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:52.639859 10019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:52.640767 10020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:52.642191 10022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:52.643532   431 server_base.cc:1061] running on GCE node
I20260709 13:03:52.646230   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:52.646426   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:52.646641   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602232646604 us; error 0 us; skew 500 ppm
I20260709 13:03:52.647272   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:52.649806   431 webserver.cc:533] Webserver started at http://127.0.107.193:35629/ using document root <none> and password file <none>
I20260709 13:03:52.650321   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:52.650528   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:52.650799   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:52.651971   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-0-root/instance:
uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
format_stamp: "Formatted at 2026-07-09 13:03:52 on dist-test-slave-jk05"
I20260709 13:03:52.656738   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 13:03:52.660447 10027 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:52.661571   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:03:52.661934   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-0-root
uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
format_stamp: "Formatted at 2026-07-09 13:03:52 on dist-test-slave-jk05"
I20260709 13:03:52.662238   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:52.676605   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:52.677968   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:52.679518   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:52.682104   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:52.682317   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:52.682631   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:52.682829   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:52.727283   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:43611
I20260709 13:03:52.727398 10089 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:43611 every 8 connection(s)
I20260709 13:03:52.732928   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:52.741851 10094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:52.745460 10095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:52.745793 10097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:52.747288   431 server_base.cc:1061] running on GCE node
I20260709 13:03:52.750070   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:52.750376   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:52.750569   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602232750547 us; error 0 us; skew 500 ppm
I20260709 13:03:52.751161   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:52.753871 10090 heartbeater.cc:344] Connected to a master server at 127.0.107.254:44803
I20260709 13:03:52.754276 10090 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:52.755101   431 webserver.cc:533] Webserver started at http://127.0.107.194:39901/ using document root <none> and password file <none>
I20260709 13:03:52.755079 10090 heartbeater.cc:507] Master 127.0.107.254:44803 requested a full tablet report, sending...
I20260709 13:03:52.755893   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:52.756139   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:52.756418   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:52.757896  9963 ts_manager.cc:194] Registered new tserver with Master: 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:52.758260   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-1-root/instance:
uuid: "bfa051e18d0549eabe79dfcc7610d99b"
format_stamp: "Formatted at 2026-07-09 13:03:52 on dist-test-slave-jk05"
I20260709 13:03:52.760571  9963 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45994
I20260709 13:03:52.764331   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:03:52.768359 10102 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:52.769387   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:03:52.769816   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-1-root
uuid: "bfa051e18d0549eabe79dfcc7610d99b"
format_stamp: "Formatted at 2026-07-09 13:03:52 on dist-test-slave-jk05"
I20260709 13:03:52.770109   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:52.783278   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:52.784583   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:52.786140   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:52.788467   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:52.788730   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:52.788970   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:52.789144   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:52.831614   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:33711
I20260709 13:03:52.831732 10164 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:33711 every 8 connection(s)
I20260709 13:03:52.836863   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:52.844285 10168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:52.845532 10169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:52.848757 10172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:52.849623 10165 heartbeater.cc:344] Connected to a master server at 127.0.107.254:44803
I20260709 13:03:52.849965   431 server_base.cc:1061] running on GCE node
I20260709 13:03:52.850090 10165 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:52.851076 10165 heartbeater.cc:507] Master 127.0.107.254:44803 requested a full tablet report, sending...
I20260709 13:03:52.851224   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:52.851547   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:52.851733   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602232851715 us; error 0 us; skew 500 ppm
I20260709 13:03:52.852515   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:52.853304  9963 ts_manager.cc:194] Registered new tserver with Master: bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:52.855137  9963 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46010
I20260709 13:03:52.855195   431 webserver.cc:533] Webserver started at http://127.0.107.195:41837/ using document root <none> and password file <none>
I20260709 13:03:52.855818   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:52.856207   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:52.856501   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:52.858071   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-2-root/instance:
uuid: "bcf45a2176234db5957dc38e32554577"
format_stamp: "Formatted at 2026-07-09 13:03:52 on dist-test-slave-jk05"
I20260709 13:03:52.862757   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:03:52.866571 10176 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:52.867350   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.004s
I20260709 13:03:52.867692   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-2-root
uuid: "bcf45a2176234db5957dc38e32554577"
format_stamp: "Formatted at 2026-07-09 13:03:52 on dist-test-slave-jk05"
I20260709 13:03:52.868062   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:52.895025   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:52.896332   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:52.897958   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:52.900383   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:52.900632   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:52.900894   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:52.901096   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:52.944860   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:37673
I20260709 13:03:52.944969 10238 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:37673 every 8 connection(s)
I20260709 13:03:52.949904   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:03:52.960192 10242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:03:52.962214 10243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:52.963486 10239 heartbeater.cc:344] Connected to a master server at 127.0.107.254:44803
I20260709 13:03:52.963982 10239 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:52.965049 10239 heartbeater.cc:507] Master 127.0.107.254:44803 requested a full tablet report, sending...
I20260709 13:03:52.965938   431 server_base.cc:1061] running on GCE node
W20260709 13:03:52.966307 10245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:03:52.967383   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:03:52.967646   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:03:52.967721  9963 ts_manager.cc:194] Registered new tserver with Master: bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:52.967867   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602232967851 us; error 0 us; skew 500 ppm
I20260709 13:03:52.968814   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:03:52.969575  9963 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46016
I20260709 13:03:52.971767   431 webserver.cc:533] Webserver started at http://127.0.107.196:40919/ using document root <none> and password file <none>
I20260709 13:03:52.972519   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:03:52.972728   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:03:52.973018   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:03:52.974264   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-3-root/instance:
uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
format_stamp: "Formatted at 2026-07-09 13:03:52 on dist-test-slave-jk05"
I20260709 13:03:52.979070   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:03:52.982713 10250 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:52.983511   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:03:52.983857   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-3-root
uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
format_stamp: "Formatted at 2026-07-09 13:03:52 on dist-test-slave-jk05"
I20260709 13:03:52.984179   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:03:53.014917   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:03:53.016251   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:03:53.018221   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:03:53.020555   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:03:53.020797   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:53.021061   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:03:53.021240   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:03:53.066152   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.196:42219
I20260709 13:03:53.066237 10312 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.196:42219 every 8 connection(s)
I20260709 13:03:53.080884 10313 heartbeater.cc:344] Connected to a master server at 127.0.107.254:44803
I20260709 13:03:53.081284 10313 heartbeater.cc:461] Registering TS with master...
I20260709 13:03:53.082101 10313 heartbeater.cc:507] Master 127.0.107.254:44803 requested a full tablet report, sending...
I20260709 13:03:53.084226  9963 ts_manager.cc:194] Registered new tserver with Master: 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:53.084684   431 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015256618s
I20260709 13:03:53.085938  9963 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46024
I20260709 13:03:53.118903  9963 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46034:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:03:53.366734 10055 tablet_service.cc:1511] Processing CreateTablet for tablet 220a3db850dd4a5280bf7c8e9519d9d2 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:53.367164 10054 tablet_service.cc:1511] Processing CreateTablet for tablet ed4225af888b4e36ad1b03bdddc8f95f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:53.368343 10055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 220a3db850dd4a5280bf7c8e9519d9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.369006 10054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed4225af888b4e36ad1b03bdddc8f95f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.370638 10204 tablet_service.cc:1511] Processing CreateTablet for tablet 220a3db850dd4a5280bf7c8e9519d9d2 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:53.372247 10204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 220a3db850dd4a5280bf7c8e9519d9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.374596 10053 tablet_service.cc:1511] Processing CreateTablet for tablet e2ab3c2067a1463ca3f4319c644bd2f2 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:53.375304 10203 tablet_service.cc:1511] Processing CreateTablet for tablet e2ab3c2067a1463ca3f4319c644bd2f2 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:53.376191 10053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2ab3c2067a1463ca3f4319c644bd2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.376653 10203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2ab3c2067a1463ca3f4319c644bd2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.379707 10052 tablet_service.cc:1511] Processing CreateTablet for tablet 01f3be65cb0c478194f88fb404dd329f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:53.385030 10051 tablet_service.cc:1511] Processing CreateTablet for tablet 16541bea7ded48538adada244223f2c9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:53.386143 10052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01f3be65cb0c478194f88fb404dd329f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.388163 10051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16541bea7ded48538adada244223f2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.395670 10050 tablet_service.cc:1511] Processing CreateTablet for tablet 9a78dad3cd8b43a0bf32fc168eb162d4 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:53.399186 10049 tablet_service.cc:1511] Processing CreateTablet for tablet 0065434c541d4f10a459f9003a463d86 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:53.397341 10050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a78dad3cd8b43a0bf32fc168eb162d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.405999 10049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0065434c541d4f10a459f9003a463d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.410203 10202 tablet_service.cc:1511] Processing CreateTablet for tablet 8dc5cf0e36da48d0946769d2bb27c981 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:53.412178 10202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc5cf0e36da48d0946769d2bb27c981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.415369 10201 tablet_service.cc:1511] Processing CreateTablet for tablet 01f3be65cb0c478194f88fb404dd329f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:53.416947 10197 tablet_service.cc:1511] Processing CreateTablet for tablet c1e9969e98ab4be4942e5d55666fd86f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:53.418659 10199 tablet_service.cc:1511] Processing CreateTablet for tablet 9ea8a48dadff4881a7cdc0b27cf97db3 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:53.420933 10048 tablet_service.cc:1511] Processing CreateTablet for tablet 7939918ac27c421f99a6747c1ea4a994 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:53.421175 10198 tablet_service.cc:1511] Processing CreateTablet for tablet 0065434c541d4f10a459f9003a463d86 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:53.422412 10048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7939918ac27c421f99a6747c1ea4a994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.423403 10200 tablet_service.cc:1511] Processing CreateTablet for tablet 9a78dad3cd8b43a0bf32fc168eb162d4 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:53.416968 10201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01f3be65cb0c478194f88fb404dd329f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.425822 10197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1e9969e98ab4be4942e5d55666fd86f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.429811 10199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ea8a48dadff4881a7cdc0b27cf97db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.430959 10047 tablet_service.cc:1511] Processing CreateTablet for tablet c1e9969e98ab4be4942e5d55666fd86f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:53.432483 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1e9969e98ab4be4942e5d55666fd86f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.433959 10200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a78dad3cd8b43a0bf32fc168eb162d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.434623 10046 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0709ff33d24996ae638187a8e7c57f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:53.437919 10198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0065434c541d4f10a459f9003a463d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.441841 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0709ff33d24996ae638187a8e7c57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.442915 10196 tablet_service.cc:1511] Processing CreateTablet for tablet 22b9c093052a4414be72162a905b9a1d (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:53.444411 10196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22b9c093052a4414be72162a905b9a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.458983 10195 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0709ff33d24996ae638187a8e7c57f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:53.460619 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0709ff33d24996ae638187a8e7c57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.475946 10331 tablet_bootstrap.cc:492] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:53.484802 10055 tablet_service.cc:1511] Processing CreateTablet for tablet 711d81634fdc47aba353f79bcbfede48 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:53.486282 10055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 711d81634fdc47aba353f79bcbfede48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.501662 10054 tablet_service.cc:1511] Processing CreateTablet for tablet b84c07107a48406d9fd585e9982d3069 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:03:53.504472 10054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b84c07107a48406d9fd585e9982d3069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.508270 10047 tablet_service.cc:1511] Processing CreateTablet for tablet c29bf223e8224f72926bd39f6dcb3f08 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:53.516723 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c29bf223e8224f72926bd39f6dcb3f08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.510021 10052 tablet_service.cc:1511] Processing CreateTablet for tablet a68082d0468d49ada47812f6c8e3f106 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:53.520406 10331 tablet_bootstrap.cc:654] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:53.531843 10048 tablet_service.cc:1511] Processing CreateTablet for tablet ee9b66ba4b81428f896a77bd88bc7e4d (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:53.544629 10052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a68082d0468d49ada47812f6c8e3f106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.574193 10048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee9b66ba4b81428f896a77bd88bc7e4d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:53.577505 10011 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:53.582067 10200 tablet_service.cc:1511] Processing CreateTablet for tablet b84c07107a48406d9fd585e9982d3069 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
W20260709 13:03:53.576152  9948 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 72779.7us
I20260709 13:03:53.587627 10200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b84c07107a48406d9fd585e9982d3069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.574702 10051 tablet_service.cc:1511] Processing CreateTablet for tablet 6ea0b03477ff4d1ab861dc093edbe68a (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:53.615240 10201 tablet_service.cc:1511] Processing CreateTablet for tablet d4c2e323c7ec414eb53a9c1622b81dcc (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:53.615587 10198 tablet_service.cc:1511] Processing CreateTablet for tablet ee9b66ba4b81428f896a77bd88bc7e4d (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:53.617085 10201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4c2e323c7ec414eb53a9c1622b81dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.618040 10199 tablet_service.cc:1511] Processing CreateTablet for tablet c29bf223e8224f72926bd39f6dcb3f08 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:53.618409 10202 tablet_service.cc:1511] Processing CreateTablet for tablet 20d07303bc2640fe9d6ea20f824ce00e (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:53.619396 10199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c29bf223e8224f72926bd39f6dcb3f08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.620054 10202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20d07303bc2640fe9d6ea20f824ce00e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.631546 10333 tablet_bootstrap.cc:492] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:53.632344 10130 tablet_service.cc:1511] Processing CreateTablet for tablet 220a3db850dd4a5280bf7c8e9519d9d2 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:03:53.633085 10331 tablet_bootstrap.cc:492] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:53.633775 10331 ts_tablet_manager.cc:1403] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.158s	user 0.011s	sys 0.009s
I20260709 13:03:53.633769 10130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 220a3db850dd4a5280bf7c8e9519d9d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.642218 10197 tablet_service.cc:1511] Processing CreateTablet for tablet 2679daef0c4447c5823833ed4577441c (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:53.643216 10046 tablet_service.cc:1511] Processing CreateTablet for tablet d4c2e323c7ec414eb53a9c1622b81dcc (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:53.644832 10198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee9b66ba4b81428f896a77bd88bc7e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.647192 10203 tablet_service.cc:1511] Processing CreateTablet for tablet 6ea0b03477ff4d1ab861dc093edbe68a (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:53.648685 10203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea0b03477ff4d1ab861dc093edbe68a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.648437 10047 tablet_service.cc:1511] Processing CreateTablet for tablet 0d8d5b10e4fe4ce994c81bf45aa93591 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:53.649988 10051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea0b03477ff4d1ab861dc093edbe68a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.647184 10054 tablet_service.cc:1511] Processing CreateTablet for tablet bdf60e84958749f9826c9e8de7d266d9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:53.650940 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d8d5b10e4fe4ce994c81bf45aa93591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.652828 10129 tablet_service.cc:1511] Processing CreateTablet for tablet ed4225af888b4e36ad1b03bdddc8f95f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:53.654157 10054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdf60e84958749f9826c9e8de7d266d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.654292 10129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed4225af888b4e36ad1b03bdddc8f95f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.653991 10202 tablet_service.cc:1511] Processing CreateTablet for tablet ccc0007e852f46b2b3ce899d1101e17c (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:53.655503 10202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc0007e852f46b2b3ce899d1101e17c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.659988 10333 tablet_bootstrap.cc:654] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:53.663522 10128 tablet_service.cc:1511] Processing CreateTablet for tablet 8dc5cf0e36da48d0946769d2bb27c981 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:53.665107 10128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc5cf0e36da48d0946769d2bb27c981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.670017 10197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2679daef0c4447c5823833ed4577441c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.678609 10053 tablet_service.cc:1511] Processing CreateTablet for tablet 2c9785f60ee640d686f599267cf005fb (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:53.680052 10053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c9785f60ee640d686f599267cf005fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.687832 10127 tablet_service.cc:1511] Processing CreateTablet for tablet 01f3be65cb0c478194f88fb404dd329f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:03:53.689405 10127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01f3be65cb0c478194f88fb404dd329f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.689890 10047 tablet_service.cc:1511] Processing CreateTablet for tablet be75bbbdef504a5793ba0b07feaff6f8 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:53.694347 10049 tablet_service.cc:1511] Processing CreateTablet for tablet 0504e36ef8da48b189ebb41995e15efd (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:53.700573 10202 tablet_service.cc:1511] Processing CreateTablet for tablet bdf60e84958749f9826c9e8de7d266d9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:53.707850 10125 tablet_service.cc:1511] Processing CreateTablet for tablet 9ea8a48dadff4881a7cdc0b27cf97db3 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:53.710489 10126 tablet_service.cc:1511] Processing CreateTablet for tablet 16541bea7ded48538adada244223f2c9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:53.710939 10124 tablet_service.cc:1511] Processing CreateTablet for tablet 0065434c541d4f10a459f9003a463d86 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:03:53.713529 10123 tablet_service.cc:1511] Processing CreateTablet for tablet 7939918ac27c421f99a6747c1ea4a994 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:53.715773 10122 tablet_service.cc:1511] Processing CreateTablet for tablet 22b9c093052a4414be72162a905b9a1d (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:53.717974 10121 tablet_service.cc:1511] Processing CreateTablet for tablet 8f0709ff33d24996ae638187a8e7c57f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:03:53.722993 10125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ea8a48dadff4881a7cdc0b27cf97db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.725826 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f0709ff33d24996ae638187a8e7c57f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.728686 10123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7939918ac27c421f99a6747c1ea4a994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.730165 10202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdf60e84958749f9826c9e8de7d266d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.730297 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4c2e323c7ec414eb53a9c1622b81dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.730203 10126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16541bea7ded48538adada244223f2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.713551 10050 tablet_service.cc:1511] Processing CreateTablet for tablet dde8fc103c3d453abd4d298b1beffd74 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:53.742118 10335 tablet_bootstrap.cc:492] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:53.742558 10050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde8fc103c3d453abd4d298b1beffd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.744519 10195 tablet_service.cc:1511] Processing CreateTablet for tablet 2c9785f60ee640d686f599267cf005fb (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:53.756124 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be75bbbdef504a5793ba0b07feaff6f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.758970 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c9785f60ee640d686f599267cf005fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.763900 10090 heartbeater.cc:499] Master 127.0.107.254:44803 was elected leader, sending a full tablet report...
I20260709 13:03:53.764593 10196 tablet_service.cc:1511] Processing CreateTablet for tablet cd01deadf87945439c630e949d9c2e7b (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:53.766662 10130 tablet_service.cc:1511] Processing CreateTablet for tablet 0d8d5b10e4fe4ce994c81bf45aa93591 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:53.727066 10331 raft_consensus.cc:359] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:53.748284 10204 tablet_service.cc:1511] Processing CreateTablet for tablet aae810c860eb4b05a93e815248f988fc (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:53.749620 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22b9c093052a4414be72162a905b9a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.790499 10127 tablet_service.cc:1511] Processing CreateTablet for tablet 711d81634fdc47aba353f79bcbfede48 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:53.795198 10270 tablet_service.cc:1511] Processing CreateTablet for tablet 22b9c093052a4414be72162a905b9a1d (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:03:53.796221 10127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 711d81634fdc47aba353f79bcbfede48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.796675 10270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22b9c093052a4414be72162a905b9a1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.751698 10278 tablet_service.cc:1511] Processing CreateTablet for tablet ed4225af888b4e36ad1b03bdddc8f95f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:03:53.752249 10277 tablet_service.cc:1511] Processing CreateTablet for tablet e2ab3c2067a1463ca3f4319c644bd2f2 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:03:53.754741 10276 tablet_service.cc:1511] Processing CreateTablet for tablet 8dc5cf0e36da48d0946769d2bb27c981 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:03:53.797742 10130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d8d5b10e4fe4ce994c81bf45aa93591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.798458 10278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed4225af888b4e36ad1b03bdddc8f95f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.799108 10276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8dc5cf0e36da48d0946769d2bb27c981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.802058 10277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2ab3c2067a1463ca3f4319c644bd2f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.803458 10128 tablet_service.cc:1511] Processing CreateTablet for tablet 2679daef0c4447c5823833ed4577441c (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:53.772672 10275 tablet_service.cc:1511] Processing CreateTablet for tablet 16541bea7ded48538adada244223f2c9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:03:53.810490 10128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2679daef0c4447c5823833ed4577441c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.811189 10275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16541bea7ded48538adada244223f2c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.770735 10049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0504e36ef8da48b189ebb41995e15efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.812901 10196 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd01deadf87945439c630e949d9c2e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.815215 10199 tablet_service.cc:1511] Processing CreateTablet for tablet 102b42f582ce49458990caefcc8d451c (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:53.818207 10199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102b42f582ce49458990caefcc8d451c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.810518 10203 tablet_service.cc:1511] Processing CreateTablet for tablet be75bbbdef504a5793ba0b07feaff6f8 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:53.821153 10203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be75bbbdef504a5793ba0b07feaff6f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.829778 10204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aae810c860eb4b05a93e815248f988fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.779243 10274 tablet_service.cc:1511] Processing CreateTablet for tablet 9a78dad3cd8b43a0bf32fc168eb162d4 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:03:53.781898 10273 tablet_service.cc:1511] Processing CreateTablet for tablet 9ea8a48dadff4881a7cdc0b27cf97db3 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:03:53.834506 10274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9a78dad3cd8b43a0bf32fc168eb162d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.784874 10333 tablet_bootstrap.cc:492] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:53.791011 10272 tablet_service.cc:1511] Processing CreateTablet for tablet 7939918ac27c421f99a6747c1ea4a994 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:03:53.793716 10271 tablet_service.cc:1511] Processing CreateTablet for tablet c1e9969e98ab4be4942e5d55666fd86f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:03:53.807003 10269 tablet_service.cc:1511] Processing CreateTablet for tablet 711d81634fdc47aba353f79bcbfede48 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:03:53.807138 10046 tablet_service.cc:1511] Processing CreateTablet for tablet 080320e522f941c9961ebf9de652365a (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:53.841629 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 080320e522f941c9961ebf9de652365a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.832360 10200 tablet_service.cc:1511] Processing CreateTablet for tablet 080320e522f941c9961ebf9de652365a (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:53.769703 10331 raft_consensus.cc:385] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:53.774309 10048 tablet_service.cc:1511] Processing CreateTablet for tablet cd01deadf87945439c630e949d9c2e7b (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:53.845721 10124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0065434c541d4f10a459f9003a463d86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.858778 10052 tablet_service.cc:1511] Processing CreateTablet for tablet 940beae71dca43e58d3879c233f87dbb (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:53.860306 10052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940beae71dca43e58d3879c233f87dbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.861807 10165 heartbeater.cc:499] Master 127.0.107.254:44803 was elected leader, sending a full tablet report...
I20260709 13:03:53.849352 10333 ts_tablet_manager.cc:1403] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.218s	user 0.014s	sys 0.000s
I20260709 13:03:53.878163 10271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1e9969e98ab4be4942e5d55666fd86f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.847276 10331 raft_consensus.cc:740] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:53.888873 10055 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4f4fa6d20f4f1ca0b1befcc3a605b5 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:53.889671 10201 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4f4fa6d20f4f1ca0b1befcc3a605b5 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:53.891979 10272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7939918ac27c421f99a6747c1ea4a994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.897370 10273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ea8a48dadff4881a7cdc0b27cf97db3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.897707 10129 tablet_service.cc:1511] Processing CreateTablet for tablet c29bf223e8224f72926bd39f6dcb3f08 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:03:53.898715 10269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 711d81634fdc47aba353f79bcbfede48. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.899283 10129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c29bf223e8224f72926bd39f6dcb3f08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.902565 10048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd01deadf87945439c630e949d9c2e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.903541 10055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4f4fa6d20f4f1ca0b1befcc3a605b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.905243 10331 consensus_queue.cc:260] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:53.930303 10124 tablet_service.cc:1511] Processing CreateTablet for tablet 20d07303bc2640fe9d6ea20f824ce00e (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:53.931696 10124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20d07303bc2640fe9d6ea20f824ce00e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.950189 10197 tablet_service.cc:1511] Processing CreateTablet for tablet af4aeb2cefe7417e9c4364dd33b00847 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:03:53.958637 10331 ts_tablet_manager.cc:1434] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.324s	user 0.000s	sys 0.011s
I20260709 13:03:53.960000 10331 tablet_bootstrap.cc:492] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:53.943392 10200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 080320e522f941c9961ebf9de652365a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.969905 10197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af4aeb2cefe7417e9c4364dd33b00847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.970940 10201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4f4fa6d20f4f1ca0b1befcc3a605b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.972961 10239 heartbeater.cc:499] Master 127.0.107.254:44803 was elected leader, sending a full tablet report...
I20260709 13:03:53.983990 10123 tablet_service.cc:1511] Processing CreateTablet for tablet 6ea0b03477ff4d1ab861dc093edbe68a (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:03:53.928665 10127 tablet_service.cc:1511] Processing CreateTablet for tablet a68082d0468d49ada47812f6c8e3f106 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
W20260709 13:03:53.988893 10091 tablet.cc:2406] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:03:54.004271 10335 tablet_bootstrap.cc:654] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:54.013209 10122 tablet_service.cc:1511] Processing CreateTablet for tablet 0504e36ef8da48b189ebb41995e15efd (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:53.986559 10333 raft_consensus.cc:359] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.033702 10333 raft_consensus.cc:385] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:54.034065 10333 raft_consensus.cc:740] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:54.034966 10333 consensus_queue.cc:260] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.039765 10126 tablet_service.cc:1511] Processing CreateTablet for tablet ccc0007e852f46b2b3ce899d1101e17c (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:53.989809 10125 tablet_service.cc:1511] Processing CreateTablet for tablet be75bbbdef504a5793ba0b07feaff6f8 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:03:53.985414 10123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea0b03477ff4d1ab861dc093edbe68a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:53.999091 10331 tablet_bootstrap.cc:654] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:54.068109 10333 ts_tablet_manager.cc:1434] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.191s	user 0.004s	sys 0.003s
I20260709 13:03:54.054656 10277 tablet_service.cc:1511] Processing CreateTablet for tablet b84c07107a48406d9fd585e9982d3069 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:03:54.055706 10124 tablet_service.cc:1511] Processing CreateTablet for tablet bdf60e84958749f9826c9e8de7d266d9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:03:54.076256 10271 tablet_service.cc:1511] Processing CreateTablet for tablet ee9b66ba4b81428f896a77bd88bc7e4d (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:03:54.079612 10127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a68082d0468d49ada47812f6c8e3f106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.081779 10125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be75bbbdef504a5793ba0b07feaff6f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.088074 10130 tablet_service.cc:1511] Processing CreateTablet for tablet aae810c860eb4b05a93e815248f988fc (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:54.088925 10313 heartbeater.cc:499] Master 127.0.107.254:44803 was elected leader, sending a full tablet report...
I20260709 13:03:54.093281 10126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc0007e852f46b2b3ce899d1101e17c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.098554 10124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdf60e84958749f9826c9e8de7d266d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.103006 10333 tablet_bootstrap.cc:492] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:54.069329 10276 tablet_service.cc:1511] Processing CreateTablet for tablet a68082d0468d49ada47812f6c8e3f106 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:03:54.107493 10276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a68082d0468d49ada47812f6c8e3f106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.107963 10278 tablet_service.cc:1511] Processing CreateTablet for tablet 0504e36ef8da48b189ebb41995e15efd (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:03:54.110185 10271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee9b66ba4b81428f896a77bd88bc7e4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.110913 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0504e36ef8da48b189ebb41995e15efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.113935 10046 tablet_service.cc:1511] Processing CreateTablet for tablet 7c0cced7e11946f09a13c6adcf95833b (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:54.115439 10270 tablet_service.cc:1511] Processing CreateTablet for tablet d4c2e323c7ec414eb53a9c1622b81dcc (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:03:54.116333 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0cced7e11946f09a13c6adcf95833b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.116730 10270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4c2e323c7ec414eb53a9c1622b81dcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.121810 10130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aae810c860eb4b05a93e815248f988fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.122033 10275 tablet_service.cc:1511] Processing CreateTablet for tablet 2679daef0c4447c5823833ed4577441c (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:03:54.122777 10269 tablet_service.cc:1511] Processing CreateTablet for tablet 20d07303bc2640fe9d6ea20f824ce00e (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:03:54.123399 10275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2679daef0c4447c5823833ed4577441c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.138622 10338 tablet_bootstrap.cc:492] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:54.139964 10129 tablet_service.cc:1511] Processing CreateTablet for tablet 102b42f582ce49458990caefcc8d451c (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:54.140473 10272 tablet_service.cc:1511] Processing CreateTablet for tablet ccc0007e852f46b2b3ce899d1101e17c (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:03:54.141394 10129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102b42f582ce49458990caefcc8d451c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.142951 10278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0504e36ef8da48b189ebb41995e15efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.163450 10204 tablet_service.cc:1511] Processing CreateTablet for tablet 7c0cced7e11946f09a13c6adcf95833b (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:54.135406 10128 tablet_service.cc:1511] Processing CreateTablet for tablet dde8fc103c3d453abd4d298b1beffd74 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:54.170742 10128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde8fc103c3d453abd4d298b1beffd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.172026 10121 tablet_service.cc:1511] Processing CreateTablet for tablet 080320e522f941c9961ebf9de652365a (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:03:54.173769 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 080320e522f941c9961ebf9de652365a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.182780 10204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0cced7e11946f09a13c6adcf95833b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.184144 10331 tablet_bootstrap.cc:492] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:54.184789 10331 ts_tablet_manager.cc:1403] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.225s	user 0.014s	sys 0.001s
I20260709 13:03:54.188235 10331 raft_consensus.cc:359] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:54.189024 10331 raft_consensus.cc:385] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:54.197484 10338 tablet_bootstrap.cc:654] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:54.197980 10277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b84c07107a48406d9fd585e9982d3069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.205322 10339 raft_consensus.cc:493] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.205942 10339 raft_consensus.cc:515] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.219375 10333 tablet_bootstrap.cc:654] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:54.214232 10339 leader_election.cc:290] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:54.189321 10331 raft_consensus.cc:740] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:54.226815 10338 tablet_bootstrap.cc:492] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:54.226277 10331 consensus_queue.cc:260] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:54.227367 10338 ts_tablet_manager.cc:1403] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.089s	user 0.009s	sys 0.007s
I20260709 13:03:54.243628 10269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 20d07303bc2640fe9d6ea20f824ce00e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.247442 10273 tablet_service.cc:1511] Processing CreateTablet for tablet 0d8d5b10e4fe4ce994c81bf45aa93591 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:03:54.230955 10126 tablet_service.cc:1511] Processing CreateTablet for tablet 940beae71dca43e58d3879c233f87dbb (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:54.252592 10126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940beae71dca43e58d3879c233f87dbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.253082 10123 tablet_service.cc:1511] Processing CreateTablet for tablet af4aeb2cefe7417e9c4364dd33b00847 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:03:54.254457 10123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af4aeb2cefe7417e9c4364dd33b00847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.255366 10337 raft_consensus.cc:493] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.255483 10271 tablet_service.cc:1511] Processing CreateTablet for tablet aae810c860eb4b05a93e815248f988fc (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:03:54.256279 10337 raft_consensus.cc:515] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:54.243341 10331 ts_tablet_manager.cc:1434] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.058s	user 0.012s	sys 0.000s
I20260709 13:03:54.261489 10273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d8d5b10e4fe4ce994c81bf45aa93591. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.275780 10337 leader_election.cc:290] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:54.276602 10271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aae810c860eb4b05a93e815248f988fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.297384 10338 raft_consensus.cc:359] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.299474 10338 raft_consensus.cc:385] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:54.299919 10338 raft_consensus.cc:740] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:54.301169 10338 consensus_queue.cc:260] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.302416 10331 tablet_bootstrap.cc:492] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:54.298159 10274 tablet_service.cc:1511] Processing CreateTablet for tablet 2c9785f60ee640d686f599267cf005fb (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:03:54.312417 10278 tablet_service.cc:1511] Processing CreateTablet for tablet dde8fc103c3d453abd4d298b1beffd74 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:03:54.292855 10272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc0007e852f46b2b3ce899d1101e17c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.336030 10335 tablet_bootstrap.cc:492] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:54.345983 10048 tablet_service.cc:1511] Processing CreateTablet for tablet 737f9858e8dc4fd488d562cac2d54aa4 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:54.346258 10200 tablet_service.cc:1511] Processing CreateTablet for tablet f1aaf32c658c483eb6a4a3858745515f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:54.352643 10200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1aaf32c658c483eb6a4a3858745515f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.363088 10048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737f9858e8dc4fd488d562cac2d54aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.365774 10333 tablet_bootstrap.cc:492] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:54.366349 10333 ts_tablet_manager.cc:1403] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.264s	user 0.017s	sys 0.000s
I20260709 13:03:54.355496 10335 ts_tablet_manager.cc:1403] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.612s	user 0.027s	sys 0.017s
I20260709 13:03:54.369602 10333 raft_consensus.cc:359] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.359472 10274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c9785f60ee640d686f599267cf005fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.317338 10124 tablet_service.cc:1511] Processing CreateTablet for tablet 7c0cced7e11946f09a13c6adcf95833b (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:03:54.322597 10122 tablet_service.cc:1511] Processing CreateTablet for tablet 737f9858e8dc4fd488d562cac2d54aa4 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:54.373972 10333 raft_consensus.cc:385] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:54.378988 10197 tablet_service.cc:1511] Processing CreateTablet for tablet a9d3e10044ce408294e1b86ce960b84a (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:54.384950 10197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9d3e10044ce408294e1b86ce960b84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.385725 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737f9858e8dc4fd488d562cac2d54aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.381356 10129 tablet_service.cc:1511] Processing CreateTablet for tablet a9d3e10044ce408294e1b86ce960b84a (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:54.383227 10333 raft_consensus.cc:740] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:54.388473 10333 consensus_queue.cc:260] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.389830 10129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9d3e10044ce408294e1b86ce960b84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.394515 10333 ts_tablet_manager.cc:1434] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:03:54.395989 10333 tablet_bootstrap.cc:492] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:54.408118 10124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7c0cced7e11946f09a13c6adcf95833b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.409760 10278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dde8fc103c3d453abd4d298b1beffd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.413146 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ab3c2067a1463ca3f4319c644bd2f2" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:03:54.415230 10179 leader_election.cc:343] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:54.424669 10275 tablet_service.cc:1511] Processing CreateTablet for tablet cd01deadf87945439c630e949d9c2e7b (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:03:54.441481 10339 raft_consensus.cc:493] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.442066 10339 raft_consensus.cc:515] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.442378 10204 tablet_service.cc:1511] Processing CreateTablet for tablet 13fc2f8cffc54c41b5bdde0305fe868c (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:54.443704 10204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13fc2f8cffc54c41b5bdde0305fe868c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.444298 10339 leader_election.cc:290] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:54.446543 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:54.447216 10288 raft_consensus.cc:2468] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:54.448580 10179 leader_election.cc:304] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:54.449572 10339 raft_consensus.cc:2804] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:54.450034 10339 raft_consensus.cc:493] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.450402 10339 raft_consensus.cc:3060] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:54.447232 10130 tablet_service.cc:1511] Processing CreateTablet for tablet 13fc2f8cffc54c41b5bdde0305fe868c (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:54.454602 10130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13fc2f8cffc54c41b5bdde0305fe868c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.456868 10333 tablet_bootstrap.cc:654] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:54.430361 10331 tablet_bootstrap.cc:654] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:54.496775 10272 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4f4fa6d20f4f1ca0b1befcc3a605b5 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:03:54.463584 10046 tablet_service.cc:1511] Processing CreateTablet for tablet f1aaf32c658c483eb6a4a3858745515f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:54.498497 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1aaf32c658c483eb6a4a3858745515f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.500461 10274 tablet_service.cc:1511] Processing CreateTablet for tablet 737f9858e8dc4fd488d562cac2d54aa4 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:03:54.502471 10048 tablet_service.cc:1511] Processing CreateTablet for tablet bdc9831f2c60444db097f11766636c56 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:54.503823 10048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdc9831f2c60444db097f11766636c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.464658 10121 tablet_service.cc:1511] Processing CreateTablet for tablet 17c5f39847af4013aa501ce097f58b87 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:54.508581 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17c5f39847af4013aa501ce097f58b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.471011 10055 tablet_service.cc:1511] Processing CreateTablet for tablet 13fc2f8cffc54c41b5bdde0305fe868c (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:03:54.427047 10335 raft_consensus.cc:359] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:54.475065 10203 tablet_service.cc:1511] Processing CreateTablet for tablet bdc9831f2c60444db097f11766636c56 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:54.509307 10335 raft_consensus.cc:385] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:54.509732 10335 raft_consensus.cc:740] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:54.510210 10203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdc9831f2c60444db097f11766636c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.510512 10335 consensus_queue.cc:260] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:54.512166 10278 tablet_service.cc:1511] Processing CreateTablet for tablet f1aaf32c658c483eb6a4a3858745515f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:03:54.476696 10052 tablet_service.cc:1511] Processing CreateTablet for tablet 17c5f39847af4013aa501ce097f58b87 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:54.514258 10052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17c5f39847af4013aa501ce097f58b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.480971 10276 tablet_service.cc:1511] Processing CreateTablet for tablet 102b42f582ce49458990caefcc8d451c (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:03:54.482627 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ab3c2067a1463ca3f4319c644bd2f2" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
W20260709 13:03:54.519539 10178 leader_election.cc:343] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:54.520021 10178 leader_election.cc:304] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:54.520742 10200 tablet_service.cc:1511] Processing CreateTablet for tablet c6314d5916384527a155e2c956145d98 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:54.521723 10055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13fc2f8cffc54c41b5bdde0305fe868c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.522179 10200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6314d5916384527a155e2c956145d98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.482870 10339 raft_consensus.cc:515] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.527143 10339 leader_election.cc:290] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:54.527724 10339 raft_consensus.cc:2749] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:54.483623 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:54.483721 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "220a3db850dd4a5280bf7c8e9519d9d2" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:54.529140 10140 raft_consensus.cc:2468] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:03:54.486177 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "220a3db850dd4a5280bf7c8e9519d9d2" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
W20260709 13:03:54.531435 10028 leader_election.cc:343] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:54.533762 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:54.534571 10288 raft_consensus.cc:3060] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:54.537228 10030 leader_election.cc:304] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:03:54.538581 10337 raft_consensus.cc:2804] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:54.538947 10337 raft_consensus.cc:493] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:54.539287 10337 raft_consensus.cc:3060] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:54.487161 10269 tablet_service.cc:1511] Processing CreateTablet for tablet 940beae71dca43e58d3879c233f87dbb (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:03:54.516813 10271 tablet_service.cc:1511] Processing CreateTablet for tablet a9d3e10044ce408294e1b86ce960b84a (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:03:54.544697 10333 tablet_bootstrap.cc:492] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:54.545291 10333 ts_tablet_manager.cc:1403] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.150s	user 0.013s	sys 0.000s
I20260709 13:03:54.545085 10049 tablet_service.cc:1511] Processing CreateTablet for tablet 1ffb355c77dc43e78812c2b2a2df92c9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:54.546566 10049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ffb355c77dc43e78812c2b2a2df92c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.549362 10273 tablet_service.cc:1511] Processing CreateTablet for tablet 17c5f39847af4013aa501ce097f58b87 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:03:54.572304 10122 tablet_service.cc:1511] Processing CreateTablet for tablet c6314d5916384527a155e2c956145d98 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:54.574378 10122 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6314d5916384527a155e2c956145d98. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:54.584228 10178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 64863us
W20260709 13:03:54.585634 10178 leader_election.cc:343] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:54.586107 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
W20260709 13:03:54.588150 10178 leader_election.cc:343] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:54.588778 10011 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:54.590876 10333 raft_consensus.cc:359] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.591693 10333 raft_consensus.cc:385] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:54.592007 10333 raft_consensus.cc:740] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:54.598814 10197 tablet_service.cc:1511] Processing CreateTablet for tablet 1ffb355c77dc43e78812c2b2a2df92c9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:54.600268 10197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ffb355c77dc43e78812c2b2a2df92c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.601611 10047 tablet_service.cc:1511] Processing CreateTablet for tablet 027075b60dc34541a40249671ad2fcc9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:54.603132 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 027075b60dc34541a40249671ad2fcc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.605994 10277 tablet_service.cc:1511] Processing CreateTablet for tablet bdc9831f2c60444db097f11766636c56 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:03:54.494747 10270 tablet_service.cc:1511] Processing CreateTablet for tablet af4aeb2cefe7417e9c4364dd33b00847 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:03:54.434013 10338 ts_tablet_manager.cc:1434] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.204s	user 0.016s	sys 0.012s
I20260709 13:03:54.610669 10338 tablet_bootstrap.cc:492] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:54.465605 10275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd01deadf87945439c630e949d9c2e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.613772 10269 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 940beae71dca43e58d3879c233f87dbb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.615067 10273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17c5f39847af4013aa501ce097f58b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.615741 10272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4f4fa6d20f4f1ca0b1befcc3a605b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.618620 10124 tablet_service.cc:1511] Processing CreateTablet for tablet 1ffb355c77dc43e78812c2b2a2df92c9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:03:54.620072 10124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ffb355c77dc43e78812c2b2a2df92c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.621806 10276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 102b42f582ce49458990caefcc8d451c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.593047 10333 consensus_queue.cc:260] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.625023 10338 tablet_bootstrap.cc:654] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:54.625785 10274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 737f9858e8dc4fd488d562cac2d54aa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.629813 10271 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9d3e10044ce408294e1b86ce960b84a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.630854 10337 raft_consensus.cc:515] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:54.633780 10278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1aaf32c658c483eb6a4a3858745515f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.634178 10333 ts_tablet_manager.cc:1434] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.088s	user 0.009s	sys 0.001s
I20260709 13:03:54.634200 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "220a3db850dd4a5280bf7c8e9519d9d2" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:54.635422 10333 tablet_bootstrap.cc:492] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
W20260709 13:03:54.635804 10028 leader_election.cc:343] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:54.637091 10270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af4aeb2cefe7417e9c4364dd33b00847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.659883 10195 tablet_service.cc:1511] Processing CreateTablet for tablet 5893ec967cb241af8709bba918f3429a (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:03:54.661456 10195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5893ec967cb241af8709bba918f3429a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.659993 10130 tablet_service.cc:1511] Processing CreateTablet for tablet 027075b60dc34541a40249671ad2fcc9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:54.662606 10130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 027075b60dc34541a40249671ad2fcc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.662887 10277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdc9831f2c60444db097f11766636c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.666178 10288 raft_consensus.cc:2468] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:54.672667 10179 leader_election.cc:304] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:03:54.674144 10356 raft_consensus.cc:2804] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:54.674670 10356 raft_consensus.cc:697] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:03:54.675527 10356 consensus_queue.cc:237] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.689270 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "220a3db850dd4a5280bf7c8e9519d9d2" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:54.690053 10140 raft_consensus.cc:3060] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:54.694162 10047 tablet_service.cc:1511] Processing CreateTablet for tablet 5893ec967cb241af8709bba918f3429a (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:03:54.695916 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5893ec967cb241af8709bba918f3429a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.687027 10337 leader_election.cc:290] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:54.700897 10331 tablet_bootstrap.cc:492] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:54.701527 10331 ts_tablet_manager.cc:1403] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.412s	user 0.020s	sys 0.018s
I20260709 13:03:54.707885 10129 tablet_service.cc:1511] Processing CreateTablet for tablet b6c597501a7d4e89a7a60fe4a56fcef8 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:54.709442 10129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6c597501a7d4e89a7a60fe4a56fcef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.717458 10337 raft_consensus.cc:493] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.718040 10337 raft_consensus.cc:515] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:54.704795 10331 raft_consensus.cc:359] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:54.725720 10331 raft_consensus.cc:385] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:54.728055 10331 raft_consensus.cc:740] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:54.729364 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01f3be65cb0c478194f88fb404dd329f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:54.730703 10331 consensus_queue.cc:260] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
W20260709 13:03:54.731390 10030 leader_election.cc:343] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:54.723104 10140 raft_consensus.cc:2468] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:03:54.733552 10030 leader_election.cc:304] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:03:54.743184 10275 tablet_service.cc:1511] Processing CreateTablet for tablet 027075b60dc34541a40249671ad2fcc9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:03:54.744891 10275 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 027075b60dc34541a40249671ad2fcc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.748624 10276 tablet_service.cc:1511] Processing CreateTablet for tablet 5893ec967cb241af8709bba918f3429a (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:03:54.752660 10338 tablet_bootstrap.cc:492] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:54.753224 10338 ts_tablet_manager.cc:1403] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.143s	user 0.006s	sys 0.022s
I20260709 13:03:54.762620 10124 tablet_service.cc:1511] Processing CreateTablet for tablet 3384978cd09b40f29b899f22ded46604 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:54.763240 10200 tablet_service.cc:1511] Processing CreateTablet for tablet 3384978cd09b40f29b899f22ded46604 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:54.764644 10200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3384978cd09b40f29b899f22ded46604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.764686 10124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3384978cd09b40f29b899f22ded46604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.753299 10276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5893ec967cb241af8709bba918f3429a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.767975 10204 tablet_service.cc:1511] Processing CreateTablet for tablet b6c597501a7d4e89a7a60fe4a56fcef8 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:54.774118 10204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6c597501a7d4e89a7a60fe4a56fcef8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.775470 10274 tablet_service.cc:1511] Processing CreateTablet for tablet b6c597501a7d4e89a7a60fe4a56fcef8 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:03:54.776857 10274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6c597501a7d4e89a7a60fe4a56fcef8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:03:54.777712 10030 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:33711, user_credentials={real_user=slave}} blocked reactor thread for 44316.7us
I20260709 13:03:54.780206 10055 tablet_service.cc:1511] Processing CreateTablet for tablet 3384978cd09b40f29b899f22ded46604 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:03:54.789609 10337 leader_election.cc:290] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:54.791047 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01f3be65cb0c478194f88fb404dd329f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
W20260709 13:03:54.792732 10028 leader_election.cc:343] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:54.793196 10028 leader_election.cc:304] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:54.795518 10331 ts_tablet_manager.cc:1434] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.094s	user 0.016s	sys 0.006s
I20260709 13:03:54.795964 10333 tablet_bootstrap.cc:654] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:54.796267 10055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3384978cd09b40f29b899f22ded46604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.756690 10338 raft_consensus.cc:359] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.793612 10335 ts_tablet_manager.cc:1434] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.419s	user 0.012s	sys 0.015s
I20260709 13:03:54.808512 10126 tablet_service.cc:1511] Processing CreateTablet for tablet 1feaf9d0d9ca42c083b85e63a09d0081 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:54.809792 10360 raft_consensus.cc:2749] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:54.810410 10047 tablet_service.cc:1511] Processing CreateTablet for tablet 1feaf9d0d9ca42c083b85e63a09d0081 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:54.811800 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1feaf9d0d9ca42c083b85e63a09d0081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.813306  9962 catalog_manager.cc:5697] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:54.809981 10126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1feaf9d0d9ca42c083b85e63a09d0081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.821456 10331 tablet_bootstrap.cc:492] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:54.821626 10277 tablet_service.cc:1511] Processing CreateTablet for tablet c6314d5916384527a155e2c956145d98 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:03:54.823058 10277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6314d5916384527a155e2c956145d98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.817162 10335 tablet_bootstrap.cc:492] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:54.829234 10331 tablet_bootstrap.cc:654] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:54.803050 10337 raft_consensus.cc:2804] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:54.830042 10337 raft_consensus.cc:697] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:03:54.831044 10337 consensus_queue.cc:237] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:54.803467 10338 raft_consensus.cc:385] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:54.840986 10203 tablet_service.cc:1511] Processing CreateTablet for tablet dd372c208b444c4e9e4e02ee51c19601 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:54.843505 10338 raft_consensus.cc:740] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:54.844578 10338 consensus_queue.cc:260] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.849531 10203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd372c208b444c4e9e4e02ee51c19601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.851317 10338 ts_tablet_manager.cc:1434] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.098s	user 0.005s	sys 0.002s
I20260709 13:03:54.850986 10278 tablet_service.cc:1511] Processing CreateTablet for tablet dd372c208b444c4e9e4e02ee51c19601 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:54.837208 10274 tablet_service.cc:1511] Processing CreateTablet for tablet 1feaf9d0d9ca42c083b85e63a09d0081 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:03:54.877683 10278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd372c208b444c4e9e4e02ee51c19601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.878479 10274 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1feaf9d0d9ca42c083b85e63a09d0081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.882802 10046 tablet_service.cc:1511] Processing CreateTablet for tablet dd372c208b444c4e9e4e02ee51c19601 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:03:54.884298 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd372c208b444c4e9e4e02ee51c19601. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.886652 10335 tablet_bootstrap.cc:654] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:54.893834 10338 tablet_bootstrap.cc:492] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:54.895239 10129 tablet_service.cc:1511] Processing CreateTablet for tablet 1e4d469562c14664860f49baa277f1d9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:54.897861 10270 tablet_service.cc:1511] Processing CreateTablet for tablet 1e4d469562c14664860f49baa277f1d9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:54.899299 10270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e4d469562c14664860f49baa277f1d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.909155 10129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e4d469562c14664860f49baa277f1d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.918612 10047 tablet_service.cc:1511] Processing CreateTablet for tablet a8bc0e52b9424f6dab141d4c1e28023b (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:54.920157 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8bc0e52b9424f6dab141d4c1e28023b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.922518 10121 tablet_service.cc:1511] Processing CreateTablet for tablet a8bc0e52b9424f6dab141d4c1e28023b (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:54.923940 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8bc0e52b9424f6dab141d4c1e28023b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.931991 10197 tablet_service.cc:1511] Processing CreateTablet for tablet 1e4d469562c14664860f49baa277f1d9 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:03:54.932878 10200 tablet_service.cc:1511] Processing CreateTablet for tablet a8bc0e52b9424f6dab141d4c1e28023b (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:03:54.941015 10338 tablet_bootstrap.cc:654] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:54.946372 10331 tablet_bootstrap.cc:492] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:54.946980 10331 ts_tablet_manager.cc:1403] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.126s	user 0.018s	sys 0.004s
I20260709 13:03:54.933414 10197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e4d469562c14664860f49baa277f1d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.953622 10273 tablet_service.cc:1511] Processing CreateTablet for tablet 63e94ff7589e401ba16e6019e0a4d6eb (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:54.954510 10049 tablet_service.cc:1511] Processing CreateTablet for tablet 63e94ff7589e401ba16e6019e0a4d6eb (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:54.955186 10273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63e94ff7589e401ba16e6019e0a4d6eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.955857 10049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63e94ff7589e401ba16e6019e0a4d6eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.943202  9963 catalog_manager.cc:5697] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:54.964231 10200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8bc0e52b9424f6dab141d4c1e28023b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:54.970388 10356 raft_consensus.cc:493] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:54.971005 10356 raft_consensus.cc:515] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:54.973214 10356 leader_election.cc:290] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:54.975561 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:54.977705 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:54.978231 10331 raft_consensus.cc:359] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
W20260709 13:03:54.978920 10178 leader_election.cc:343] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:54.980160 10179 leader_election.cc:343] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:54.980623 10179 leader_election.cc:304] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:54.981823 10356 raft_consensus.cc:2749] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:54.979063 10331 raft_consensus.cc:385] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:54.989264 10333 tablet_bootstrap.cc:492] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:54.990020 10333 ts_tablet_manager.cc:1403] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.355s	user 0.005s	sys 0.019s
I20260709 13:03:54.990757 10335 tablet_bootstrap.cc:492] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:54.991467 10335 ts_tablet_manager.cc:1403] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.175s	user 0.013s	sys 0.006s
I20260709 13:03:55.002257 10333 raft_consensus.cc:359] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.003554 10333 raft_consensus.cc:385] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.004037 10333 raft_consensus.cc:740] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.005168 10333 consensus_queue.cc:260] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:54.983129 10331 raft_consensus.cc:740] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.044332 10356 raft_consensus.cc:493] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.044839 10356 raft_consensus.cc:515] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.044896 10331 consensus_queue.cc:260] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.047408 10331 ts_tablet_manager.cc:1434] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.100s	user 0.007s	sys 0.000s
I20260709 13:03:55.048396 10331 tablet_bootstrap.cc:492] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:55.049201 10356 leader_election.cc:290] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:55.015215 10337 raft_consensus.cc:493] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.049945 10337 raft_consensus.cc:515] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.051460 10333 ts_tablet_manager.cc:1434] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.000s
I20260709 13:03:55.052258 10337 leader_election.cc:290] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:55.052567 10333 tablet_bootstrap.cc:492] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:54.987226 10124 tablet_service.cc:1511] Processing CreateTablet for tablet 63e94ff7589e401ba16e6019e0a4d6eb (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:03:55.054112 10124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63e94ff7589e401ba16e6019e0a4d6eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.021385 10338 tablet_bootstrap.cc:492] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:55.021409 10335 raft_consensus.cc:359] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.032333 10203 tablet_service.cc:1511] Processing CreateTablet for tablet 4d87b05b458741adb45ae4a1753447d0 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:03:55.058323 10335 raft_consensus.cc:385] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.058771 10335 raft_consensus.cc:740] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.059183 10203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d87b05b458741adb45ae4a1753447d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.059603 10335 consensus_queue.cc:260] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.042728 10270 tablet_service.cc:1511] Processing CreateTablet for tablet 4d87b05b458741adb45ae4a1753447d0 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:03:55.058461 10338 ts_tablet_manager.cc:1403] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.165s	user 0.026s	sys 0.036s
I20260709 13:03:55.062314 10270 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d87b05b458741adb45ae4a1753447d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.067322 10335 ts_tablet_manager.cc:1434] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.075s	user 0.010s	sys 0.000s
I20260709 13:03:55.065755 10338 raft_consensus.cc:359] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.068063 10338 raft_consensus.cc:385] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.068380 10338 raft_consensus.cc:740] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.068702 10335 tablet_bootstrap.cc:492] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:55.069196 10338 consensus_queue.cc:260] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.071897 10338 ts_tablet_manager.cc:1434] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 13:03:55.073196 10338 tablet_bootstrap.cc:492] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:55.093281 10333 tablet_bootstrap.cc:654] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.095402 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed4225af888b4e36ad1b03bdddc8f95f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:55.100991 10273 tablet_service.cc:1511] Processing CreateTablet for tablet b88ba89b2d1849848da78b9b3747e66e (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:55.102499 10273 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88ba89b2d1849848da78b9b3747e66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.106978 10278 tablet_service.cc:1511] Processing CreateTablet for tablet 96acb559acc745428add2c682cf83cfc (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:55.107251 10129 tablet_service.cc:1511] Processing CreateTablet for tablet 60fa8e2d405c485dbf19dff403d2328f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:55.107620 10121 tablet_service.cc:1511] Processing CreateTablet for tablet b88ba89b2d1849848da78b9b3747e66e (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:55.108418 10278 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96acb559acc745428add2c682cf83cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.108418 10129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60fa8e2d405c485dbf19dff403d2328f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.112455 10335 tablet_bootstrap.cc:654] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.113868 10121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88ba89b2d1849848da78b9b3747e66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.114421 10126 tablet_service.cc:1511] Processing CreateTablet for tablet 96acb559acc745428add2c682cf83cfc (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:55.115785 10126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96acb559acc745428add2c682cf83cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.117084 10276 tablet_service.cc:1511] Processing CreateTablet for tablet aa60f11d5c674f3180f2616f97adf40f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:55.118855 10276 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa60f11d5c674f3180f2616f97adf40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.122514 10338 tablet_bootstrap.cc:654] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.126726 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea8a48dadff4881a7cdc0b27cf97db3" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:55.134634 10331 tablet_bootstrap.cc:654] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:55.140100 10179 leader_election.cc:343] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.142108 10203 tablet_service.cc:1511] Processing CreateTablet for tablet b88ba89b2d1849848da78b9b3747e66e (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:03:55.143093 10197 tablet_service.cc:1511] Processing CreateTablet for tablet 60fa8e2d405c485dbf19dff403d2328f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:03:55.143123 10200 tablet_service.cc:1511] Processing CreateTablet for tablet aa60f11d5c674f3180f2616f97adf40f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:55.144920 10047 tablet_service.cc:1511] Processing CreateTablet for tablet 60fa8e2d405c485dbf19dff403d2328f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
W20260709 13:03:55.145509 10030 leader_election.cc:343] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.146178 10047 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60fa8e2d405c485dbf19dff403d2328f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.146787 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea8a48dadff4881a7cdc0b27cf97db3" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:55.148001 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed4225af888b4e36ad1b03bdddc8f95f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:55.148063 10046 tablet_service.cc:1511] Processing CreateTablet for tablet 96acb559acc745428add2c682cf83cfc (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:03:55.143553 10203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b88ba89b2d1849848da78b9b3747e66e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.157004 10055 tablet_service.cc:1511] Processing CreateTablet for tablet aa60f11d5c674f3180f2616f97adf40f (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:03:55.148322 10049 tablet_service.cc:1511] Processing CreateTablet for tablet 4d87b05b458741adb45ae4a1753447d0 (DEFAULT_TABLE table=test-workload [id=d178cc47895a4ce68477df58e4429411]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
W20260709 13:03:55.157256 10030 leader_election.cc:343] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:55.158638 10179 leader_election.cc:343] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.160910 10179 leader_election.cc:304] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:55.161324 10200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa60f11d5c674f3180f2616f97adf40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.161825 10046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96acb559acc745428add2c682cf83cfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.169737 10356 raft_consensus.cc:2749] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:55.170794 10055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa60f11d5c674f3180f2616f97adf40f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.171459 10197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60fa8e2d405c485dbf19dff403d2328f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.172441 10030 leader_election.cc:304] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:55.175328 10359 raft_consensus.cc:2749] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:55.180876 10049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d87b05b458741adb45ae4a1753447d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:03:55.181864 10333 tablet_bootstrap.cc:492] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:55.182928 10350 raft_consensus.cc:493] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.185815 10350 raft_consensus.cc:515] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.188724 10338 tablet_bootstrap.cc:492] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
W20260709 13:03:55.191309 10178 consensus_peers.cc:597] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:55.189800 10338 ts_tablet_manager.cc:1403] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.116s	user 0.006s	sys 0.005s
I20260709 13:03:55.189903 10350 leader_election.cc:290] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:55.194018 10333 ts_tablet_manager.cc:1403] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.142s	user 0.016s	sys 0.004s
I20260709 13:03:55.205175 10335 tablet_bootstrap.cc:492] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:55.207784 10335 ts_tablet_manager.cc:1403] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.139s	user 0.008s	sys 0.006s
I20260709 13:03:55.211515 10331 tablet_bootstrap.cc:492] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:55.205591 10338 raft_consensus.cc:359] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.212219 10338 raft_consensus.cc:385] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.212383 10331 ts_tablet_manager.cc:1403] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.164s	user 0.012s	sys 0.009s
I20260709 13:03:55.212651 10338 raft_consensus.cc:740] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.214371 10338 consensus_queue.cc:260] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.230139 10350 raft_consensus.cc:493] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.230671 10350 raft_consensus.cc:515] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.232986 10350 leader_election.cc:290] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:55.234380 10333 raft_consensus.cc:359] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.211746 10335 raft_consensus.cc:359] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.245847 10335 raft_consensus.cc:385] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.246212 10335 raft_consensus.cc:740] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.248581 10335 consensus_queue.cc:260] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.252019 10350 raft_consensus.cc:493] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.238065 10333 raft_consensus.cc:385] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.253309 10350 raft_consensus.cc:515] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.262943 10338 ts_tablet_manager.cc:1434] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
I20260709 13:03:55.263140 10350 leader_election.cc:290] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:55.264225 10338 tablet_bootstrap.cc:492] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:55.269510 10359 raft_consensus.cc:493] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.270006 10335 ts_tablet_manager.cc:1434] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.002s
I20260709 13:03:55.270140 10359 raft_consensus.cc:515] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.271178 10335 tablet_bootstrap.cc:492] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:55.273607 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:55.274309 10214 raft_consensus.cc:2468] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:03:55.275907 10028 leader_election.cc:304] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:55.272435 10359 leader_election.cc:290] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:55.276405 10331 raft_consensus.cc:359] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.276800 10337 raft_consensus.cc:2804] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:55.277637 10331 raft_consensus.cc:385] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.277671 10337 raft_consensus.cc:493] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.279122 10331 raft_consensus.cc:740] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.279196 10337 raft_consensus.cc:3060] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.280509 10331 consensus_queue.cc:260] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.260058 10333 raft_consensus.cc:740] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.285534 10333 consensus_queue.cc:260] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.287853 10333 ts_tablet_manager.cc:1434] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.087s	user 0.007s	sys 0.000s
I20260709 13:03:55.288403 10331 ts_tablet_manager.cc:1434] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.075s	user 0.010s	sys 0.002s
I20260709 13:03:55.290690 10331 tablet_bootstrap.cc:492] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:55.293104 10356 consensus_queue.cc:1048] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:55.296329 10338 tablet_bootstrap.cc:654] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.297757 10331 tablet_bootstrap.cc:654] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.299736 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:55.300472 10288 raft_consensus.cc:2468] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:03:55.308069 10335 tablet_bootstrap.cc:654] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.300464 10337 raft_consensus.cc:515] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.323848 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:55.323761 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:55.324718 10288 raft_consensus.cc:3060] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.324932 10214 raft_consensus.cc:3060] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.325843 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ab3c2067a1463ca3f4319c644bd2f2" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:55.326558 10213 raft_consensus.cc:2468] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:03:55.327936 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:55.331396 10337 leader_election.cc:290] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:55.337646 10251 leader_election.cc:304] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:55.338565 10350 raft_consensus.cc:2804] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:55.338977 10350 raft_consensus.cc:493] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.339329 10350 raft_consensus.cc:3060] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.344918 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ab3c2067a1463ca3f4319c644bd2f2" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:55.345202 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "711d81634fdc47aba353f79bcbfede48" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:55.345801 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:55.344551 10214 raft_consensus.cc:2468] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:03:55.346710 10063 raft_consensus.cc:2393] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in current term 1: Already voted for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in this term.
W20260709 13:03:55.347209 10252 leader_election.cc:343] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.347195 10028 leader_election.cc:304] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
W20260709 13:03:55.348378 10252 leader_election.cc:343] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.349080 10359 raft_consensus.cc:2804] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:55.349390 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "711d81634fdc47aba353f79bcbfede48" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:55.350556 10252 leader_election.cc:304] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:03:55.336079 10333 tablet_bootstrap.cc:492] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:55.357231 10331 tablet_bootstrap.cc:492] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:55.358268 10331 ts_tablet_manager.cc:1403] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.000s
I20260709 13:03:55.359759 10338 tablet_bootstrap.cc:492] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:55.351326 10288 raft_consensus.cc:2468] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
W20260709 13:03:55.360339 10253 leader_election.cc:343] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.360854 10253 leader_election.cc:304] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:55.361764 10359 raft_consensus.cc:697] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:03:55.362149 10338 ts_tablet_manager.cc:1403] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.098s	user 0.007s	sys 0.015s
I20260709 13:03:55.363046 10359 consensus_queue.cc:237] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.365973 10333 tablet_bootstrap.cc:654] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.367125 10338 raft_consensus.cc:359] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.370607 10350 raft_consensus.cc:515] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.369894 10338 raft_consensus.cc:385] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.371001 10331 raft_consensus.cc:359] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.376315 10331 raft_consensus.cc:385] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.376911 10331 raft_consensus.cc:740] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.378206 10350 leader_election.cc:290] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:55.379977 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ab3c2067a1463ca3f4319c644bd2f2" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:55.375108 10338 raft_consensus.cc:740] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.379583 10350 raft_consensus.cc:2749] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:55.377812 10331 consensus_queue.cc:260] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.383849 10350 raft_consensus.cc:2749] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:55.384610 10252 leader_election.cc:343] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.383857 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2ab3c2067a1463ca3f4319c644bd2f2" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:55.385720 10214 raft_consensus.cc:3060] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.385921 10331 ts_tablet_manager.cc:1434] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:03:55.387177 10331 tablet_bootstrap.cc:492] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:55.386710  9962 catalog_manager.cc:5697] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:55.393867 10338 consensus_queue.cc:260] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.400012 10338 ts_tablet_manager.cc:1434] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:03:55.396284 10335 tablet_bootstrap.cc:492] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:55.402567 10335 ts_tablet_manager.cc:1403] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.132s	user 0.011s	sys 0.008s
I20260709 13:03:55.406929 10335 raft_consensus.cc:359] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.408602 10214 raft_consensus.cc:2468] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:03:55.410403 10331 tablet_bootstrap.cc:654] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.410435 10251 leader_election.cc:304] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:03:55.413897 10390 raft_consensus.cc:2804] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:55.414788 10390 raft_consensus.cc:697] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:03:55.415853 10333 tablet_bootstrap.cc:492] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:55.408015 10335 raft_consensus.cc:385] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.416436 10333 ts_tablet_manager.cc:1403] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.085s	user 0.020s	sys 0.002s
I20260709 13:03:55.418913 10335 raft_consensus.cc:740] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.419850 10335 consensus_queue.cc:260] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.422245 10335 ts_tablet_manager.cc:1434] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 13:03:55.423436 10335 tablet_bootstrap.cc:492] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:55.423005 10333 raft_consensus.cc:359] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.424183 10333 raft_consensus.cc:385] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.424599 10333 raft_consensus.cc:740] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.416034 10390 consensus_queue.cc:237] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.427824 10388 consensus_queue.cc:1048] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:55.429266 10333 consensus_queue.cc:260] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.412043 10338 tablet_bootstrap.cc:492] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
W20260709 13:03:55.432775 10028 consensus_peers.cc:597] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:55.450693 10359 raft_consensus.cc:493] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.454015 10359 raft_consensus.cc:515] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.457063 10338 tablet_bootstrap.cc:654] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.459241 10335 tablet_bootstrap.cc:654] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.459051 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7939918ac27c421f99a6747c1ea4a994" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:03:55.461441 10030 leader_election.cc:343] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.466382 10333 ts_tablet_manager.cc:1434] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.047s	user 0.021s	sys 0.009s
I20260709 13:03:55.470450 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7939918ac27c421f99a6747c1ea4a994" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:55.467625 10333 tablet_bootstrap.cc:492] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
W20260709 13:03:55.472385 10030 leader_election.cc:343] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.473562 10359 leader_election.cc:290] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:55.475059 10030 leader_election.cc:304] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:55.476666 10359 raft_consensus.cc:2749] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:55.481925 10333 tablet_bootstrap.cc:654] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.484062 10331 tablet_bootstrap.cc:492] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:55.485414 10331 ts_tablet_manager.cc:1403] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.098s	user 0.033s	sys 0.016s
I20260709 13:03:55.490828 10355 raft_consensus.cc:493] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.491583 10355 raft_consensus.cc:515] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.497967 10355 leader_election.cc:290] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:55.507112 10331 raft_consensus.cc:359] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.508219 10331 raft_consensus.cc:385] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.508595 10331 raft_consensus.cc:740] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.513774  9963 catalog_manager.cc:5697] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:55.513607 10331 consensus_queue.cc:260] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.525341 10333 tablet_bootstrap.cc:492] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:55.526115 10333 ts_tablet_manager.cc:1403] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.059s	user 0.017s	sys 0.019s
I20260709 13:03:55.532671 10338 tablet_bootstrap.cc:492] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:55.533247 10338 ts_tablet_manager.cc:1403] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.122s	user 0.013s	sys 0.001s
I20260709 13:03:55.534035 10331 ts_tablet_manager.cc:1434] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.002s
I20260709 13:03:55.535202 10333 raft_consensus.cc:359] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.535998 10333 raft_consensus.cc:385] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.536288 10333 raft_consensus.cc:740] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.536551 10338 raft_consensus.cc:359] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.537447 10333 consensus_queue.cc:260] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.538472 10355 raft_consensus.cc:493] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.539005 10355 raft_consensus.cc:515] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.540014 10333 ts_tablet_manager.cc:1434] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.002s
I20260709 13:03:55.540004 10390 raft_consensus.cc:493] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.540633 10390 raft_consensus.cc:515] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.543025 10390 leader_election.cc:290] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:55.541460 10355 leader_election.cc:290] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:55.547358 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed4225af888b4e36ad1b03bdddc8f95f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:55.547848 10333 tablet_bootstrap.cc:492] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
W20260709 13:03:55.549108 10105 leader_election.cc:343] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.549882 10331 tablet_bootstrap.cc:492] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:55.550904 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16541bea7ded48538adada244223f2c9" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:55.551695 10063 raft_consensus.cc:2468] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:03:55.553699 10252 leader_election.cc:304] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 
I20260709 13:03:55.554858 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16541bea7ded48538adada244223f2c9" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:55.555469 10390 raft_consensus.cc:2804] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:03:55.556559 10253 leader_election.cc:343] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.557898 10390 raft_consensus.cc:493] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.558442 10390 raft_consensus.cc:3060] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.537300 10338 raft_consensus.cc:385] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.554848 10333 tablet_bootstrap.cc:654] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.561856 10338 raft_consensus.cc:740] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.563963 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01f3be65cb0c478194f88fb404dd329f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:55.564736 10214 raft_consensus.cc:2468] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:03:55.566385 10103 leader_election.cc:304] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:55.567546 10338 consensus_queue.cc:260] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.568969 10335 tablet_bootstrap.cc:492] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:55.569564 10335 ts_tablet_manager.cc:1403] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.146s	user 0.010s	sys 0.006s
I20260709 13:03:55.570017 10338 ts_tablet_manager.cc:1434] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 13:03:55.570008 10390 raft_consensus.cc:515] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.572474 10331 tablet_bootstrap.cc:654] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.573253 10335 raft_consensus.cc:359] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.574101 10335 raft_consensus.cc:385] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.575042 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01f3be65cb0c478194f88fb404dd329f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:55.574679 10355 raft_consensus.cc:2804] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:55.575718 10355 raft_consensus.cc:493] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.575771 10065 raft_consensus.cc:2468] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:03:55.575197 10335 raft_consensus.cc:740] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.575109 10390 leader_election.cc:290] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:55.576992 10335 consensus_queue.cc:260] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.579100 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16541bea7ded48538adada244223f2c9" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:55.576211 10355 raft_consensus.cc:3060] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.574294 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed4225af888b4e36ad1b03bdddc8f95f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:55.582062 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16541bea7ded48538adada244223f2c9" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:55.582693 10064 raft_consensus.cc:3060] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:55.582715 10253 leader_election.cc:343] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.582077 10063 raft_consensus.cc:2468] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:03:55.587154 10104 leader_election.cc:304] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:55.587877 10355 raft_consensus.cc:515] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.590009 10338 tablet_bootstrap.cc:492] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
W20260709 13:03:55.592618 10011 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:55.598698 10355 leader_election.cc:290] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:55.599210 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01f3be65cb0c478194f88fb404dd329f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:55.599917 10214 raft_consensus.cc:3060] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.604243 10338 tablet_bootstrap.cc:654] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.599383 10355 raft_consensus.cc:2804] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:55.609715 10355 raft_consensus.cc:493] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.611397 10333 tablet_bootstrap.cc:492] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:55.611946 10333 ts_tablet_manager.cc:1403] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.071s	user 0.011s	sys 0.011s
I20260709 13:03:55.612695 10064 raft_consensus.cc:2468] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:03:55.615587 10333 raft_consensus.cc:359] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.616195 10214 raft_consensus.cc:2468] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:03:55.616698 10333 raft_consensus.cc:385] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.618144 10333 raft_consensus.cc:740] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.619231 10252 leader_election.cc:304] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:55.619185 10333 consensus_queue.cc:260] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.620769 10390 raft_consensus.cc:2804] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:55.620921 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01f3be65cb0c478194f88fb404dd329f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:55.621876 10064 raft_consensus.cc:3060] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.622411 10333 ts_tablet_manager.cc:1434] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:03:55.621292 10390 raft_consensus.cc:697] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:03:55.623701 10333 tablet_bootstrap.cc:492] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:55.624291 10390 consensus_queue.cc:237] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.636150 10331 tablet_bootstrap.cc:492] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:55.636686 10331 ts_tablet_manager.cc:1403] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.087s	user 0.029s	sys 0.027s
I20260709 13:03:55.639550 10331 raft_consensus.cc:359] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.640256 10331 raft_consensus.cc:385] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.640529 10331 raft_consensus.cc:740] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.640861 10397 raft_consensus.cc:493] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.641398 10397 raft_consensus.cc:515] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.618191 10355 raft_consensus.cc:3060] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.641518 10331 consensus_queue.cc:260] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.636719 10333 tablet_bootstrap.cc:654] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.643697 10397 leader_election.cc:290] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:55.644608 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5cf0e36da48d0946769d2bb27c981" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:55.645227 10064 raft_consensus.cc:2468] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:03:55.645382 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5cf0e36da48d0946769d2bb27c981" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:55.648694 10287 raft_consensus.cc:2468] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:55.640432 10103 leader_election.cc:304] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:55.647207 10331 ts_tablet_manager.cc:1434] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 13:03:55.647917 10335 ts_tablet_manager.cc:1434] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.007s
I20260709 13:03:55.649210 10388 raft_consensus.cc:493] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.645321 10140 raft_consensus.cc:2468] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:55.656558 10411 raft_consensus.cc:2804] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:55.650427 10331 tablet_bootstrap.cc:492] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:55.650966 10179 leader_election.cc:304] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:55.657187 10411 raft_consensus.cc:697] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:03:55.656832 10388 raft_consensus.cc:515] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.658594 10411 consensus_queue.cc:237] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.662563 10397 raft_consensus.cc:2804] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:55.663168 10397 raft_consensus.cc:493] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.663242 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:55.663800 10397 raft_consensus.cc:3060] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.668789 10335 tablet_bootstrap.cc:492] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:55.666985  9963 catalog_manager.cc:5697] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:55.671526 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:55.672017 10355 raft_consensus.cc:515] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.673489 10030 leader_election.cc:304] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:03:55.676244 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed4225af888b4e36ad1b03bdddc8f95f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:55.678643 10337 raft_consensus.cc:2749] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:55.679665 10105 leader_election.cc:343] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.679658 10397 raft_consensus.cc:515] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.680811 10388 leader_election.cc:290] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:55.681125 10335 tablet_bootstrap.cc:654] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.683641 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5cf0e36da48d0946769d2bb27c981" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:55.684309 10140 raft_consensus.cc:3060] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.684414 10355 leader_election.cc:290] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:55.684597 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed4225af888b4e36ad1b03bdddc8f95f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:55.685165 10064 raft_consensus.cc:3060] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.692734 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5cf0e36da48d0946769d2bb27c981" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:55.690569 10397 leader_election.cc:290] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:55.693465 10287 raft_consensus.cc:3060] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.693858 10333 tablet_bootstrap.cc:492] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:55.696496 10333 ts_tablet_manager.cc:1403] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.073s	user 0.012s	sys 0.016s
I20260709 13:03:55.700470 10333 raft_consensus.cc:359] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.701392 10333 raft_consensus.cc:385] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.701730 10333 raft_consensus.cc:740] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.702497 10333 consensus_queue.cc:260] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.696360 10064 raft_consensus.cc:2468] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:03:55.705372 10338 tablet_bootstrap.cc:492] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:55.706799 10338 ts_tablet_manager.cc:1403] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.117s	user 0.021s	sys 0.010s
I20260709 13:03:55.706959 10104 leader_election.cc:304] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:55.707345 10140 raft_consensus.cc:2468] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:55.708245 10355 raft_consensus.cc:2804] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:55.708806 10355 raft_consensus.cc:697] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:03:55.709117 10179 leader_election.cc:304] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:55.710392 10355 consensus_queue.cc:237] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.712980 10397 raft_consensus.cc:2804] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:55.713820 10397 raft_consensus.cc:697] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:03:55.714742 10397 consensus_queue.cc:237] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.718834 10338 raft_consensus.cc:359] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.719857 10338 raft_consensus.cc:385] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.721098 10338 raft_consensus.cc:740] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.718678  9962 catalog_manager.cc:5697] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:55.722098 10338 consensus_queue.cc:260] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.724130 10333 ts_tablet_manager.cc:1434] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.003s
I20260709 13:03:55.724663 10338 ts_tablet_manager.cc:1434] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.003s
I20260709 13:03:55.725229 10333 tablet_bootstrap.cc:492] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:55.720628 10287 raft_consensus.cc:2468] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:55.726233 10338 tablet_bootstrap.cc:492] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:55.732962 10333 tablet_bootstrap.cc:654] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.734606 10331 tablet_bootstrap.cc:654] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.738930 10335 tablet_bootstrap.cc:492] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:55.739789 10335 ts_tablet_manager.cc:1403] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.071s	user 0.014s	sys 0.010s
I20260709 13:03:55.734572 10338 tablet_bootstrap.cc:654] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.744783 10335 raft_consensus.cc:359] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.750574 10335 raft_consensus.cc:385] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.751009 10335 raft_consensus.cc:740] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.752043 10335 consensus_queue.cc:260] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.752723  9960 catalog_manager.cc:5697] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:55.755772 10335 ts_tablet_manager.cc:1434] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:03:55.757300 10335 tablet_bootstrap.cc:492] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:55.760136  9962 catalog_manager.cc:5697] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:55.772365 10335 tablet_bootstrap.cc:654] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.775754 10331 tablet_bootstrap.cc:492] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:55.776364 10331 ts_tablet_manager.cc:1403] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.126s	user 0.014s	sys 0.002s
I20260709 13:03:55.779253 10331 raft_consensus.cc:359] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.781185 10331 raft_consensus.cc:385] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.781574 10331 raft_consensus.cc:740] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.782258 10331 consensus_queue.cc:260] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.783245 10397 raft_consensus.cc:493] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.783998 10331 ts_tablet_manager.cc:1434] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:03:55.783928 10397 raft_consensus.cc:515] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.784876 10331 tablet_bootstrap.cc:492] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:55.787082 10333 tablet_bootstrap.cc:492] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:55.787588 10333 ts_tablet_manager.cc:1403] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.039s
I20260709 13:03:55.787640 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065434c541d4f10a459f9003a463d86" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:55.788391 10064 raft_consensus.cc:2468] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:55.789625 10338 tablet_bootstrap.cc:492] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:55.789812 10178 leader_election.cc:304] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:55.790123 10338 ts_tablet_manager.cc:1403] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.064s	user 0.007s	sys 0.005s
I20260709 13:03:55.790546 10333 raft_consensus.cc:359] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.790872 10392 raft_consensus.cc:2804] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:55.791258 10333 raft_consensus.cc:385] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.791357 10392 raft_consensus.cc:493] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:55.791651 10333 raft_consensus.cc:740] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.791844 10392 raft_consensus.cc:3060] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.792212 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065434c541d4f10a459f9003a463d86" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:55.792428 10333 consensus_queue.cc:260] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.792882 10140 raft_consensus.cc:2468] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:55.797931 10397 leader_election.cc:290] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:55.798467 10331 tablet_bootstrap.cc:654] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.797116 10338 raft_consensus.cc:359] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.799374 10388 consensus_queue.cc:1048] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:55.799839 10338 raft_consensus.cc:385] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.800414 10338 raft_consensus.cc:740] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.801595 10338 consensus_queue.cc:260] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.805274 10338 ts_tablet_manager.cc:1434] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 13:03:55.806324 10338 tablet_bootstrap.cc:492] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:55.806450 10333 ts_tablet_manager.cc:1434] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 13:03:55.807518 10333 tablet_bootstrap.cc:492] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:55.812268 10335 tablet_bootstrap.cc:492] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:55.812831 10335 ts_tablet_manager.cc:1403] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.056s	user 0.022s	sys 0.000s
I20260709 13:03:55.813591 10338 tablet_bootstrap.cc:654] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.813776 10333 tablet_bootstrap.cc:654] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.809095 10392 raft_consensus.cc:515] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.817317 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065434c541d4f10a459f9003a463d86" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:55.817952 10064 raft_consensus.cc:3060] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.822073 10331 tablet_bootstrap.cc:492] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:55.822636 10331 ts_tablet_manager.cc:1403] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.000s
I20260709 13:03:55.825623 10064 raft_consensus.cc:2468] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:55.825778 10331 raft_consensus.cc:359] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.826850 10331 raft_consensus.cc:385] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.827159 10331 raft_consensus.cc:740] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.827906 10331 consensus_queue.cc:260] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.828138 10178 leader_election.cc:304] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:55.828951 10397 raft_consensus.cc:2804] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:55.829408 10397 raft_consensus.cc:697] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:03:55.830025 10331 ts_tablet_manager.cc:1434] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:03:55.830215 10397 consensus_queue.cc:237] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.833668 10331 tablet_bootstrap.cc:492] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:55.854530 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065434c541d4f10a459f9003a463d86" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:55.854032  9962 catalog_manager.cc:5697] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:55.855191 10140 raft_consensus.cc:3060] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.857808 10392 leader_election.cc:290] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:55.865751 10335 raft_consensus.cc:359] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.866636 10335 raft_consensus.cc:385] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.866991 10335 raft_consensus.cc:740] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.867810 10335 consensus_queue.cc:260] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
W20260709 13:03:55.868585 10252 consensus_peers.cc:597] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:55.871640 10335 ts_tablet_manager.cc:1434] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.058s	user 0.010s	sys 0.000s
I20260709 13:03:55.874444 10331 tablet_bootstrap.cc:654] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.878485 10388 consensus_queue.cc:1048] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:55.880028 10337 raft_consensus.cc:493] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.880579 10337 raft_consensus.cc:515] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:55.882819 10337 leader_election.cc:290] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:55.884222 10333 tablet_bootstrap.cc:492] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:55.885933 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84c07107a48406d9fd585e9982d3069" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:55.887526 10335 tablet_bootstrap.cc:492] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:55.890003 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84c07107a48406d9fd585e9982d3069" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:55.890551 10338 tablet_bootstrap.cc:492] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:55.891093 10338 ts_tablet_manager.cc:1403] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.085s	user 0.014s	sys 0.031s
W20260709 13:03:55.887805 10028 leader_election.cc:343] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.885489 10333 ts_tablet_manager.cc:1403] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.078s	user 0.018s	sys 0.038s
I20260709 13:03:55.879761 10140 raft_consensus.cc:2468] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:55.899574 10389 raft_consensus.cc:493] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.900137 10389 raft_consensus.cc:515] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.896694 10333 raft_consensus.cc:359] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.901849 10333 raft_consensus.cc:385] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.902200 10333 raft_consensus.cc:740] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.902963 10333 consensus_queue.cc:260] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.903775 10389 leader_election.cc:290] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:55.905470 10333 ts_tablet_manager.cc:1434] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 13:03:55.910435 10333 tablet_bootstrap.cc:492] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:55.915026 10331 tablet_bootstrap.cc:492] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:55.916559 10333 tablet_bootstrap.cc:654] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.916682 10331 ts_tablet_manager.cc:1403] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.083s	user 0.010s	sys 0.005s
W20260709 13:03:55.906926 10030 leader_election.cc:343] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:55.907251 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed4225af888b4e36ad1b03bdddc8f95f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:55.918721 10030 leader_election.cc:304] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:03:55.920120 10388 raft_consensus.cc:2749] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:55.907423 10335 tablet_bootstrap.cc:654] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.906814 10338 raft_consensus.cc:359] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.921615 10338 raft_consensus.cc:385] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.921898 10338 raft_consensus.cc:740] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.922739 10338 consensus_queue.cc:260] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.924351 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed4225af888b4e36ad1b03bdddc8f95f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:55.927003 10064 raft_consensus.cc:2393] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in current term 1: Already voted for candidate bfa051e18d0549eabe79dfcc7610d99b in this term.
I20260709 13:03:55.923204 10214 raft_consensus.cc:3060] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.928378 10338 ts_tablet_manager.cc:1434] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:03:55.928974 10333 tablet_bootstrap.cc:492] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:55.929144 10252 leader_election.cc:304] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:55.929466 10338 tablet_bootstrap.cc:492] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:55.929524 10333 ts_tablet_manager.cc:1403] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.019s	user 0.017s	sys 0.002s
I20260709 13:03:55.930522 10389 raft_consensus.cc:3060] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:55.932514 10333 raft_consensus.cc:359] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.933228 10333 raft_consensus.cc:385] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.933522 10333 raft_consensus.cc:740] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.934201 10333 consensus_queue.cc:260] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.935985 10338 tablet_bootstrap.cc:654] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.938607 10389 raft_consensus.cc:2749] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:55.939940 10333 ts_tablet_manager.cc:1434] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:03:55.940979 10333 tablet_bootstrap.cc:492] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:55.945724 10335 tablet_bootstrap.cc:492] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:55.946322 10335 ts_tablet_manager.cc:1403] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.059s	user 0.016s	sys 0.000s
I20260709 13:03:55.949249 10335 raft_consensus.cc:359] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.950311 10335 raft_consensus.cc:385] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.950628 10335 raft_consensus.cc:740] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.950083 10331 raft_consensus.cc:359] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.951471 10335 consensus_queue.cc:260] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.956915 10331 raft_consensus.cc:385] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.957481 10331 raft_consensus.cc:740] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.959048 10335 ts_tablet_manager.cc:1434] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:03:55.958555 10331 consensus_queue.cc:260] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:55.960794 10335 tablet_bootstrap.cc:492] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:55.961596 10333 tablet_bootstrap.cc:654] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.962700 10331 ts_tablet_manager.cc:1434] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.017s	user 0.001s	sys 0.006s
I20260709 13:03:55.966909 10331 tablet_bootstrap.cc:492] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:55.975571 10331 tablet_bootstrap.cc:654] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.985724 10335 tablet_bootstrap.cc:654] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:55.990998 10338 tablet_bootstrap.cc:492] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:55.991670 10338 ts_tablet_manager.cc:1403] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.062s	user 0.024s	sys 0.007s
I20260709 13:03:55.986564 10333 tablet_bootstrap.cc:492] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:55.994817 10333 ts_tablet_manager.cc:1403] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.054s	user 0.017s	sys 0.001s
I20260709 13:03:55.995040 10338 raft_consensus.cc:359] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.995746 10338 raft_consensus.cc:385] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.996054 10389 raft_consensus.cc:493] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.996029 10338 raft_consensus.cc:740] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.996593 10389 raft_consensus.cc:515] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.997562 10338 consensus_queue.cc:260] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:55.998558 10333 raft_consensus.cc:359] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:55.998735 10389 leader_election.cc:290] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:55.998809 10390 raft_consensus.cc:493] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:55.999333 10333 raft_consensus.cc:385] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:55.999815 10333 raft_consensus.cc:740] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:55.999791 10390 raft_consensus.cc:515] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.000741 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea8a48dadff4881a7cdc0b27cf97db3" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:56.001389 10212 raft_consensus.cc:2468] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:03:56.001080 10333 consensus_queue.cc:260] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.004573 10333 ts_tablet_manager.cc:1434] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:03:56.004612 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b9c093052a4414be72162a905b9a1d" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:55.999926 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea8a48dadff4881a7cdc0b27cf97db3" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:56.006443 10140 raft_consensus.cc:2468] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:03:56.004918 10251 leader_election.cc:304] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:56.009264 10333 tablet_bootstrap.cc:492] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:56.002172 10390 leader_election.cc:290] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:56.010007 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b9c093052a4414be72162a905b9a1d" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:56.013019 10214 raft_consensus.cc:2468] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
W20260709 13:03:56.006870 10253 leader_election.cc:343] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.016036 10389 raft_consensus.cc:2804] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:56.016784 10389 raft_consensus.cc:493] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.017336 10389 raft_consensus.cc:3060] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.016372 10331 tablet_bootstrap.cc:492] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:56.020895 10331 ts_tablet_manager.cc:1403] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.054s	user 0.016s	sys 0.002s
I20260709 13:03:56.031740 10251 leader_election.cc:304] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:56.032471 10389 raft_consensus.cc:515] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.033003 10438 raft_consensus.cc:2804] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:56.026050 10331 raft_consensus.cc:359] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.033993 10331 raft_consensus.cc:385] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.033738 10390 consensus_queue.cc:1048] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:56.034422 10333 tablet_bootstrap.cc:654] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.033526 10438 raft_consensus.cc:493] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.035729 10331 raft_consensus.cc:740] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.036077 10438 raft_consensus.cc:3060] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.036725 10331 consensus_queue.cc:260] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.035379 10338 ts_tablet_manager.cc:1434] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.043s	user 0.011s	sys 0.000s
I20260709 13:03:56.035421 10335 tablet_bootstrap.cc:492] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:56.038231 10335 ts_tablet_manager.cc:1403] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.078s	user 0.024s	sys 0.014s
I20260709 13:03:56.041543 10335 raft_consensus.cc:359] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.042344 10335 raft_consensus.cc:385] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.042666 10335 raft_consensus.cc:740] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.045778 10438 raft_consensus.cc:515] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.047624 10338 tablet_bootstrap.cc:492] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:56.048207 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea8a48dadff4881a7cdc0b27cf97db3" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:56.048902 10140 raft_consensus.cc:3060] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.050014 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b9c093052a4414be72162a905b9a1d" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:56.050730 10212 raft_consensus.cc:3060] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.053570 10389 leader_election.cc:290] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:56.043454 10335 consensus_queue.cc:260] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.054103 10392 raft_consensus.cc:493] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.054497 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b9c093052a4414be72162a905b9a1d" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:56.054631 10392 raft_consensus.cc:515] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.055452 10139 raft_consensus.cc:3060] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.055788 10335 ts_tablet_manager.cc:1434] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:03:56.056999 10335 tablet_bootstrap.cc:492] T 16541bea7ded48538adada244223f2c9 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:56.059095 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0007e852f46b2b3ce899d1101e17c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:56.060020 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0007e852f46b2b3ce899d1101e17c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:03:56.061458 10179 leader_election.cc:343] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.062189 10392 leader_election.cc:290] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
W20260709 13:03:56.062356 10179 leader_election.cc:343] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.063227 10179 leader_election.cc:304] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:56.065719 10392 raft_consensus.cc:2749] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:56.066699 10331 ts_tablet_manager.cc:1434] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 13:03:56.067836 10331 tablet_bootstrap.cc:492] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:56.071465 10355 consensus_queue.cc:1048] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:56.071723 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea8a48dadff4881a7cdc0b27cf97db3" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:56.071869 10411 raft_consensus.cc:493] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.072479 10213 raft_consensus.cc:3060] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.072505 10411 raft_consensus.cc:515] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.075132 10338 tablet_bootstrap.cc:654] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.067927 10438 leader_election.cc:290] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:56.083108 10331 tablet_bootstrap.cc:654] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.086501 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "711d81634fdc47aba353f79bcbfede48" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:56.091827 10335 tablet_bootstrap.cc:654] T 16541bea7ded48538adada244223f2c9 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.093667 10333 tablet_bootstrap.cc:492] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
W20260709 13:03:56.093603 10104 leader_election.cc:343] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.094515 10333 ts_tablet_manager.cc:1403] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.086s	user 0.017s	sys 0.020s
I20260709 13:03:56.095373 10212 raft_consensus.cc:2468] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:03:56.096632 10251 leader_election.cc:304] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:56.088505 10411 leader_election.cc:290] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:56.097394 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "711d81634fdc47aba353f79bcbfede48" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:56.105870 10287 raft_consensus.cc:2468] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:03:56.107285 10105 leader_election.cc:304] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:03:56.108176 10355 consensus_queue.cc:1048] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:56.109841 10355 raft_consensus.cc:2804] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:56.110330 10355 raft_consensus.cc:493] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.110774 10355 raft_consensus.cc:3060] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.117887 10392 raft_consensus.cc:493] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.118669 10392 raft_consensus.cc:515] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.122956 10392 leader_election.cc:290] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:56.123709 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d07303bc2640fe9d6ea20f824ce00e" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:56.124514 10213 raft_consensus.cc:2468] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
W20260709 13:03:56.125464 10179 leader_election.cc:343] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.098276 10140 raft_consensus.cc:2468] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:03:56.127903 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d07303bc2640fe9d6ea20f824ce00e" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:56.100755 10397 consensus_queue.cc:1048] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 13:03:56.099740 10139 raft_consensus.cc:2468] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
W20260709 13:03:56.130188 10179 leader_election.cc:343] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.130838 10179 leader_election.cc:304] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:56.131168 10253 leader_election.cc:304] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:56.132011 10389 raft_consensus.cc:2804] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:56.132488 10389 raft_consensus.cc:697] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:03:56.132824 10397 raft_consensus.cc:2749] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:56.133327 10389 consensus_queue.cc:237] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.098446 10438 raft_consensus.cc:2804] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:56.136057 10438 raft_consensus.cc:697] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:03:56.136940 10438 consensus_queue.cc:237] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.139788 10411 consensus_queue.cc:1048] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:56.097525 10333 raft_consensus.cc:359] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.145649 10333 raft_consensus.cc:385] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.145926 10333 raft_consensus.cc:740] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.174731 10338 tablet_bootstrap.cc:492] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:56.175391 10338 ts_tablet_manager.cc:1403] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.128s	user 0.013s	sys 0.013s
I20260709 13:03:56.180421 10338 raft_consensus.cc:359] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.181260 10338 raft_consensus.cc:385] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.181813 10338 raft_consensus.cc:740] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.167408 10355 raft_consensus.cc:515] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.186180 10392 raft_consensus.cc:493] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.186808 10392 raft_consensus.cc:515] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.187614 10355 leader_election.cc:290] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:56.188953 10392 leader_election.cc:290] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:56.190343 10331 tablet_bootstrap.cc:492] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:56.191020 10331 ts_tablet_manager.cc:1403] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.123s	user 0.018s	sys 0.005s
I20260709 13:03:56.191732 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "711d81634fdc47aba353f79bcbfede48" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:56.190933 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0709ff33d24996ae638187a8e7c57f" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
W20260709 13:03:56.194245 10104 leader_election.cc:343] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.199692 10338 consensus_queue.cc:260] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.198846 10331 raft_consensus.cc:359] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.201324 10331 raft_consensus.cc:385] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.201808 10331 raft_consensus.cc:740] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.200320 10064 raft_consensus.cc:3060] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.203038 10331 consensus_queue.cc:260] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.204551 10333 consensus_queue.cc:260] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.209193 10335 tablet_bootstrap.cc:492] T 16541bea7ded48538adada244223f2c9 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:56.209733 10335 ts_tablet_manager.cc:1403] T 16541bea7ded48538adada244223f2c9 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.153s	user 0.022s	sys 0.047s
I20260709 13:03:56.211009 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "711d81634fdc47aba353f79bcbfede48" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:56.211930 10288 raft_consensus.cc:3060] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.213815 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0709ff33d24996ae638187a8e7c57f" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:56.215476 10430 raft_consensus.cc:493] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.216106 10430 raft_consensus.cc:515] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.218325 10430 leader_election.cc:290] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:56.201624 10063 raft_consensus.cc:2468] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:56.220605 10178 leader_election.cc:304] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:56.222723 10397 raft_consensus.cc:2804] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:56.223110 10397 raft_consensus.cc:493] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.223456 10397 raft_consensus.cc:3060] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:56.224470 10179 leader_election.cc:343] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.227556 10335 raft_consensus.cc:359] T 16541bea7ded48538adada244223f2c9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.228303 10335 raft_consensus.cc:385] T 16541bea7ded48538adada244223f2c9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.228551 10335 raft_consensus.cc:740] T 16541bea7ded48538adada244223f2c9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.229204 10335 consensus_queue.cc:260] T 16541bea7ded48538adada244223f2c9 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.230837 10331 ts_tablet_manager.cc:1434] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.000s
I20260709 13:03:56.231529 10335 ts_tablet_manager.cc:1434] T 16541bea7ded48538adada244223f2c9 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:03:56.231866 10331 tablet_bootstrap.cc:492] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:56.232535 10335 tablet_bootstrap.cc:492] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:56.227629 10338 ts_tablet_manager.cc:1434] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.052s	user 0.004s	sys 0.004s
I20260709 13:03:56.234673 10338 tablet_bootstrap.cc:492] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:56.239610 10335 tablet_bootstrap.cc:654] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.239789 10439 consensus_queue.cc:1048] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:56.240351 10338 tablet_bootstrap.cc:654] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.243420 10411 consensus_queue.cc:1048] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:56.243832 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c29bf223e8224f72926bd39f6dcb3f08" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:56.247267 10333 ts_tablet_manager.cc:1434] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.152s	user 0.004s	sys 0.008s
I20260709 13:03:56.249158 10288 raft_consensus.cc:2468] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:03:56.250896 10333 tablet_bootstrap.cc:492] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:56.247670  9962 catalog_manager.cc:5697] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:56.262430 10331 tablet_bootstrap.cc:654] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.246308 10213 raft_consensus.cc:2468] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:03:56.270841 10028 leader_election.cc:304] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:56.271730 10430 raft_consensus.cc:2804] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:56.272140 10430 raft_consensus.cc:493] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.272501 10430 raft_consensus.cc:3060] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.252763 10392 raft_consensus.cc:493] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.272554 10105 leader_election.cc:304] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:03:56.278754 10356 consensus_queue.cc:1048] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:56.278496 10389 raft_consensus.cc:493] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.293526 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c29bf223e8224f72926bd39f6dcb3f08" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
W20260709 13:03:56.299597 10030 leader_election.cc:343] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.280269 10392 raft_consensus.cc:515] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.300908 10139 raft_consensus.cc:3060] T 16541bea7ded48538adada244223f2c9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.302958 10392 leader_election.cc:290] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:56.285745 10411 raft_consensus.cc:2804] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:56.313700 10411 raft_consensus.cc:697] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:03:56.313457 10430 raft_consensus.cc:515] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.314914 10411 consensus_queue.cc:237] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.289575 10389 raft_consensus.cc:515] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.317298 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c29bf223e8224f72926bd39f6dcb3f08" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
W20260709 13:03:56.319445 10030 leader_election.cc:343] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.319818 10389 leader_election.cc:290] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:56.305733 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0b03477ff4d1ab861dc093edbe68a" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:56.321064 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7939918ac27c421f99a6747c1ea4a994" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:56.322010 10138 raft_consensus.cc:2468] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:03:56.323812 10253 leader_election.cc:304] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
W20260709 13:03:56.325281 10178 leader_election.cc:343] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.327009  9962 catalog_manager.cc:5697] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:56.334244 10331 tablet_bootstrap.cc:492] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:56.335322 10331 ts_tablet_manager.cc:1403] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.104s	user 0.013s	sys 0.001s
I20260709 13:03:56.330562 10389 raft_consensus.cc:2804] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:03:56.343664 10179 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:33711, user_credentials={real_user=slave}} blocked reactor thread for 74423.8us
I20260709 13:03:56.345007 10397 raft_consensus.cc:515] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.347892 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7939918ac27c421f99a6747c1ea4a994" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:56.348601 10062 raft_consensus.cc:2468] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:03:56.353055 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0b03477ff4d1ab861dc093edbe68a" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:56.353610 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c29bf223e8224f72926bd39f6dcb3f08" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:56.354223 10213 raft_consensus.cc:3060] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:56.354627 10179 leader_election.cc:343] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.355288 10179 leader_election.cc:304] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:56.359412 10453 raft_consensus.cc:2749] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:56.365677 10439 consensus_queue.cc:1048] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:56.370067 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0709ff33d24996ae638187a8e7c57f" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:56.370746 10064 raft_consensus.cc:3060] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.370636 10062 raft_consensus.cc:3060] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.342003 10389 raft_consensus.cc:493] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.375566 10389 raft_consensus.cc:3060] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.379441 10333 tablet_bootstrap.cc:654] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.379673 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0709ff33d24996ae638187a8e7c57f" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:56.384158 10338 tablet_bootstrap.cc:492] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:56.384929 10338 ts_tablet_manager.cc:1403] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.150s	user 0.007s	sys 0.031s
I20260709 13:03:56.389533 10338 raft_consensus.cc:359] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.390494 10338 raft_consensus.cc:385] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.390846 10338 raft_consensus.cc:740] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.354691 10397 leader_election.cc:290] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:56.398934 10338 consensus_queue.cc:260] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.392238 10430 leader_election.cc:290] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
W20260709 13:03:56.397372 10179 leader_election.cc:343] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.401798 10064 raft_consensus.cc:2468] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:56.407022 10331 raft_consensus.cc:359] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.408367 10331 raft_consensus.cc:385] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.408718 10331 raft_consensus.cc:740] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.409984 10331 consensus_queue.cc:260] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.411255 10178 leader_election.cc:304] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:56.412027 10338 ts_tablet_manager.cc:1434] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.005s
I20260709 13:03:56.414124 10459 raft_consensus.cc:2804] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:56.414690 10335 tablet_bootstrap.cc:492] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:56.415203 10335 ts_tablet_manager.cc:1403] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.183s	user 0.026s	sys 0.034s
I20260709 13:03:56.415560 10459 raft_consensus.cc:697] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:03:56.416479 10331 ts_tablet_manager.cc:1434] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.081s	user 0.007s	sys 0.000s
I20260709 13:03:56.417107 10338 tablet_bootstrap.cc:492] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:56.421315 10331 tablet_bootstrap.cc:492] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:56.422855 10335 raft_consensus.cc:359] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.423659 10335 raft_consensus.cc:385] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.423465  9962 catalog_manager.cc:5697] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:56.424075 10335 raft_consensus.cc:740] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.425683 10335 consensus_queue.cc:260] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.429759 10331 tablet_bootstrap.cc:654] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.429817 10335 ts_tablet_manager.cc:1434] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:03:56.431118 10335 tablet_bootstrap.cc:492] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:56.440107 10335 tablet_bootstrap.cc:654] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.417105 10459 consensus_queue.cc:237] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.454437 10331 tablet_bootstrap.cc:492] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:56.454977 10331 ts_tablet_manager.cc:1403] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.034s	user 0.008s	sys 0.012s
I20260709 13:03:56.457878 10213 raft_consensus.cc:2468] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:03:56.462461 10028 leader_election.cc:304] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:56.462932 10333 tablet_bootstrap.cc:492] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:56.463431 10333 ts_tablet_manager.cc:1403] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.213s	user 0.010s	sys 0.007s
I20260709 13:03:56.464124 10389 raft_consensus.cc:515] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.466809 10430 raft_consensus.cc:2804] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:56.467880 10430 raft_consensus.cc:697] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:03:56.468670 10389 leader_election.cc:290] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:56.469820 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7939918ac27c421f99a6747c1ea4a994" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:56.469383 10430 consensus_queue.cc:237] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.470425 10139 raft_consensus.cc:3060] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.470798 10397 consensus_queue.cc:1048] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:56.477739 10397 consensus_queue.cc:1048] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:56.479167 10337 raft_consensus.cc:493] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.479713 10337 raft_consensus.cc:515] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.477984 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7939918ac27c421f99a6747c1ea4a994" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:56.481735 10065 raft_consensus.cc:3060] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.481815 10337 leader_election.cc:290] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:56.482299 10331 raft_consensus.cc:359] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.483016 10331 raft_consensus.cc:385] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.483291 10331 raft_consensus.cc:740] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.483974 10331 consensus_queue.cc:260] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.486999 10335 tablet_bootstrap.cc:492] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:56.487576 10335 ts_tablet_manager.cc:1403] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.003s
I20260709 13:03:56.487767 10333 raft_consensus.cc:359] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.489295 10333 raft_consensus.cc:385] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.489678 10333 raft_consensus.cc:740] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.490483 10333 consensus_queue.cc:260] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.493204 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8d5b10e4fe4ce994c81bf45aa93591" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:56.494058 10139 raft_consensus.cc:2468] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:03:56.494410 10331 ts_tablet_manager.cc:1434] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.001s
I20260709 13:03:56.495217 10253 leader_election.cc:304] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:56.496174 10389 raft_consensus.cc:2804] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:56.496701 10389 raft_consensus.cc:697] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:03:56.497346 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8d5b10e4fe4ce994c81bf45aa93591" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:56.497545 10389 consensus_queue.cc:237] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.502421 10337 raft_consensus.cc:493] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.503051 10337 raft_consensus.cc:515] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.505336 10337 leader_election.cc:290] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:56.507664 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf60e84958749f9826c9e8de7d266d9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
W20260709 13:03:56.509584 10028 leader_election.cc:343] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.515637 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf60e84958749f9826c9e8de7d266d9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:56.516000 10335 raft_consensus.cc:359] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.517057 10335 raft_consensus.cc:385] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.517469 10335 raft_consensus.cc:740] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
W20260709 13:03:56.518558 10030 leader_election.cc:343] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:56.519337 10030 leader_election.cc:343] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.519809 10030 leader_election.cc:304] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:56.521944 10333 ts_tablet_manager.cc:1434] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.000s
I20260709 13:03:56.523154 10333 tablet_bootstrap.cc:492] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:56.541836 10338 tablet_bootstrap.cc:654] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.545766 10335 consensus_queue.cc:260] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.547092 10411 raft_consensus.cc:493] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.547744 10411 raft_consensus.cc:515] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.550028 10337 raft_consensus.cc:2749] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:03:56.550551 10030 leader_election.cc:343] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.551128 10030 leader_election.cc:304] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:56.554984 10337 raft_consensus.cc:2749] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:56.561303 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8d5b10e4fe4ce994c81bf45aa93591" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:56.562258 10439 consensus_queue.cc:1048] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:56.562794 10411 leader_election.cc:290] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
W20260709 13:03:56.562919 10105 leader_election.cc:343] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.563031 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8d5b10e4fe4ce994c81bf45aa93591" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:56.563858 10062 raft_consensus.cc:2468] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:03:56.495447 10331 tablet_bootstrap.cc:492] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:56.565245 10104 leader_election.cc:304] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:56.566205 10411 raft_consensus.cc:2804] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:56.566716 10333 tablet_bootstrap.cc:654] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.566583 10411 raft_consensus.cc:493] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.567176 10411 raft_consensus.cc:3060] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.568847 10355 raft_consensus.cc:493] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.569725 10355 raft_consensus.cc:515] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.571923 10065 raft_consensus.cc:2468] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:03:56.572408 10355 leader_election.cc:290] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:56.573242 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d07303bc2640fe9d6ea20f824ce00e" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:56.573964 10213 raft_consensus.cc:2468] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:03:56.574270 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d07303bc2640fe9d6ea20f824ce00e" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:56.575160 10103 leader_election.cc:304] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
W20260709 13:03:56.575821 10105 leader_election.cc:343] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.576689 10355 raft_consensus.cc:2804] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:56.577098 10355 raft_consensus.cc:493] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.577414 10355 raft_consensus.cc:3060] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.585527 10355 raft_consensus.cc:515] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.589488 10335 ts_tablet_manager.cc:1434] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.102s	user 0.012s	sys 0.000s
I20260709 13:03:56.589330 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d07303bc2640fe9d6ea20f824ce00e" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:56.590086 10213 raft_consensus.cc:3060] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.595292 10335 tablet_bootstrap.cc:492] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:56.597337 10337 raft_consensus.cc:493] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.588979  9962 catalog_manager.cc:5697] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:56.597904 10337 raft_consensus.cc:515] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.599875 10337 leader_election.cc:290] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:56.609244 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d07303bc2640fe9d6ea20f824ce00e" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
W20260709 13:03:56.611413 10105 leader_election.cc:343] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.613185 10355 leader_election.cc:290] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:56.613377 10338 tablet_bootstrap.cc:492] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:56.614154 10338 ts_tablet_manager.cc:1403] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.201s	user 0.008s	sys 0.006s
I20260709 13:03:56.615190 10438 consensus_queue.cc:1048] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:56.617233 10338 raft_consensus.cc:359] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.620666 10338 raft_consensus.cc:385] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.617952 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "711d81634fdc47aba353f79bcbfede48" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:56.621516 10287 raft_consensus.cc:2393] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in current term 1: Already voted for candidate bfa051e18d0549eabe79dfcc7610d99b in this term.
I20260709 13:03:56.620318 10335 tablet_bootstrap.cc:654] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.627411  9960 catalog_manager.cc:5697] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:56.618145 10331 tablet_bootstrap.cc:654] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.628443 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "711d81634fdc47aba353f79bcbfede48" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:56.627462 10338 raft_consensus.cc:740] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.631685 10213 raft_consensus.cc:2468] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:03:56.631975 10338 consensus_queue.cc:260] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.637104 10338 ts_tablet_manager.cc:1434] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
W20260709 13:03:56.634238 10011 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:56.640820 10030 leader_election.cc:304] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:56.633253 10103 leader_election.cc:304] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:56.637218 10389 consensus_queue.cc:1048] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:56.642729 10338 tablet_bootstrap.cc:492] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:56.643419 10337 raft_consensus.cc:3060] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.647063 10335 tablet_bootstrap.cc:492] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:56.647593 10335 ts_tablet_manager.cc:1403] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.001s
I20260709 13:03:56.647810 10410 raft_consensus.cc:2804] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:56.652190 10337 raft_consensus.cc:2749] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:56.657248 10331 tablet_bootstrap.cc:492] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:56.658039 10331 ts_tablet_manager.cc:1403] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.163s	user 0.008s	sys 0.005s
I20260709 13:03:56.661343 10338 tablet_bootstrap.cc:654] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.641532 10411 raft_consensus.cc:515] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.667821 10335 raft_consensus.cc:359] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.670639 10335 raft_consensus.cc:385] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.669822 10331 raft_consensus.cc:359] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.671115 10335 raft_consensus.cc:740] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.672628 10335 consensus_queue.cc:260] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.671515 10331 raft_consensus.cc:385] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.673569 10331 raft_consensus.cc:740] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.671470 10410 raft_consensus.cc:697] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:03:56.674903 10410 consensus_queue.cc:237] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.676783 10331 consensus_queue.cc:260] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.678718 10438 consensus_queue.cc:1048] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:56.679280 10331 ts_tablet_manager.cc:1434] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 13:03:56.680284 10331 tablet_bootstrap.cc:492] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:56.682134 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8d5b10e4fe4ce994c81bf45aa93591" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
W20260709 13:03:56.683652 10105 leader_election.cc:343] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.686537 10335 ts_tablet_manager.cc:1434] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.039s	user 0.006s	sys 0.001s
I20260709 13:03:56.687585 10335 tablet_bootstrap.cc:492] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:56.687796 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8d5b10e4fe4ce994c81bf45aa93591" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:56.689111 10063 raft_consensus.cc:3060] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.692212 10335 tablet_bootstrap.cc:654] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.698921 10331 tablet_bootstrap.cc:654] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.699249 10411 leader_election.cc:290] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:56.702600 10063 raft_consensus.cc:2468] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:03:56.704097 10104 leader_election.cc:304] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:56.705360 10411 raft_consensus.cc:2804] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:56.706173 10411 raft_consensus.cc:697] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:03:56.708896 10438 raft_consensus.cc:493] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.710289 10438 raft_consensus.cc:515] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.711014 10411 consensus_queue.cc:237] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.717588 10438 leader_election.cc:290] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:56.721732 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9b66ba4b81428f896a77bd88bc7e4d" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
W20260709 13:03:56.723603 10251 leader_election.cc:343] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.730019 10392 raft_consensus.cc:493] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.730252 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9b66ba4b81428f896a77bd88bc7e4d" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:56.731267 10065 raft_consensus.cc:2468] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:03:56.732867 10252 leader_election.cc:304] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:03:56.730913 10392 raft_consensus.cc:515] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.737519 10392 leader_election.cc:290] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:56.739151 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:56.740597 10438 raft_consensus.cc:2804] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:56.741140 10438 raft_consensus.cc:493] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.741560 10438 raft_consensus.cc:3060] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:56.744853 10179 leader_election.cc:343] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.747210 10410 consensus_queue.cc:1048] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:56.692795  9962 catalog_manager.cc:5697] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:56.755077 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
W20260709 13:03:56.760648 10178 leader_election.cc:343] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.761355 10178 leader_election.cc:304] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:56.764194 10459 raft_consensus.cc:2749] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:56.773766  9963 catalog_manager.cc:5697] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:56.795413 10439 raft_consensus.cc:493] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.796245 10439 raft_consensus.cc:515] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.800266 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2679daef0c4447c5823833ed4577441c" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:56.799786 10439 leader_election.cc:290] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:56.802124 10410 consensus_queue.cc:1048] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:56.803238 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2679daef0c4447c5823833ed4577441c" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
W20260709 13:03:56.805171 10251 leader_election.cc:343] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.806834 10389 raft_consensus.cc:493] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.808457 10389 raft_consensus.cc:515] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
W20260709 13:03:56.809058 10253 leader_election.cc:343] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.809609 10253 leader_election.cc:304] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:56.810737 10389 leader_election.cc:290] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:56.810859 10439 raft_consensus.cc:2749] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:56.820467 10333 tablet_bootstrap.cc:492] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:56.823228 10333 ts_tablet_manager.cc:1403] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.300s	user 0.016s	sys 0.017s
W20260709 13:03:56.820679 10480 log.cc:927] Time spent T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:03:56.826292 10333 raft_consensus.cc:359] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.826764 10331 tablet_bootstrap.cc:492] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:56.827040 10333 raft_consensus.cc:385] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.827215 10338 tablet_bootstrap.cc:492] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:56.827430 10333 raft_consensus.cc:740] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.827464 10331 ts_tablet_manager.cc:1403] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.147s	user 0.045s	sys 0.067s
I20260709 13:03:56.828289 10333 consensus_queue.cc:260] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.827698 10338 ts_tablet_manager.cc:1403] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.185s	user 0.020s	sys 0.032s
I20260709 13:03:56.818799 10438 raft_consensus.cc:515] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.832383 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0504e36ef8da48b189ebb41995e15efd" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:56.834615 10331 raft_consensus.cc:359] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.835659 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9b66ba4b81428f896a77bd88bc7e4d" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:56.836028 10438 leader_election.cc:290] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:56.837311 10331 raft_consensus.cc:385] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.837715 10331 raft_consensus.cc:740] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.838389 10335 tablet_bootstrap.cc:492] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:56.838925 10335 ts_tablet_manager.cc:1403] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.152s	user 0.024s	sys 0.012s
I20260709 13:03:56.838579 10331 consensus_queue.cc:260] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:56.841269 10331 ts_tablet_manager.cc:1434] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.002s
I20260709 13:03:56.842303 10331 tablet_bootstrap.cc:492] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:56.842432 10335 raft_consensus.cc:359] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.843171 10335 raft_consensus.cc:385] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.843472 10335 raft_consensus.cc:740] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
W20260709 13:03:56.845942 10251 leader_election.cc:343] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.848153 10335 consensus_queue.cc:260] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.849373 10331 tablet_bootstrap.cc:654] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.850397 10335 ts_tablet_manager.cc:1434] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:03:56.851449 10335 tablet_bootstrap.cc:492] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:56.851848  9963 catalog_manager.cc:5697] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:56.853812 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0504e36ef8da48b189ebb41995e15efd" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
W20260709 13:03:56.854331 10253 leader_election.cc:343] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:56.855257 10252 leader_election.cc:343] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.855713 10252 leader_election.cc:304] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:56.857278 10335 tablet_bootstrap.cc:654] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.857102 10338 raft_consensus.cc:359] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.858136 10338 raft_consensus.cc:385] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.858498 10338 raft_consensus.cc:740] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.858839 10438 raft_consensus.cc:2749] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:56.859344 10338 consensus_queue.cc:260] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.862136 10338 ts_tablet_manager.cc:1434] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
W20260709 13:03:56.862545 10179 consensus_peers.cc:597] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:56.880033 10331 tablet_bootstrap.cc:492] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:56.880749 10331 ts_tablet_manager.cc:1403] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.039s	user 0.009s	sys 0.008s
I20260709 13:03:56.884027 10333 ts_tablet_manager.cc:1434] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260709 13:03:56.883745 10331 raft_consensus.cc:359] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.884732 10331 raft_consensus.cc:385] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.885092 10331 raft_consensus.cc:740] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.885667 10459 raft_consensus.cc:493] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.886273 10331 consensus_queue.cc:260] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.886538 10459 raft_consensus.cc:515] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.889078 10331 ts_tablet_manager.cc:1434] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:03:56.890195 10331 tablet_bootstrap.cc:492] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:56.890877 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84c07107a48406d9fd585e9982d3069" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:56.891858 10064 raft_consensus.cc:2468] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:56.893873 10178 leader_election.cc:304] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:56.895023 10453 raft_consensus.cc:2804] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:56.895532 10453 raft_consensus.cc:493] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:56.896073 10453 raft_consensus.cc:3060] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.897279 10331 tablet_bootstrap.cc:654] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.895651 10338 tablet_bootstrap.cc:492] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:56.904335 10438 consensus_queue.cc:1048] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:56.907297 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84c07107a48406d9fd585e9982d3069" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:03:56.909442 10179 leader_election.cc:343] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:56.910681 10331 tablet_bootstrap.cc:492] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:56.911290 10331 ts_tablet_manager.cc:1403] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.002s
I20260709 13:03:56.911118 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9b66ba4b81428f896a77bd88bc7e4d" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:56.915202 10333 tablet_bootstrap.cc:492] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:56.915115 10331 raft_consensus.cc:359] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.916116 10331 raft_consensus.cc:385] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.916472 10331 raft_consensus.cc:740] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.917387 10331 consensus_queue.cc:260] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.920576 10331 ts_tablet_manager.cc:1434] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:03:56.921700 10331 tablet_bootstrap.cc:492] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:56.923791 10338 tablet_bootstrap.cc:654] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.924165 10063 raft_consensus.cc:3060] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.928215 10331 tablet_bootstrap.cc:654] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.937268 10439 raft_consensus.cc:493] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:56.938129 10439 raft_consensus.cc:515] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:56.940348 10439 leader_election.cc:290] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:56.941190 10335 tablet_bootstrap.cc:492] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:56.941792 10335 ts_tablet_manager.cc:1403] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.091s	user 0.036s	sys 0.041s
I20260709 13:03:56.944739 10335 raft_consensus.cc:359] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.945523 10335 raft_consensus.cc:385] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:56.945966 10335 raft_consensus.cc:740] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:56.946319 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d07303bc2640fe9d6ea20f824ce00e" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:56.947142 10213 raft_consensus.cc:2393] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in current term 1: Already voted for candidate bfa051e18d0549eabe79dfcc7610d99b in this term.
I20260709 13:03:56.946794 10335 consensus_queue.cc:260] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:56.949698 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "20d07303bc2640fe9d6ea20f824ce00e" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:56.953620 10253 leader_election.cc:304] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:56.954502 10389 raft_consensus.cc:3060] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.954751 10335 ts_tablet_manager.cc:1434] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 13:03:56.955844 10335 tablet_bootstrap.cc:492] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:56.959232 10392 consensus_queue.cc:1048] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:56.963791 10335 tablet_bootstrap.cc:654] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.968072 10389 raft_consensus.cc:2749] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:56.970271 10459 leader_election.cc:290] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:56.973289 10453 raft_consensus.cc:515] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:56.983042 10453 leader_election.cc:290] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:56.986145 10138 raft_consensus.cc:3060] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:56.986536 10333 tablet_bootstrap.cc:654] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:56.997823 10063 raft_consensus.cc:2468] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:03:57.001880 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84c07107a48406d9fd585e9982d3069" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
W20260709 13:03:57.003912 10179 leader_election.cc:343] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:57.005242 10252 leader_election.cc:304] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:03:57.006798 10389 raft_consensus.cc:2804] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:57.007401 10389 raft_consensus.cc:697] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:03:57.008455 10389 consensus_queue.cc:237] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:57.011334 10337 raft_consensus.cc:493] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.013770 10337 raft_consensus.cc:515] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.018235 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0b03477ff4d1ab861dc093edbe68a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:57.019119 10213 raft_consensus.cc:2468] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:03:57.020776 10028 leader_election.cc:304] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:57.021993 10430 raft_consensus.cc:2804] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:57.022477 10430 raft_consensus.cc:493] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.022907 10430 raft_consensus.cc:3060] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.033579 10337 leader_election.cc:290] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:57.041175 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0b03477ff4d1ab861dc093edbe68a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:57.043515 10390 consensus_queue.cc:1048] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:57.062240 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84c07107a48406d9fd585e9982d3069" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:57.063158 10064 raft_consensus.cc:3060] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.055977 10430 raft_consensus.cc:515] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.074651  9960 catalog_manager.cc:5697] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:57.083272 10498 consensus_queue.cc:1048] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:57.083896 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0b03477ff4d1ab861dc093edbe68a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:57.086062 10211 raft_consensus.cc:3060] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.093392 10498 raft_consensus.cc:493] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.102977 10430 leader_election.cc:290] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:57.112079 10498 raft_consensus.cc:515] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
W20260709 13:03:57.119976 10030 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:33711, user_credentials={real_user=slave}} blocked reactor thread for 77397.8us
I20260709 13:03:57.120496 10498 leader_election.cc:290] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:57.120797 10411 raft_consensus.cc:493] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.125914 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0b03477ff4d1ab861dc093edbe68a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:57.121312 10411 raft_consensus.cc:515] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.129798 10211 raft_consensus.cc:2468] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:03:57.133370 10411 leader_election.cc:290] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:57.138949 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c29bf223e8224f72926bd39f6dcb3f08" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:57.141218 10333 tablet_bootstrap.cc:492] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:57.141902 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:57.142196 10333 ts_tablet_manager.cc:1403] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.227s	user 0.018s	sys 0.026s
I20260709 13:03:57.144503 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c29bf223e8224f72926bd39f6dcb3f08" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:57.146522 10333 raft_consensus.cc:359] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:57.147461 10333 raft_consensus.cc:385] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.147972 10333 raft_consensus.cc:740] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.148900 10333 consensus_queue.cc:260] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:57.147002 10411 raft_consensus.cc:493] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.153450 10335 tablet_bootstrap.cc:492] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:57.154110 10335 ts_tablet_manager.cc:1403] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.199s	user 0.022s	sys 0.049s
I20260709 13:03:57.154592 10338 tablet_bootstrap.cc:492] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:57.155045 10338 ts_tablet_manager.cc:1403] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.260s	user 0.059s	sys 0.060s
I20260709 13:03:57.157011 10333 ts_tablet_manager.cc:1434] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:03:57.150177 10411 raft_consensus.cc:515] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.153182 10103 leader_election.cc:304] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:03:57.158160 10333 tablet_bootstrap.cc:492] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
W20260709 13:03:57.150694 10030 leader_election.cc:343] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:57.162174 10030 leader_election.cc:343] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:57.162386 10500 consensus_queue.cc:1048] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:03:57.163580 10030 leader_election.cc:343] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:57.163941 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2679daef0c4447c5823833ed4577441c" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:57.164615 10287 raft_consensus.cc:2468] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:03:57.165953 10479 raft_consensus.cc:2749] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:57.166994 10105 leader_election.cc:304] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:57.167196 10028 leader_election.cc:304] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:57.168006 10479 raft_consensus.cc:2804] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:57.168546 10479 raft_consensus.cc:493] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.168967 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2679daef0c4447c5823833ed4577441c" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:57.169226 10479 raft_consensus.cc:3060] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.168053 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:57.171043 10213 raft_consensus.cc:2468] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:03:57.168668 10337 raft_consensus.cc:2804] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:57.162317 10411 leader_election.cc:290] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:57.172000 10338 raft_consensus.cc:359] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
W20260709 13:03:57.174309 10103 leader_election.cc:343] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:57.177549 10338 raft_consensus.cc:385] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.178056 10338 raft_consensus.cc:740] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.178663 10338 consensus_queue.cc:260] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:57.181140 10064 raft_consensus.cc:2468] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:57.184618 10479 raft_consensus.cc:515] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.185616 10337 raft_consensus.cc:697] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:03:57.186460 10178 leader_election.cc:304] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:57.187223 10337 consensus_queue.cc:237] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.187716 10392 raft_consensus.cc:2804] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:57.188247 10333 tablet_bootstrap.cc:654] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:57.188259 10392 raft_consensus.cc:697] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:03:57.186235 10502 raft_consensus.cc:493] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.190161 10411 raft_consensus.cc:493] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.190097 10392 consensus_queue.cc:237] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:57.190606 10502 raft_consensus.cc:515] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:57.190747 10411 raft_consensus.cc:515] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.193003 10411 leader_election.cc:290] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:57.194998 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a68082d0468d49ada47812f6c8e3f106" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:57.193003 10502 leader_election.cc:290] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:57.195730 10287 raft_consensus.cc:2468] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:03:57.195924 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2679daef0c4447c5823833ed4577441c" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:57.197463 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0709ff33d24996ae638187a8e7c57f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
W20260709 13:03:57.198017 10103 leader_election.cc:343] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:57.195000 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0709ff33d24996ae638187a8e7c57f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:57.203969 10331 tablet_bootstrap.cc:492] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:57.204671 10331 ts_tablet_manager.cc:1403] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.283s	user 0.039s	sys 0.076s
I20260709 13:03:57.223098 10338 ts_tablet_manager.cc:1434] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 13:03:57.224697 10338 tablet_bootstrap.cc:492] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:57.200263 10103 leader_election.cc:304] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:03:57.237234 10333 tablet_bootstrap.cc:492] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:57.237761 10333 ts_tablet_manager.cc:1403] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.080s	user 0.015s	sys 0.032s
I20260709 13:03:57.238277 10410 raft_consensus.cc:2749] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:57.239733 10479 leader_election.cc:290] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:57.239976 10335 raft_consensus.cc:359] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.232062 10028 leader_election.cc:304] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:57.246543 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a68082d0468d49ada47812f6c8e3f106" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:57.237120 10030 leader_election.cc:304] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 
I20260709 13:03:57.246131 10335 raft_consensus.cc:385] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.249089 10335 raft_consensus.cc:740] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.249873 10335 consensus_queue.cc:260] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.250104 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2679daef0c4447c5823833ed4577441c" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:57.251681 10337 raft_consensus.cc:2804] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:57.251856 10502 raft_consensus.cc:2804] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:57.252554 10502 raft_consensus.cc:493] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.252025 10287 raft_consensus.cc:3060] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.252951 10502 raft_consensus.cc:3060] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.253598 10411 consensus_queue.cc:1048] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:57.264734 10333 raft_consensus.cc:359] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.265709 10397 raft_consensus.cc:493] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.265816 10333 raft_consensus.cc:385] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.266633 10335 ts_tablet_manager.cc:1434] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.112s	user 0.012s	sys 0.000s
I20260709 13:03:57.268800 10335 tablet_bootstrap.cc:492] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:57.269366 10331 raft_consensus.cc:359] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:57.270354 10331 raft_consensus.cc:385] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.271015 10331 raft_consensus.cc:740] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
W20260709 13:03:57.265945 10030 leader_election.cc:343] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:57.248616 10103 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 48463us
I20260709 13:03:57.274060 10331 consensus_queue.cc:260] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:57.266573 10333 raft_consensus.cc:740] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.261560 10337 raft_consensus.cc:493] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.276235 10502 raft_consensus.cc:515] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:57.278859 10331 ts_tablet_manager.cc:1434] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.074s	user 0.008s	sys 0.000s
I20260709 13:03:57.282223 10331 tablet_bootstrap.cc:492] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:57.285972 10333 consensus_queue.cc:260] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.266433 10397 raft_consensus.cc:515] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:57.280874 10337 raft_consensus.cc:3060] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.295444 10335 tablet_bootstrap.cc:654] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:57.299122 10333 ts_tablet_manager.cc:1434] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.055s	user 0.005s	sys 0.003s
I20260709 13:03:57.299997 10333 tablet_bootstrap.cc:492] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:57.311089 10397 leader_election.cc:290] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:57.312618 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9b66ba4b81428f896a77bd88bc7e4d" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:57.308741 10502 leader_election.cc:290] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:57.315162 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9b66ba4b81428f896a77bd88bc7e4d" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:57.315948 10063 raft_consensus.cc:2393] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bcf45a2176234db5957dc38e32554577 in current term 1: Already voted for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in this term.
I20260709 13:03:57.316696 10410 consensus_queue.cc:1048] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:57.318801 10338 tablet_bootstrap.cc:654] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:57.318985 10105 consensus_peers.cc:597] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b -> Peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Couldn't send request to peer 8c3d3c9022c94705ac4ee834e1dc49d0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:57.320309 10337 raft_consensus.cc:515] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:57.322726 10410 consensus_queue.cc:1048] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:57.323103 10178 leader_election.cc:304] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:57.324127  9962 catalog_manager.cc:5697] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:57.322734 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a68082d0468d49ada47812f6c8e3f106" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:57.324779 10459 raft_consensus.cc:3060] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.329282 10333 tablet_bootstrap.cc:654] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
W20260709 13:03:57.329756 10030 leader_election.cc:343] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:57.330268 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a68082d0468d49ada47812f6c8e3f106" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:57.330932 10288 raft_consensus.cc:3060] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.335214 10287 raft_consensus.cc:2468] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:03:57.325176 10331 tablet_bootstrap.cc:654] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:57.378799 10337 leader_election.cc:290] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:57.379909 10105 leader_election.cc:304] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:03:57.381338 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:57.382053 10211 raft_consensus.cc:3060] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.382617 10459 raft_consensus.cc:2749] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:57.383216 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
W20260709 13:03:57.385161 10030 leader_election.cc:343] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:57.387531 10288 raft_consensus.cc:2468] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:03:57.387662 10411 raft_consensus.cc:2804] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:57.394896 10411 raft_consensus.cc:697] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:03:57.399014 10411 consensus_queue.cc:237] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.401525 10030 leader_election.cc:304] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:57.402673 10211 raft_consensus.cc:2468] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:03:57.403017 10502 raft_consensus.cc:2804] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:57.403512 10502 raft_consensus.cc:697] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:03:57.405761 10028 leader_election.cc:304] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:57.406186 10502 consensus_queue.cc:237] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:57.407877 10338 tablet_bootstrap.cc:492] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:57.408398 10338 ts_tablet_manager.cc:1403] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.184s	user 0.028s	sys 0.005s
I20260709 13:03:57.416730 10500 raft_consensus.cc:2804] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:57.417341 10500 raft_consensus.cc:697] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:03:57.418246 10500 consensus_queue.cc:237] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:57.420089 10338 raft_consensus.cc:359] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:57.421339 10338 raft_consensus.cc:385] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.421638 10338 raft_consensus.cc:740] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.422407 10338 consensus_queue.cc:260] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
W20260709 13:03:57.423833 10105 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:42219, user_credentials={real_user=slave}} blocked reactor thread for 47799.2us
I20260709 13:03:57.437352 10138 raft_consensus.cc:3060] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.443858 10338 ts_tablet_manager.cc:1434] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:03:57.445362 10338 tablet_bootstrap.cc:492] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:57.429414 10410 raft_consensus.cc:493] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.457248 10331 tablet_bootstrap.cc:492] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:57.457842 10331 ts_tablet_manager.cc:1403] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.176s	user 0.005s	sys 0.011s
I20260709 13:03:57.464041 10333 tablet_bootstrap.cc:492] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:57.464563 10333 ts_tablet_manager.cc:1403] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.165s	user 0.019s	sys 0.023s
I20260709 13:03:57.453680 10410 raft_consensus.cc:515] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.469179 10410 leader_election.cc:290] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:57.469817 10338 tablet_bootstrap.cc:654] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:57.469774 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0b03477ff4d1ab861dc093edbe68a" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:57.469774 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0b03477ff4d1ab861dc093edbe68a" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:57.470836 10211 raft_consensus.cc:2393] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bfa051e18d0549eabe79dfcc7610d99b in current term 1: Already voted for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in this term.
I20260709 13:03:57.474247 10103 leader_election.cc:304] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:03:57.477878 10331 raft_consensus.cc:359] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:57.479718 10331 raft_consensus.cc:385] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.479981 10331 raft_consensus.cc:740] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.485944 10331 consensus_queue.cc:260] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:57.488945  9960 catalog_manager.cc:5697] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:57.493909 10335 tablet_bootstrap.cc:492] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:57.494472 10335 ts_tablet_manager.cc:1403] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.226s	user 0.015s	sys 0.008s
I20260709 13:03:57.494694  9963 catalog_manager.cc:5697] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:57.497212 10335 raft_consensus.cc:359] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.498484 10335 raft_consensus.cc:385] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.498776 10335 raft_consensus.cc:740] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.499413 10335 consensus_queue.cc:260] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.503592 10335 ts_tablet_manager.cc:1434] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:03:57.504567 10430 raft_consensus.cc:493] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.479358 10410 raft_consensus.cc:3060] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.505162 10430 raft_consensus.cc:515] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:57.504613 10335 tablet_bootstrap.cc:492] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:57.507560 10333 raft_consensus.cc:359] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.509339 10333 raft_consensus.cc:385] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.509652 10333 raft_consensus.cc:740] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.508430 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9785f60ee640d686f599267cf005fb" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
W20260709 13:03:57.511790 10028 leader_election.cc:343] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:57.507622 10430 leader_election.cc:290] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:57.520114 10331 ts_tablet_manager.cc:1434] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.062s	user 0.004s	sys 0.002s
I20260709 13:03:57.519804 10333 consensus_queue.cc:260] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.523976 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9785f60ee640d686f599267cf005fb" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:03:57.526489 10030 leader_election.cc:343] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:57.526985 10030 leader_election.cc:304] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:03:57.527281 10389 consensus_queue.cc:1048] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:57.529148 10333 ts_tablet_manager.cc:1434] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.064s	user 0.006s	sys 0.001s
I20260709 13:03:57.530424 10333 tablet_bootstrap.cc:492] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:57.534772 10338 tablet_bootstrap.cc:492] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:57.535181 10338 ts_tablet_manager.cc:1403] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.090s	user 0.014s	sys 0.029s
I20260709 13:03:57.536998 10331 tablet_bootstrap.cc:492] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:57.535714 10430 raft_consensus.cc:2749] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:57.552783 10410 raft_consensus.cc:2749] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:57.555670 10338 raft_consensus.cc:359] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:57.556375 10338 raft_consensus.cc:385] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.556603 10338 raft_consensus.cc:740] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.559718 10333 tablet_bootstrap.cc:654] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:57.560482 10338 consensus_queue.cc:260] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:57.563035 10335 tablet_bootstrap.cc:654] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:57.568148 10338 ts_tablet_manager.cc:1434] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.003s
I20260709 13:03:57.569777 10338 tablet_bootstrap.cc:492] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:57.570835 10331 tablet_bootstrap.cc:654] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:57.575698 10333 tablet_bootstrap.cc:492] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:57.576244 10333 ts_tablet_manager.cc:1403] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.002s
I20260709 13:03:57.579794 10338 tablet_bootstrap.cc:654] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:57.580917 10335 tablet_bootstrap.cc:492] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:57.581580 10335 ts_tablet_manager.cc:1403] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.077s	user 0.009s	sys 0.004s
I20260709 13:03:57.628405 10335 raft_consensus.cc:359] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:57.630628 10335 raft_consensus.cc:385] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.631079 10335 raft_consensus.cc:740] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.632156 10335 consensus_queue.cc:260] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:57.634361 10333 raft_consensus.cc:359] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.635712 10333 raft_consensus.cc:385] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.636124 10333 raft_consensus.cc:740] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.636775 10390 consensus_queue.cc:1048] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:03:57.640911 10333 consensus_queue.cc:260] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.655431 10459 raft_consensus.cc:493] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.656663 10459 raft_consensus.cc:515] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.670409 10502 consensus_queue.cc:1048] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:57.683522 10331 tablet_bootstrap.cc:492] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:57.689640 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2679daef0c4447c5823833ed4577441c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:57.702697 10331 ts_tablet_manager.cc:1403] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.169s	user 0.014s	sys 0.005s
I20260709 13:03:57.706140 10410 raft_consensus.cc:493] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.706934 10410 raft_consensus.cc:515] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:57.707811 10459 leader_election.cc:290] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:57.710282 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2679daef0c4447c5823833ed4577441c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:57.719515 10288 raft_consensus.cc:2393] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bcf45a2176234db5957dc38e32554577 in current term 1: Already voted for candidate bfa051e18d0549eabe79dfcc7610d99b in this term.
I20260709 13:03:57.719521 10338 tablet_bootstrap.cc:492] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:57.722097 10397 raft_consensus.cc:493] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.723255 10338 ts_tablet_manager.cc:1403] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.154s	user 0.009s	sys 0.009s
I20260709 13:03:57.723019 10397 raft_consensus.cc:515] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.706178 10333 ts_tablet_manager.cc:1434] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.130s	user 0.010s	sys 0.002s
I20260709 13:03:57.711424 10335 ts_tablet_manager.cc:1434] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.129s	user 0.006s	sys 0.008s
I20260709 13:03:57.721518 10179 leader_election.cc:304] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:57.731076 10397 leader_election.cc:290] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:57.739632 10337 consensus_queue.cc:1048] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:57.742215 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0007e852f46b2b3ce899d1101e17c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:57.742950 10335 tablet_bootstrap.cc:492] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:57.742560 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0007e852f46b2b3ce899d1101e17c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:57.743273 10139 raft_consensus.cc:2468] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:57.744697 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0504e36ef8da48b189ebb41995e15efd" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:57.745692 10288 raft_consensus.cc:2468] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:03:57.747452 10105 leader_election.cc:304] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:57.748605 10514 raft_consensus.cc:2804] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:57.749045 10514 raft_consensus.cc:493] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.749383 10514 raft_consensus.cc:3060] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.753857 10410 leader_election.cc:290] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:57.755569 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0504e36ef8da48b189ebb41995e15efd" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:57.736127 10459 raft_consensus.cc:3060] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.766059 10179 leader_election.cc:304] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:57.766577 10333 tablet_bootstrap.cc:492] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:57.767118 10397 raft_consensus.cc:2804] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:03:57.767446 10179 consensus_peers.cc:597] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 -> Peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Couldn't send request to peer 8c3d3c9022c94705ac4ee834e1dc49d0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:57.767606 10397 raft_consensus.cc:493] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:57.768112 10397 raft_consensus.cc:3060] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:57.768252 10179 leader_election.cc:343] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:57.785212 10104 leader_election.cc:343] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:57.782528  9962 catalog_manager.cc:5697] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:57.798301 10392 consensus_queue.cc:1048] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:57.799923 10397 raft_consensus.cc:515] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.801728 10411 consensus_queue.cc:1048] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:57.802174 10397 leader_election.cc:290] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:57.783887 10335 tablet_bootstrap.cc:654] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:57.802372 10333 tablet_bootstrap.cc:654] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:57.804080 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0007e852f46b2b3ce899d1101e17c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:57.809434 10285 raft_consensus.cc:3060] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.785138 10338 raft_consensus.cc:359] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.820139 10338 raft_consensus.cc:385] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.820580 10338 raft_consensus.cc:740] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.822938 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0007e852f46b2b3ce899d1101e17c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:57.823930 10138 raft_consensus.cc:3060] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.827641 10338 consensus_queue.cc:260] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.831385 10286 raft_consensus.cc:3060] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:57.769737 10331 raft_consensus.cc:359] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.835603 10331 raft_consensus.cc:385] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.835922 10331 raft_consensus.cc:740] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.840288 10331 consensus_queue.cc:260] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.842072 10331 ts_tablet_manager.cc:1434] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.139s	user 0.010s	sys 0.000s
I20260709 13:03:57.843528 10331 tablet_bootstrap.cc:492] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:57.799453  9962 catalog_manager.cc:5697] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:57.861274 10331 tablet_bootstrap.cc:654] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:57.869493 10397 raft_consensus.cc:493] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.870280 10397 raft_consensus.cc:515] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:57.870959 10502 raft_consensus.cc:493] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.871773 10502 raft_consensus.cc:515] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:57.872676 10397 leader_election.cc:290] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:57.876498 10502 leader_election.cc:290] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:57.879976 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde8fc103c3d453abd4d298b1beffd74" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
W20260709 13:03:57.882944 10030 leader_election.cc:343] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:57.885939 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102b42f582ce49458990caefcc8d451c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:57.808557 10514 raft_consensus.cc:515] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:57.856963 10286 raft_consensus.cc:2468] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
W20260709 13:03:57.894855 10179 leader_election.cc:343] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:57.913646 10389 raft_consensus.cc:493] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.914537 10389 raft_consensus.cc:515] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:57.909991 10138 raft_consensus.cc:2468] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:57.918013 10333 tablet_bootstrap.cc:492] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:57.918736 10333 ts_tablet_manager.cc:1403] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.153s	user 0.032s	sys 0.021s
I20260709 13:03:57.922602 10500 consensus_queue.cc:1048] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:57.941946 10533 raft_consensus.cc:493] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.942494 10333 raft_consensus.cc:359] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.943490 10333 raft_consensus.cc:385] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:57.943917 10333 raft_consensus.cc:740] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:57.944766 10333 consensus_queue.cc:260] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.947979 10333 ts_tablet_manager.cc:1434] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.010s
I20260709 13:03:57.949384 10333 tablet_bootstrap.cc:492] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:57.949591 10179 leader_election.cc:304] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:57.952219 10397 raft_consensus.cc:2804] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:57.907935 10514 leader_election.cc:290] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:57.942559 10533 raft_consensus.cc:515] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:57.943945 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102b42f582ce49458990caefcc8d451c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:57.939985 10390 raft_consensus.cc:493] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:57.979784 10333 tablet_bootstrap.cc:654] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:57.980942 10502 consensus_queue.cc:1048] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:57.961536 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0504e36ef8da48b189ebb41995e15efd" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:57.969910 10459 raft_consensus.cc:2749] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:57.998759 10335 tablet_bootstrap.cc:492] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:57.999436 10335 ts_tablet_manager.cc:1403] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.257s	user 0.089s	sys 0.082s
I20260709 13:03:57.999645 10331 tablet_bootstrap.cc:492] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:58.000330 10331 ts_tablet_manager.cc:1403] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.157s	user 0.022s	sys 0.027s
I20260709 13:03:58.014285 10397 raft_consensus.cc:697] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:03:58.017668 10397 consensus_queue.cc:237] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.003103 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0504e36ef8da48b189ebb41995e15efd" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:57.995023 10390 raft_consensus.cc:515] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.044520 10533 leader_election.cc:290] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
W20260709 13:03:58.047108 10104 leader_election.cc:343] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.051123 10284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde8fc103c3d453abd4d298b1beffd74" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:58.055773 10390 leader_election.cc:290] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
W20260709 13:03:58.059506 10011 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:58.063148 10284 raft_consensus.cc:2468] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
W20260709 13:03:58.065699 10179 leader_election.cc:343] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.066563 10179 leader_election.cc:304] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:58.037614 10288 raft_consensus.cc:3060] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.071928 10533 raft_consensus.cc:2749] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:58.065608 10335 raft_consensus.cc:359] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:58.075393 10335 raft_consensus.cc:385] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.074862 10333 tablet_bootstrap.cc:492] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:58.075735 10335 raft_consensus.cc:740] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.076268 10333 ts_tablet_manager.cc:1403] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.127s	user 0.026s	sys 0.017s
I20260709 13:03:58.065475 10331 raft_consensus.cc:359] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.080865 10331 raft_consensus.cc:385] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.081163 10331 raft_consensus.cc:740] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.078996 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf60e84958749f9826c9e8de7d266d9" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:58.091115 10335 consensus_queue.cc:260] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:58.101094 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:58.105191 10211 raft_consensus.cc:2468] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:03:58.106367 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde8fc103c3d453abd4d298b1beffd74" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:58.103971 10331 consensus_queue.cc:260] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.113632 10063 raft_consensus.cc:2468] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:58.120008 10338 ts_tablet_manager.cc:1434] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.396s	user 0.019s	sys 0.044s
I20260709 13:03:58.121637 10338 tablet_bootstrap.cc:492] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
W20260709 13:03:58.129393 10253 leader_election.cc:343] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.131021 10514 consensus_queue.cc:1048] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:58.114768 10389 leader_election.cc:290] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:58.137055 10335 ts_tablet_manager.cc:1434] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.137s	user 0.009s	sys 0.000s
I20260709 13:03:58.138902 10335 tablet_bootstrap.cc:492] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:58.141448 10392 raft_consensus.cc:493] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.142038 10392 raft_consensus.cc:515] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:58.144479 10392 leader_election.cc:290] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:58.149307 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde8fc103c3d453abd4d298b1beffd74" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:58.125272 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf60e84958749f9826c9e8de7d266d9" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:58.151263 10138 raft_consensus.cc:2468] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:58.150342 10063 raft_consensus.cc:2468] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:03:58.154911 10333 raft_consensus.cc:359] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.160264 10333 raft_consensus.cc:385] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.165696 10333 raft_consensus.cc:740] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.169505 10430 consensus_queue.cc:1048] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:03:58.160822 10178 leader_election.cc:304] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:58.177372 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be75bbbdef504a5793ba0b07feaff6f8" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:58.181802 10392 raft_consensus.cc:2804] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:58.182475 10392 raft_consensus.cc:493] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.183010 10392 raft_consensus.cc:3060] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.188184 10288 raft_consensus.cc:2468] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:03:58.196223 10333 consensus_queue.cc:260] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
W20260709 13:03:58.206225 10179 leader_election.cc:343] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:58.201148 10028 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 87425.5us
I20260709 13:03:58.210830 10105 leader_election.cc:304] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:03:58.216266 10514 raft_consensus.cc:2804] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:58.216850 10514 raft_consensus.cc:697] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:03:58.217738 10514 consensus_queue.cc:237] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.219465 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be75bbbdef504a5793ba0b07feaff6f8" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:58.220862 10063 raft_consensus.cc:2468] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:58.225626 10252 leader_election.cc:304] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bfa051e18d0549eabe79dfcc7610d99b
W20260709 13:03:58.232224 10030 consensus_peers.cc:597] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:58.250114 10333 ts_tablet_manager.cc:1434] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.173s	user 0.006s	sys 0.003s
I20260709 13:03:58.251854 10544 raft_consensus.cc:2804] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:58.252316 10544 raft_consensus.cc:493] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.252650 10544 raft_consensus.cc:3060] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.256397 10251 leader_election.cc:304] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
W20260709 13:03:58.257241 10251 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 96110.8us
I20260709 13:03:58.258637 10541 raft_consensus.cc:2804] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:03:58.259162 10541 raft_consensus.cc:493] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:03:58.259591 10541 raft_consensus.cc:3060] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:58.267159 10335 tablet_bootstrap.cc:654] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.246847 10392 raft_consensus.cc:515] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:58.269093 10333 tablet_bootstrap.cc:492] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
W20260709 13:03:58.269307 10252 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 44120.4us
I20260709 13:03:58.273918 10331 ts_tablet_manager.cc:1434] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.273s	user 0.012s	sys 0.008s
I20260709 13:03:58.276466 10397 raft_consensus.cc:493] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.266258 10544 raft_consensus.cc:515] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.285589 10397 raft_consensus.cc:515] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.287130 10338 tablet_bootstrap.cc:654] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.288470 10397 leader_election.cc:290] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:58.298895 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9785f60ee640d686f599267cf005fb" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:58.301206 10331 tablet_bootstrap.cc:492] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:58.303673 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde8fc103c3d453abd4d298b1beffd74" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
W20260709 13:03:58.306025 10253 leader_election.cc:343] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.306949 10392 leader_election.cc:290] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:58.307693 10544 leader_election.cc:290] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:58.300973 10541 raft_consensus.cc:515] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.313863 10338 tablet_bootstrap.cc:492] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:58.313683 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
W20260709 13:03:58.314145 10179 leader_election.cc:343] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.314309 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf60e84958749f9826c9e8de7d266d9" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:58.315021 10139 raft_consensus.cc:3060] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.317072 10338 ts_tablet_manager.cc:1403] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.196s	user 0.012s	sys 0.019s
I20260709 13:03:58.320703 10541 leader_election.cc:290] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:58.321682 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:58.325232 10178 leader_election.cc:304] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:58.326447 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde8fc103c3d453abd4d298b1beffd74" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:58.327226 10063 raft_consensus.cc:3060] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.326689 10397 raft_consensus.cc:2804] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:58.327739 10397 raft_consensus.cc:493] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.328146 10397 raft_consensus.cc:3060] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.329073 10333 tablet_bootstrap.cc:654] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.329595 10030 leader_election.cc:304] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:58.332968  9963 catalog_manager.cc:5697] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:58.336160 10331 tablet_bootstrap.cc:654] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.341180 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9785f60ee640d686f599267cf005fb" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:58.342304 10062 raft_consensus.cc:2468] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:58.348367 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf60e84958749f9826c9e8de7d266d9" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:58.349484 10062 raft_consensus.cc:3060] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.355194 10139 raft_consensus.cc:2468] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:58.356945 10179 leader_election.cc:304] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:58.357951 10392 raft_consensus.cc:2804] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:58.358657 10392 raft_consensus.cc:697] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:03:58.359572 10392 consensus_queue.cc:237] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:58.360616 10063 raft_consensus.cc:2468] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:03:58.361405 10546 raft_consensus.cc:2764] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:03:58.345002 10397 raft_consensus.cc:515] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:58.352917 10338 raft_consensus.cc:359] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.365653 10338 raft_consensus.cc:385] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.366044 10338 raft_consensus.cc:740] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.363907 10335 tablet_bootstrap.cc:492] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:58.367964 10397 leader_election.cc:290] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:58.369716 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be75bbbdef504a5793ba0b07feaff6f8" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:58.372375 10335 ts_tablet_manager.cc:1403] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.234s	user 0.013s	sys 0.007s
W20260709 13:03:58.372623 10179 leader_election.cc:343] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.376227 10335 raft_consensus.cc:359] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.379058 10335 raft_consensus.cc:385] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.377588 10338 consensus_queue.cc:260] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.379863 10335 raft_consensus.cc:740] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.380985 10335 consensus_queue.cc:260] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.386960 10333 tablet_bootstrap.cc:492] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:58.387516 10333 ts_tablet_manager.cc:1403] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.119s	user 0.025s	sys 0.024s
I20260709 13:03:58.387326 10178 leader_election.cc:304] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:58.388260 10397 raft_consensus.cc:2804] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:58.388720 10397 raft_consensus.cc:493] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.389060 10397 raft_consensus.cc:3060] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.389846 10252 leader_election.cc:304] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:58.390807 10333 raft_consensus.cc:359] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.391211 10544 raft_consensus.cc:2804] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:58.391588 10333 raft_consensus.cc:385] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.391813 10544 raft_consensus.cc:697] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:03:58.391942 10333 raft_consensus.cc:740] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.388304 10288 raft_consensus.cc:1240] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Rejecting Update request from peer 5e0edfcb63bb4d16ac123a82ecb7f331 for earlier term 1. Current term is 2. Ops: []
I20260709 13:03:58.392817 10333 consensus_queue.cc:260] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.393918 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:58.392932 10544 consensus_queue.cc:237] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.396258 10252 leader_election.cc:304] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:03:58.396823 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be75bbbdef504a5793ba0b07feaff6f8" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:58.397439 10063 raft_consensus.cc:3060] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.398686 10553 consensus_queue.cc:1059] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:58.399528 10553 raft_consensus.cc:3055] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:03:58.399880 10553 raft_consensus.cc:740] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:03:58.399943 10338 ts_tablet_manager.cc:1434] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.083s	user 0.008s	sys 0.000s
I20260709 13:03:58.400765 10541 raft_consensus.cc:2749] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:03:58.400825 10553 consensus_queue.cc:260] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.401228 10338 tablet_bootstrap.cc:492] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:58.403337 10062 raft_consensus.cc:2468] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:58.403492  9962 catalog_manager.cc:5697] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.403821 10553 raft_consensus.cc:3060] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:03:58.408607 10537 consensus_queue.cc:1048] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:58.416509 10333 ts_tablet_manager.cc:1434] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.029s	user 0.010s	sys 0.000s
I20260709 13:03:58.417832 10333 tablet_bootstrap.cc:492] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:58.430897 10335 ts_tablet_manager.cc:1434] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.001s
I20260709 13:03:58.437072 10335 tablet_bootstrap.cc:492] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:58.442844 10397 raft_consensus.cc:515] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.444626 10335 tablet_bootstrap.cc:654] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.445282 10397 leader_election.cc:290] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:58.446437 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9785f60ee640d686f599267cf005fb" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:58.447072 10064 raft_consensus.cc:3060] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.448254 10333 tablet_bootstrap.cc:654] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.448053 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9785f60ee640d686f599267cf005fb" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:58.448752 10283 raft_consensus.cc:3060] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.456247 10338 tablet_bootstrap.cc:654] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.455778  9963 catalog_manager.cc:5697] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.461403 10331 tablet_bootstrap.cc:492] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:58.461998 10331 ts_tablet_manager.cc:1403] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.161s	user 0.026s	sys 0.013s
I20260709 13:03:58.475641 10514 raft_consensus.cc:493] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.476253 10514 raft_consensus.cc:515] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.481065 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102b42f582ce49458990caefcc8d451c" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:58.485359 10211 raft_consensus.cc:2468] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:03:58.487684 10103 leader_election.cc:304] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:58.491748 10411 raft_consensus.cc:2804] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:58.492327 10411 raft_consensus.cc:493] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.492746 10411 raft_consensus.cc:3060] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.435655 10063 raft_consensus.cc:2468] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:58.501120 10178 leader_election.cc:304] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:58.501513 10514 leader_election.cc:290] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:58.503206 10397 raft_consensus.cc:2804] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:58.503746 10397 raft_consensus.cc:697] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:03:58.505522 10397 consensus_queue.cc:237] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:58.506009 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102b42f582ce49458990caefcc8d451c" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:03:58.509815 10105 leader_election.cc:343] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.524422 10064 raft_consensus.cc:2468] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:58.526320 10178 leader_election.cc:304] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:58.532094 10537 raft_consensus.cc:2804] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:58.532619 10537 raft_consensus.cc:697] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:03:58.533483 10537 consensus_queue.cc:237] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.548866 10338 tablet_bootstrap.cc:492] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:58.549470 10338 ts_tablet_manager.cc:1403] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.149s	user 0.038s	sys 0.012s
I20260709 13:03:58.549907 10283 raft_consensus.cc:2468] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:58.549813  9960 catalog_manager.cc:5697] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.551963 10331 raft_consensus.cc:359] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.552290 10338 raft_consensus.cc:359] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.552724 10331 raft_consensus.cc:385] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.552984 10338 raft_consensus.cc:385] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.553056 10331 raft_consensus.cc:740] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.553366 10338 raft_consensus.cc:740] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.554428 10331 consensus_queue.cc:260] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.554706 10338 consensus_queue.cc:260] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.558145 10331 ts_tablet_manager.cc:1434] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 13:03:58.563340 10338 ts_tablet_manager.cc:1434] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:03:58.565056 10335 tablet_bootstrap.cc:492] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:58.571659 10335 ts_tablet_manager.cc:1403] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.135s	user 0.037s	sys 0.030s
I20260709 13:03:58.572433 10333 tablet_bootstrap.cc:492] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:58.574143 10333 ts_tablet_manager.cc:1403] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.157s	user 0.012s	sys 0.004s
I20260709 13:03:58.573315 10537 consensus_queue.cc:1048] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:58.564608 10411 raft_consensus.cc:515] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.575783 10335 raft_consensus.cc:359] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.578524 10335 raft_consensus.cc:385] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.578773 10335 raft_consensus.cc:740] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.580808 10335 consensus_queue.cc:260] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.576912 10553 raft_consensus.cc:493] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.591391 10338 tablet_bootstrap.cc:492] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:58.590327 10335 ts_tablet_manager.cc:1434] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.001s
I20260709 13:03:58.591957 10331 tablet_bootstrap.cc:492] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:58.598613 10335 tablet_bootstrap.cc:492] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:58.591118 10553 raft_consensus.cc:515] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.603942 10331 tablet_bootstrap.cc:654] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.605890 10553 leader_election.cc:290] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:58.606094 10335 tablet_bootstrap.cc:654] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.607623 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0504e36ef8da48b189ebb41995e15efd" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:58.612335 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0504e36ef8da48b189ebb41995e15efd" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:58.615075 10283 raft_consensus.cc:2393] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in current term 1: Already voted for candidate bfa051e18d0549eabe79dfcc7610d99b in this term.
I20260709 13:03:58.617093 10030 leader_election.cc:304] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:58.617956 10514 consensus_queue.cc:1048] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:03:58.627197 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102b42f582ce49458990caefcc8d451c" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:58.627868 10211 raft_consensus.cc:3060] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.630440 10537 raft_consensus.cc:493] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.631556 10537 raft_consensus.cc:515] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.633872 10537 leader_election.cc:290] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:58.635638 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd01deadf87945439c630e949d9c2e7b" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
W20260709 13:03:58.638092 10178 leader_election.cc:343] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.640030 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd01deadf87945439c630e949d9c2e7b" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:58.622517 10553 raft_consensus.cc:3060] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:58.652120 10179 leader_election.cc:343] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.652652 10179 leader_election.cc:304] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:58.653806 10397 raft_consensus.cc:2749] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:58.662441 10338 tablet_bootstrap.cc:654] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.668159 10333 raft_consensus.cc:359] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.669942 10333 raft_consensus.cc:385] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.670231 10333 raft_consensus.cc:740] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.674476 10335 tablet_bootstrap.cc:492] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:58.675030 10335 ts_tablet_manager.cc:1403] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.077s	user 0.009s	sys 0.012s
I20260709 13:03:58.676142 10333 consensus_queue.cc:260] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.680862 10333 ts_tablet_manager.cc:1434] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.106s	user 0.003s	sys 0.003s
I20260709 13:03:58.682281 10333 tablet_bootstrap.cc:492] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:58.681394 10335 raft_consensus.cc:359] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.682741 10335 raft_consensus.cc:385] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.683054 10335 raft_consensus.cc:740] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.684230 10411 leader_election.cc:290] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:58.685089 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102b42f582ce49458990caefcc8d451c" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:58.686440 10331 tablet_bootstrap.cc:492] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:58.686941 10331 ts_tablet_manager.cc:1403] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.106s	user 0.028s	sys 0.011s
W20260709 13:03:58.687085 10105 leader_election.cc:343] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.689162  9963 catalog_manager.cc:5697] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:58.689975 10331 raft_consensus.cc:359] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.690681 10331 raft_consensus.cc:385] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.690969 10331 raft_consensus.cc:740] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.691900 10331 consensus_queue.cc:260] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.692984 10333 tablet_bootstrap.cc:654] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.694492 10331 ts_tablet_manager.cc:1434] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:58.683835 10335 consensus_queue.cc:260] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.678730 10211 raft_consensus.cc:2468] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:03:58.698709 10335 ts_tablet_manager.cc:1434] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:03:58.705293 10335 tablet_bootstrap.cc:492] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:58.707348 10338 tablet_bootstrap.cc:492] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:58.707891 10338 ts_tablet_manager.cc:1403] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.127s	user 0.011s	sys 0.003s
I20260709 13:03:58.709662 10103 leader_election.cc:304] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:58.710780 10411 raft_consensus.cc:2804] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:58.711328 10411 raft_consensus.cc:697] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:03:58.711512 10331 tablet_bootstrap.cc:492] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:58.714064 10411 consensus_queue.cc:237] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.716217 10333 tablet_bootstrap.cc:492] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:58.716841 10333 ts_tablet_manager.cc:1403] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.005s
I20260709 13:03:58.720247 10333 raft_consensus.cc:359] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.720449 10338 raft_consensus.cc:359] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.721525 10338 raft_consensus.cc:385] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.716513  9962 catalog_manager.cc:5697] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:58.721884 10338 raft_consensus.cc:740] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.724167 10338 consensus_queue.cc:260] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.726307 10541 raft_consensus.cc:493] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.726773 10541 raft_consensus.cc:515] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.729209 10331 tablet_bootstrap.cc:654] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.730084 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae810c860eb4b05a93e815248f988fc" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:58.730832 10541 leader_election.cc:290] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:58.732329 10338 ts_tablet_manager.cc:1434] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.002s
I20260709 13:03:58.733562 10338 tablet_bootstrap.cc:492] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:58.733788 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae810c860eb4b05a93e815248f988fc" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
W20260709 13:03:58.738515 10253 leader_election.cc:343] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.747665 10335 tablet_bootstrap.cc:654] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.721088 10333 raft_consensus.cc:385] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.750067 10333 raft_consensus.cc:740] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.700696 10553 raft_consensus.cc:2749] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:58.756219 10333 consensus_queue.cc:260] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.758450 10213 raft_consensus.cc:2468] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:03:58.759348 10515 consensus_queue.cc:1048] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:03:58.762326 10338 tablet_bootstrap.cc:654] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.763100 10333 ts_tablet_manager.cc:1434] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.002s
I20260709 13:03:58.764912 10251 leader_election.cc:304] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:58.766202 10541 raft_consensus.cc:2804] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:58.766600 10541 raft_consensus.cc:493] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:58.766897 10541 raft_consensus.cc:3060] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.779482 10541 raft_consensus.cc:515] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.783948 10331 tablet_bootstrap.cc:492] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:58.784534 10331 ts_tablet_manager.cc:1403] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.079s	user 0.020s	sys 0.019s
I20260709 13:03:58.784106 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae810c860eb4b05a93e815248f988fc" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
W20260709 13:03:58.787788 10253 leader_election.cc:343] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.793696 10331 raft_consensus.cc:359] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:58.795424 10331 raft_consensus.cc:385] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.796626 10331 raft_consensus.cc:740] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.797719 10331 consensus_queue.cc:260] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:58.802662 10333 tablet_bootstrap.cc:492] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:58.804036 10541 leader_election.cc:290] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:58.805378 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae810c860eb4b05a93e815248f988fc" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:58.807562 10213 raft_consensus.cc:3060] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.814425 10331 ts_tablet_manager.cc:1434] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.001s
I20260709 13:03:58.816138 10331 tablet_bootstrap.cc:492] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:58.821967  9961 catalog_manager.cc:5697] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:58.823563 10140 raft_consensus.cc:3060] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.823993 10331 tablet_bootstrap.cc:654] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.832180 10335 tablet_bootstrap.cc:492] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:58.832813 10335 ts_tablet_manager.cc:1403] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.131s	user 0.014s	sys 0.011s
I20260709 13:03:58.835883 10335 raft_consensus.cc:359] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:58.836624 10335 raft_consensus.cc:385] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.836897 10335 raft_consensus.cc:740] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.838353 10331 tablet_bootstrap.cc:492] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:58.838878 10331 ts_tablet_manager.cc:1403] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.004s
I20260709 13:03:58.840075 10338 tablet_bootstrap.cc:492] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:58.841099 10338 ts_tablet_manager.cc:1403] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.108s	user 0.013s	sys 0.033s
I20260709 13:03:58.839788 10335 consensus_queue.cc:260] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:58.848845 10553 raft_consensus.cc:493] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.849663 10333 tablet_bootstrap.cc:654] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.850179 10544 consensus_queue.cc:1048] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:58.849776 10553 raft_consensus.cc:515] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:58.858289 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
W20260709 13:03:58.860055 10030 leader_election.cc:343] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.864049 10213 raft_consensus.cc:2468] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:03:58.860935 10338 raft_consensus.cc:359] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.864951 10459 consensus_queue.cc:1048] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:58.865365 10338 raft_consensus.cc:385] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.865619 10251 leader_election.cc:304] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:58.865679 10338 raft_consensus.cc:740] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.867694 10544 raft_consensus.cc:2804] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:58.867517 10338 consensus_queue.cc:260] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.868371 10544 raft_consensus.cc:697] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:03:58.869674 10544 consensus_queue.cc:237] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.870329 10338 ts_tablet_manager.cc:1434] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:03:58.871603 10338 tablet_bootstrap.cc:492] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:58.874230 10553 leader_election.cc:290] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:58.848963 10331 raft_consensus.cc:359] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.876677 10331 raft_consensus.cc:385] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.877094 10331 raft_consensus.cc:740] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.877959 10331 consensus_queue.cc:260] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.876092 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
W20260709 13:03:58.880198 10028 leader_election.cc:343] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:58.858515 10335 ts_tablet_manager.cc:1434] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.025s	user 0.009s	sys 0.000s
I20260709 13:03:58.880738 10028 leader_election.cc:304] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:58.880939 10459 consensus_queue.cc:1048] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:58.877985 10338 tablet_bootstrap.cc:654] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.881990 10335 tablet_bootstrap.cc:492] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:58.884377 10553 raft_consensus.cc:2749] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:58.889747 10331 ts_tablet_manager.cc:1434] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.002s
I20260709 13:03:58.891091 10331 tablet_bootstrap.cc:492] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:58.891590 10411 raft_consensus.cc:493] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.892258 10411 raft_consensus.cc:515] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:58.897773 10335 tablet_bootstrap.cc:654] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:58.914978  9961 catalog_manager.cc:5697] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } }
I20260709 13:03:58.920989 10411 leader_election.cc:290] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:58.922068 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a68082d0468d49ada47812f6c8e3f106" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:58.923600 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a68082d0468d49ada47812f6c8e3f106" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:58.928572 10104 leader_election.cc:304] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:58.935216 10515 raft_consensus.cc:2749] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:58.941080 10338 tablet_bootstrap.cc:492] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:58.941802 10338 ts_tablet_manager.cc:1403] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.070s	user 0.022s	sys 0.014s
I20260709 13:03:58.950376 10288 raft_consensus.cc:3060] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.951216 10544 consensus_queue.cc:1048] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:58.959798 10338 raft_consensus.cc:359] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.960469 10335 tablet_bootstrap.cc:492] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:58.960659 10338 raft_consensus.cc:385] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.961107 10338 raft_consensus.cc:740] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.961199 10335 ts_tablet_manager.cc:1403] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.079s	user 0.012s	sys 0.003s
I20260709 13:03:58.962507 10338 consensus_queue.cc:260] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.964480 10335 raft_consensus.cc:359] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.965322 10335 raft_consensus.cc:385] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.966184 10335 raft_consensus.cc:740] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.967221 10139 raft_consensus.cc:3060] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.970521 10333 tablet_bootstrap.cc:492] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:58.971131 10333 ts_tablet_manager.cc:1403] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.207s	user 0.026s	sys 0.010s
I20260709 13:03:58.975584 10335 consensus_queue.cc:260] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.978037 10338 ts_tablet_manager.cc:1434] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.036s	user 0.002s	sys 0.006s
I20260709 13:03:58.980103 10537 consensus_queue.cc:1048] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:58.983829 10138 raft_consensus.cc:3060] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:58.987444 10338 tablet_bootstrap.cc:492] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:58.990355 10335 ts_tablet_manager.cc:1434] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:03:58.991803 10335 tablet_bootstrap.cc:492] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:58.993249 10333 raft_consensus.cc:359] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.994516 10333 raft_consensus.cc:385] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:58.994832 10333 raft_consensus.cc:740] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:58.995545 10333 consensus_queue.cc:260] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:58.996441 10588 raft_consensus.cc:493] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:58.997038 10588 raft_consensus.cc:515] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:58.998054 10333 ts_tablet_manager.cc:1434] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.002s
I20260709 13:03:58.999874 10588 leader_election.cc:290] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:59.001151 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd01deadf87945439c630e949d9c2e7b" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:59.001886 10062 raft_consensus.cc:2468] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:03:59.003480 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd01deadf87945439c630e949d9c2e7b" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:59.004199 10213 raft_consensus.cc:2468] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:03:59.005962 10252 leader_election.cc:304] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 
I20260709 13:03:58.999085 10331 tablet_bootstrap.cc:654] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.008981 10333 tablet_bootstrap.cc:492] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:59.027678 10515 raft_consensus.cc:493] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.030007 10515 raft_consensus.cc:515] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.023141 10411 raft_consensus.cc:493] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.034312 10411 raft_consensus.cc:515] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:59.034951 10553 raft_consensus.cc:493] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.035511 10553 raft_consensus.cc:515] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.038231 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae810c860eb4b05a93e815248f988fc" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:59.039075 10213 raft_consensus.cc:2393] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bfa051e18d0549eabe79dfcc7610d99b in current term 1: Already voted for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in this term.
I20260709 13:03:59.040495 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940beae71dca43e58d3879c233f87dbb" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:59.044886 10335 tablet_bootstrap.cc:654] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.047216 10411 leader_election.cc:290] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:59.047233 10392 consensus_queue.cc:1048] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:59.050374 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be75bbbdef504a5793ba0b07feaff6f8" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:59.050861 10338 tablet_bootstrap.cc:654] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.023486 10541 raft_consensus.cc:2804] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:59.053599 10541 raft_consensus.cc:493] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.054155 10541 raft_consensus.cc:3060] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:03:59.051939 10030 leader_election.cc:343] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:59.052491 10553 leader_election.cc:290] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:59.056906 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aae810c860eb4b05a93e815248f988fc" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:59.053113 10515 leader_election.cc:290] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:59.067058 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940beae71dca43e58d3879c233f87dbb" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:59.061640 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be75bbbdef504a5793ba0b07feaff6f8" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:59.071101 10392 raft_consensus.cc:493] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:03:59.075698 10030 leader_election.cc:343] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:59.076454 10030 leader_election.cc:304] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:59.071679 10392 raft_consensus.cc:515] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.074550 10104 leader_election.cc:304] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:03:59.080302 10392 leader_election.cc:290] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:59.081241 10430 raft_consensus.cc:2749] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:59.086445 10105 leader_election.cc:304] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:03:59.091527 10515 raft_consensus.cc:3060] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.100364 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4f4fa6d20f4f1ca0b1befcc3a605b5" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:59.143432 10515 raft_consensus.cc:2749] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:59.112573 10588 raft_consensus.cc:493] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.167964 10588 raft_consensus.cc:515] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.171182 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84c07107a48406d9fd585e9982d3069" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:59.167339 10333 tablet_bootstrap.cc:654] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.171777 10459 consensus_queue.cc:1048] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:03:59.112928 10062 raft_consensus.cc:2468] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:59.148910 10541 raft_consensus.cc:515] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.185233 10178 leader_election.cc:304] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:59.189096 10459 raft_consensus.cc:2804] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:59.189592 10459 raft_consensus.cc:493] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.189963 10459 raft_consensus.cc:3060] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.194346 10541 leader_election.cc:290] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:59.195470 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd01deadf87945439c630e949d9c2e7b" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:59.196363 10213 raft_consensus.cc:3060] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.202220 10411 raft_consensus.cc:2749] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:03:59.214085 10588 leader_election.cc:290] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:59.212918 10459 raft_consensus.cc:515] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.230151 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4f4fa6d20f4f1ca0b1befcc3a605b5" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:59.230932 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4f4fa6d20f4f1ca0b1befcc3a605b5" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:59.231268 10331 tablet_bootstrap.cc:492] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:59.231629 10064 raft_consensus.cc:3060] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.231854 10331 ts_tablet_manager.cc:1403] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.341s	user 0.024s	sys 0.011s
I20260709 13:03:59.234508 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4f4fa6d20f4f1ca0b1befcc3a605b5" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:59.234807 10331 raft_consensus.cc:359] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.235531 10331 raft_consensus.cc:385] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.235617 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd01deadf87945439c630e949d9c2e7b" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:59.235881 10331 raft_consensus.cc:740] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
W20260709 13:03:59.235770 10179 leader_election.cc:343] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:59.236371 10063 raft_consensus.cc:3060] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.235342 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84c07107a48406d9fd585e9982d3069" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:59.237397 10537 consensus_queue.cc:1048] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:59.236829 10331 consensus_queue.cc:260] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.244335 10331 ts_tablet_manager.cc:1434] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:03:59.245219 10331 tablet_bootstrap.cc:492] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
W20260709 13:03:59.249361 10179 leader_election.cc:343] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:59.250618 10338 tablet_bootstrap.cc:492] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:59.251173 10338 ts_tablet_manager.cc:1403] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.264s	user 0.020s	sys 0.019s
I20260709 13:03:59.253041 10252 leader_election.cc:304] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:03:59.255023 10063 raft_consensus.cc:2468] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:03:59.255257 10335 tablet_bootstrap.cc:492] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:59.259289 10335 ts_tablet_manager.cc:1403] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.268s	user 0.026s	sys 0.016s
I20260709 13:03:59.266525 10335 raft_consensus.cc:359] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.267349 10335 raft_consensus.cc:385] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.267620 10335 raft_consensus.cc:740] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.268307 10213 raft_consensus.cc:2468] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:03:59.268429 10335 consensus_queue.cc:260] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.266896 10588 raft_consensus.cc:2749] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:59.269914 10251 leader_election.cc:304] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:59.273362 10335 ts_tablet_manager.cc:1434] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:03:59.274542 10335 tablet_bootstrap.cc:492] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:59.278663 10537 raft_consensus.cc:493] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.279243 10537 raft_consensus.cc:515] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:59.280622 10335 tablet_bootstrap.cc:654] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.260834 10459 leader_election.cc:290] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:59.278012 10541 raft_consensus.cc:2804] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:59.288463 10537 leader_election.cc:290] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:59.289820 10541 raft_consensus.cc:697] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:03:59.290796 10541 consensus_queue.cc:237] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.291904 10338 raft_consensus.cc:359] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.292835 10338 raft_consensus.cc:385] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.293236 10338 raft_consensus.cc:740] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.293963 10338 consensus_queue.cc:260] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.296847 10338 ts_tablet_manager.cc:1434] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.045s	user 0.005s	sys 0.003s
I20260709 13:03:59.298040 10338 tablet_bootstrap.cc:492] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:59.304087 10338 tablet_bootstrap.cc:654] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.282270 10064 raft_consensus.cc:2468] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:59.330765 10178 leader_election.cc:304] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
W20260709 13:03:59.329938 10011 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:03:59.332940 10459 raft_consensus.cc:2804] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:59.332396 10594 consensus_queue.cc:1048] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:03:59.350025 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af4aeb2cefe7417e9c4364dd33b00847" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:59.353051 10333 tablet_bootstrap.cc:492] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:59.353639 10333 ts_tablet_manager.cc:1403] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.345s	user 0.014s	sys 0.009s
I20260709 13:03:59.356061 10333 raft_consensus.cc:359] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.356753 10333 raft_consensus.cc:385] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.357012 10333 raft_consensus.cc:740] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.359068 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af4aeb2cefe7417e9c4364dd33b00847" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:59.343880 10459 raft_consensus.cc:697] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:03:59.361696 10459 consensus_queue.cc:237] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
W20260709 13:03:59.363456 10179 leader_election.cc:343] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:03:59.365264 10105 consensus_peers.cc:597] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b -> Peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Couldn't send request to peer 8c3d3c9022c94705ac4ee834e1dc49d0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:03:59.359254  9963 catalog_manager.cc:5697] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:59.367856 10514 consensus_queue.cc:1048] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:59.371006 10333 consensus_queue.cc:260] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.374740 10338 tablet_bootstrap.cc:492] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:59.375432 10338 ts_tablet_manager.cc:1403] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.078s	user 0.017s	sys 0.012s
I20260709 13:03:59.377916 10331 tablet_bootstrap.cc:654] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.383764 10335 tablet_bootstrap.cc:492] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:59.384308 10335 ts_tablet_manager.cc:1403] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.110s	user 0.008s	sys 0.024s
W20260709 13:03:59.386646 10179 leader_election.cc:343] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:59.387140 10179 leader_election.cc:304] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:59.389626 10288 raft_consensus.cc:3060] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.399273 10392 raft_consensus.cc:2749] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:03:59.408268 10333 ts_tablet_manager.cc:1434] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.000s
I20260709 13:03:59.415431  9963 catalog_manager.cc:5697] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:59.434149 10594 consensus_queue.cc:1048] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:59.459084 10335 raft_consensus.cc:359] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:59.460100 10335 raft_consensus.cc:385] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.462114 10335 raft_consensus.cc:740] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.463259 10335 consensus_queue.cc:260] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:59.471261 10335 ts_tablet_manager.cc:1434] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.087s	user 0.003s	sys 0.005s
I20260709 13:03:59.472679 10335 tablet_bootstrap.cc:492] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:59.479311 10335 tablet_bootstrap.cc:654] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.471227 10338 raft_consensus.cc:359] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:59.498423 10338 raft_consensus.cc:385] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.521327 10338 raft_consensus.cc:740] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.527200 10333 tablet_bootstrap.cc:492] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:59.529686 10338 consensus_queue.cc:260] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:59.531993 10338 ts_tablet_manager.cc:1434] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.156s	user 0.006s	sys 0.002s
I20260709 13:03:59.533139 10338 tablet_bootstrap.cc:492] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:59.539319 10338 tablet_bootstrap.cc:654] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.539527 10335 tablet_bootstrap.cc:492] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:59.540098 10335 ts_tablet_manager.cc:1403] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.068s	user 0.011s	sys 0.000s
I20260709 13:03:59.542441 10331 tablet_bootstrap.cc:492] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:59.542958 10331 ts_tablet_manager.cc:1403] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.298s	user 0.042s	sys 0.051s
I20260709 13:03:59.543146 10335 raft_consensus.cc:359] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.543951 10335 raft_consensus.cc:385] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.544227 10335 raft_consensus.cc:740] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.545063 10335 consensus_queue.cc:260] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.547291 10335 ts_tablet_manager.cc:1434] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:03:59.548336 10335 tablet_bootstrap.cc:492] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:59.548545 10331 raft_consensus.cc:359] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.549280 10331 raft_consensus.cc:385] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.549567 10331 raft_consensus.cc:740] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.550377 10331 consensus_queue.cc:260] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.552637 10331 ts_tablet_manager.cc:1434] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 13:03:59.553120 10333 tablet_bootstrap.cc:654] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.555208 10335 tablet_bootstrap.cc:654] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.555899 10331 tablet_bootstrap.cc:492] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:59.562106 10411 raft_consensus.cc:493] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.562897 10411 raft_consensus.cc:515] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:59.565599 10331 tablet_bootstrap.cc:654] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.566413 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af4aeb2cefe7417e9c4364dd33b00847" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:59.567375 10213 raft_consensus.cc:2468] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:03:59.569751 10103 leader_election.cc:304] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:59.570730 10514 raft_consensus.cc:2804] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:59.571087 10411 leader_election.cc:290] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:59.571269 10514 raft_consensus.cc:493] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.571857 10514 raft_consensus.cc:3060] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.573076 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af4aeb2cefe7417e9c4364dd33b00847" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:03:59.575096 10105 leader_election.cc:343] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:59.576303 10331 tablet_bootstrap.cc:492] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:59.576779 10331 ts_tablet_manager.cc:1403] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.005s
I20260709 13:03:59.579489 10338 tablet_bootstrap.cc:492] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:59.580010 10338 ts_tablet_manager.cc:1403] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.047s	user 0.015s	sys 0.011s
I20260709 13:03:59.582998 10338 raft_consensus.cc:359] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.583707 10338 raft_consensus.cc:385] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.583957 10338 raft_consensus.cc:740] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.584606 10338 consensus_queue.cc:260] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.588413 10331 raft_consensus.cc:359] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.589186 10331 raft_consensus.cc:385] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.589502 10331 raft_consensus.cc:740] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.590216 10331 consensus_queue.cc:260] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.592214 10331 ts_tablet_manager.cc:1434] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.015s	user 0.000s	sys 0.006s
I20260709 13:03:59.595099 10338 ts_tablet_manager.cc:1434] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:03:59.596382 10338 tablet_bootstrap.cc:492] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:59.602375 10331 tablet_bootstrap.cc:492] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:59.604900 10338 tablet_bootstrap.cc:654] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.605834 10514 raft_consensus.cc:515] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:59.610023 10335 tablet_bootstrap.cc:492] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:03:59.610402 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af4aeb2cefe7417e9c4364dd33b00847" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:59.610657 10335 ts_tablet_manager.cc:1403] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.063s	user 0.029s	sys 0.004s
I20260709 13:03:59.611060 10213 raft_consensus.cc:3060] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.611436 10514 leader_election.cc:290] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:59.629797 10397 raft_consensus.cc:493] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.630649 10397 raft_consensus.cc:515] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.636130 10397 leader_election.cc:290] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:59.639505 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:59.641844 10064 raft_consensus.cc:2468] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:59.649299 10553 raft_consensus.cc:493] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.649823 10333 tablet_bootstrap.cc:492] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:59.650367 10333 ts_tablet_manager.cc:1403] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.225s	user 0.017s	sys 0.016s
I20260709 13:03:59.649873 10553 raft_consensus.cc:515] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.654712 10335 raft_consensus.cc:359] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.656682 10335 raft_consensus.cc:385] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.656819 10331 tablet_bootstrap.cc:654] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.657027 10335 raft_consensus.cc:740] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.648629 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af4aeb2cefe7417e9c4364dd33b00847" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:59.658067 10335 consensus_queue.cc:260] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.659444 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:59.660913 10213 raft_consensus.cc:2468] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
W20260709 13:03:59.661402 10105 leader_election.cc:343] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:59.658088 10553 leader_election.cc:290] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:59.668942 10331 tablet_bootstrap.cc:492] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:03:59.669945 10178 leader_election.cc:304] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:03:59.675377 10331 ts_tablet_manager.cc:1403] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.073s	user 0.021s	sys 0.032s
I20260709 13:03:59.676051 10392 raft_consensus.cc:2804] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:59.676472 10392 raft_consensus.cc:493] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.676679 10514 raft_consensus.cc:493] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.676864 10392 raft_consensus.cc:3060] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.670694 10333 raft_consensus.cc:359] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.677230 10514 raft_consensus.cc:515] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.680161 10514 leader_election.cc:290] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:59.677713 10333 raft_consensus.cc:385] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.674005 10103 leader_election.cc:304] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:59.680925 10333 raft_consensus.cc:740] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.681354 10553 raft_consensus.cc:493] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.681828 10333 consensus_queue.cc:260] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.682057 10553 raft_consensus.cc:515] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.683435 10515 raft_consensus.cc:2804] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:59.684051 10333 ts_tablet_manager.cc:1434] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.033s	user 0.009s	sys 0.000s
I20260709 13:03:59.684228 10515 raft_consensus.cc:697] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:03:59.685187 10333 tablet_bootstrap.cc:492] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:59.686833 10553 leader_election.cc:290] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:59.688542 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f9858e8dc4fd488d562cac2d54aa4" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:59.691027 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f9858e8dc4fd488d562cac2d54aa4" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
W20260709 13:03:59.691385 10030 leader_election.cc:343] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:59.691797 10139 raft_consensus.cc:2468] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:03:59.693895 10515 consensus_queue.cc:237] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:03:59.698021 10030 leader_election.cc:304] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:59.699025 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:59.700999 10553 raft_consensus.cc:2804] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:59.701493 10553 raft_consensus.cc:493] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.701820 10553 raft_consensus.cc:3060] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.669978 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:59.707006 10288 raft_consensus.cc:2468] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:03:59.680496 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:59.708222 10064 raft_consensus.cc:2468] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:03:59.710067 10104 leader_election.cc:304] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:59.710547 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:59.711429 10139 raft_consensus.cc:2468] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:03:59.706542 10335 ts_tablet_manager.cc:1434] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.095s	user 0.014s	sys 0.019s
I20260709 13:03:59.712234 10411 raft_consensus.cc:2804] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:59.712656 10411 raft_consensus.cc:493] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.713014 10411 raft_consensus.cc:3060] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.708236 10030 leader_election.cc:304] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:03:59.716331 10430 raft_consensus.cc:2804] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:03:59.716746 10430 raft_consensus.cc:493] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.717073 10430 raft_consensus.cc:3060] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:59.721702 10335 tablet_bootstrap.cc:492] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:03:59.680154 10331 raft_consensus.cc:359] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.727228 10430 raft_consensus.cc:515] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.728852 10283 raft_consensus.cc:3060] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.735069 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:59.735579 10333 tablet_bootstrap.cc:654] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.727726 10331 raft_consensus.cc:385] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.739064 10331 raft_consensus.cc:740] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.747572 10331 consensus_queue.cc:260] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.729334 10541 raft_consensus.cc:493] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.752498 10541 raft_consensus.cc:515] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.754734 10541 leader_election.cc:290] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:59.755533 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc9831f2c60444db097f11766636c56" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:59.696864 10392 raft_consensus.cc:515] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.758232 10288 raft_consensus.cc:3060] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.758939 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc9831f2c60444db097f11766636c56" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:59.762992 10064 raft_consensus.cc:2468] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:03:59.727413 10411 raft_consensus.cc:515] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.769166 10621 raft_consensus.cc:493] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.772217 10211 raft_consensus.cc:2393] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bfa051e18d0549eabe79dfcc7610d99b in current term 1: Already voted for candidate bcf45a2176234db5957dc38e32554577 in this term.
I20260709 13:03:59.773692 10392 leader_election.cc:290] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
W20260709 13:03:59.767015 10251 leader_election.cc:343] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:59.781368 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:59.782019 10285 raft_consensus.cc:3060] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:03:59.782240 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:59.782829 10064 raft_consensus.cc:3060] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.783910 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:59.784751 10137 raft_consensus.cc:2393] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bcf45a2176234db5957dc38e32554577 in current term 1: Already voted for candidate bfa051e18d0549eabe79dfcc7610d99b in this term.
I20260709 13:03:59.785946 10411 leader_election.cc:290] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:59.787101 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:03:59.788554 10331 ts_tablet_manager.cc:1434] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.113s	user 0.011s	sys 0.004s
I20260709 13:03:59.790804 10064 raft_consensus.cc:2468] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:03:59.776813 10430 leader_election.cc:290] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:03:59.792048 10178 leader_election.cc:304] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:59.787945 10211 raft_consensus.cc:2393] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bfa051e18d0549eabe79dfcc7610d99b in current term 1: Already voted for candidate bcf45a2176234db5957dc38e32554577 in this term.
I20260709 13:03:59.794633 10252 leader_election.cc:304] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:03:59.796582 10537 raft_consensus.cc:2804] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:59.797132 10537 raft_consensus.cc:697] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:03:59.798033 10544 raft_consensus.cc:2804] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:59.798338 10537 consensus_queue.cc:237] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.798629  9963 catalog_manager.cc:5697] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } }
I20260709 13:03:59.799392 10514 raft_consensus.cc:493] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.800930 10514 raft_consensus.cc:515] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.802259 10594 consensus_queue.cc:1048] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:03:59.798527 10544 raft_consensus.cc:493] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.806351 10544 raft_consensus.cc:3060] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.817756 10335 tablet_bootstrap.cc:654] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.788153 10621 raft_consensus.cc:515] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:03:59.826678 10338 tablet_bootstrap.cc:492] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:03:59.828209 10331 tablet_bootstrap.cc:492] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:03:59.828483 10338 ts_tablet_manager.cc:1403] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.232s	user 0.026s	sys 0.029s
I20260709 13:03:59.830052 10459 raft_consensus.cc:493] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:03:59.830682 10459 raft_consensus.cc:515] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.833599 10459 leader_election.cc:290] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:59.832602 10338 raft_consensus.cc:359] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.834066 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:03:59.834626 10338 raft_consensus.cc:385] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.835000 10338 raft_consensus.cc:740] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.835630 10333 tablet_bootstrap.cc:492] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:03:59.836202 10333 ts_tablet_manager.cc:1403] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.151s	user 0.009s	sys 0.005s
I20260709 13:03:59.835866 10338 consensus_queue.cc:260] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.836751 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:59.837671 10064 raft_consensus.cc:2368] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bcf45a2176234db5957dc38e32554577 for earlier term 2. Current term is 3.
I20260709 13:03:59.838235 10338 ts_tablet_manager.cc:1434] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:03:59.839124 10514 leader_election.cc:290] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:03:59.839483 10338 tablet_bootstrap.cc:492] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:03:59.839116 10178 leader_election.cc:400] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611) with higher term. Message: Invalid argument: T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bcf45a2176234db5957dc38e32554577 for earlier term 2. Current term is 3.
I20260709 13:03:59.840022 10178 leader_election.cc:403] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:03:59.841060 10459 raft_consensus.cc:3060] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:03:59.842659 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940beae71dca43e58d3879c233f87dbb" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:03:59.847211 10105 leader_election.cc:343] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:59.851047 10338 tablet_bootstrap.cc:654] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.851878 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:03:59.853399 10064 raft_consensus.cc:2393] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bfa051e18d0549eabe79dfcc7610d99b in current term 1: Already voted for candidate bcf45a2176234db5957dc38e32554577 in this term.
I20260709 13:03:59.859732 10285 raft_consensus.cc:2468] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 3.
I20260709 13:03:59.861567 10030 leader_election.cc:304] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:03:59.867293 10459 raft_consensus.cc:2749] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611) with higher term. Message: Invalid argument: T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bcf45a2176234db5957dc38e32554577 for earlier term 2. Current term is 3.
I20260709 13:03:59.868806 10459 consensus_queue.cc:1048] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:59.869344 10620 raft_consensus.cc:2804] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:03:59.870083 10620 raft_consensus.cc:697] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:03:59.870814 10333 raft_consensus.cc:359] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.871572 10333 raft_consensus.cc:385] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:03:59.871825 10333 raft_consensus.cc:740] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:03:59.871110 10620 consensus_queue.cc:237] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.872529 10333 consensus_queue.cc:260] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:03:59.874125 10553 raft_consensus.cc:515] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:03:59.876348 10553 leader_election.cc:290] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:03:59.879382 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f9858e8dc4fd488d562cac2d54aa4" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:03:59.879729 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f9858e8dc4fd488d562cac2d54aa4" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:03:59.880101 10283 raft_consensus.cc:3060] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.880436 10139 raft_consensus.cc:3060] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.888890 10397 consensus_queue.cc:1048] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:03:59.891574 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0cced7e11946f09a13c6adcf95833b" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:03:59.881026 10544 raft_consensus.cc:515] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
W20260709 13:03:59.894750 10030 leader_election.cc:343] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:03:59.894922 10621 leader_election.cc:290] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:03:59.895704 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940beae71dca43e58d3879c233f87dbb" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:03:59.896131 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0cced7e11946f09a13c6adcf95833b" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:03:59.896390 10594 consensus_queue.cc:1048] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:03:59.898563 10104 leader_election.cc:304] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:03:59.903375 10333 ts_tablet_manager.cc:1434] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 13:03:59.893229 10139 raft_consensus.cc:2468] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:03:59.905323 10214 raft_consensus.cc:2468] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:03:59.899080 10624 raft_consensus.cc:493] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:03:59.907563 10062 raft_consensus.cc:2468] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:03:59.907049 10514 raft_consensus.cc:2749] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:03:59.918197 10028 leader_election.cc:304] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:03:59.919423 10621 raft_consensus.cc:2804] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:59.919835 10621 raft_consensus.cc:493] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.920192 10621 raft_consensus.cc:3060] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.920404 10331 tablet_bootstrap.cc:654] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:03:59.931455 10030 leader_election.cc:304] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:03:59.938858 10333 tablet_bootstrap.cc:492] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:03:59.935964 10620 raft_consensus.cc:2804] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:03:59.935989 10104 leader_election.cc:304] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:03:59.947558 10515 raft_consensus.cc:2804] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:03:59.948173 10515 raft_consensus.cc:493] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:03:59.948657 10515 raft_consensus.cc:3060] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:03:59.917297 10624 raft_consensus.cc:515] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.003674  9963 catalog_manager.cc:5697] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:00.004779  9960 catalog_manager.cc:5697] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:00.009019 10624 leader_election.cc:290] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:00.008381 10335 tablet_bootstrap.cc:492] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:00.009968 10335 ts_tablet_manager.cc:1403] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.297s	user 0.015s	sys 0.010s
I20260709 13:04:00.010231 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c5f39847af4013aa501ce097f58b87" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:04:00.038887 10030 leader_election.cc:343] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.012653 10335 raft_consensus.cc:359] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.053716 10335 raft_consensus.cc:385] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.077507 10335 raft_consensus.cc:740] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.080021 10333 tablet_bootstrap.cc:654] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.086047 10544 leader_election.cc:290] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:00.086541 10335 consensus_queue.cc:260] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.088070 10515 raft_consensus.cc:515] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.093885 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc9831f2c60444db097f11766636c56" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:04:00.106045 10063 raft_consensus.cc:3060] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.038399 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c5f39847af4013aa501ce097f58b87" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:00.028282 10620 raft_consensus.cc:697] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:00.051962 10283 raft_consensus.cc:2468] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:04:00.053536 10621 raft_consensus.cc:515] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.099393 10335 ts_tablet_manager.cc:1434] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.089s	user 0.013s	sys 0.005s
I20260709 13:04:00.100345 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc9831f2c60444db097f11766636c56" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:00.118211 10335 tablet_bootstrap.cc:492] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
W20260709 13:04:00.107889 10103 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 112577us
I20260709 13:04:00.133028 10515 leader_election.cc:290] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:00.134215 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940beae71dca43e58d3879c233f87dbb" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:04:00.134862 10064 raft_consensus.cc:3060] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:00.119444 10251 leader_election.cc:343] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.110175 10620 consensus_queue.cc:237] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.139608 10063 raft_consensus.cc:2468] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:04:00.123175 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0cced7e11946f09a13c6adcf95833b" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:00.143021 10335 tablet_bootstrap.cc:654] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.148059 10211 raft_consensus.cc:3060] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.150913 10335 tablet_bootstrap.cc:492] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:00.151449 10335 ts_tablet_manager.cc:1403] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.000s
I20260709 13:04:00.126233 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0cced7e11946f09a13c6adcf95833b" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:00.152009 10137 raft_consensus.cc:3060] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.121222 10621 leader_election.cc:290] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:00.158725 10137 raft_consensus.cc:2468] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:04:00.159044 10064 raft_consensus.cc:2468] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:04:00.131201 10338 tablet_bootstrap.cc:492] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:00.162487 10338 ts_tablet_manager.cc:1403] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.323s	user 0.013s	sys 0.008s
I20260709 13:04:00.162364 10104 leader_election.cc:304] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:00.163671 10618 raft_consensus.cc:2804] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:04:00.127105 10030 leader_election.cc:343] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.164350 10618 raft_consensus.cc:697] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:00.164194 10335 raft_consensus.cc:359] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.151566 10331 tablet_bootstrap.cc:492] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:04:00.166152 10335 raft_consensus.cc:385] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.166531 10331 ts_tablet_manager.cc:1403] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.348s	user 0.009s	sys 0.013s
I20260709 13:04:00.166252 10618 consensus_queue.cc:237] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.166548 10335 raft_consensus.cc:740] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.168283 10335 consensus_queue.cc:260] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.170964 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940beae71dca43e58d3879c233f87dbb" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:00.172829 10335 ts_tablet_manager.cc:1434] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
W20260709 13:04:00.175832 10105 leader_election.cc:343] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.178421 10335 tablet_bootstrap.cc:492] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:04:00.169406 10331 raft_consensus.cc:359] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.181161 10331 raft_consensus.cc:385] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.181469 10331 raft_consensus.cc:740] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.182880 10333 tablet_bootstrap.cc:492] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:00.164407 10030 leader_election.cc:304] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:00.183410 10331 consensus_queue.cc:260] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
W20260709 13:04:00.183866 10030 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:33711, user_credentials={real_user=slave}} blocked reactor thread for 56791.1us
I20260709 13:04:00.181583 10626 raft_consensus.cc:493] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.184777 10252 leader_election.cc:304] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:04:00.185989 10544 raft_consensus.cc:2804] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:00.186569 10544 raft_consensus.cc:697] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:00.186859 10333 ts_tablet_manager.cc:1403] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.255s	user 0.014s	sys 0.020s
I20260709 13:04:00.187445 10544 consensus_queue.cc:237] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:00.190534 10333 raft_consensus.cc:359] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.184754 10626 raft_consensus.cc:515] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:00.191449 10333 raft_consensus.cc:385] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.191771 10333 raft_consensus.cc:740] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.192518 10333 consensus_queue.cc:260] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.194188 10211 raft_consensus.cc:2468] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:04:00.197964 10028 leader_election.cc:304] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:00.199247 10626 leader_election.cc:290] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:00.200452 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1aaf32c658c483eb6a4a3858745515f" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:04:00.202370 10179 leader_election.cc:343] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.186569 10338 raft_consensus.cc:359] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.203045 10338 raft_consensus.cc:385] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.203390 10338 raft_consensus.cc:740] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.203626 10335 tablet_bootstrap.cc:654] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.204149 10338 consensus_queue.cc:260] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.206102 10338 ts_tablet_manager.cc:1434] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 13:04:00.207283 10338 tablet_bootstrap.cc:492] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:00.208026 10333 ts_tablet_manager.cc:1434] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.002s
I20260709 13:04:00.213929 10624 raft_consensus.cc:2749] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:00.214844 10338 tablet_bootstrap.cc:654] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.218984 10620 raft_consensus.cc:2804] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:00.219766 10620 raft_consensus.cc:697] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:00.220948 10620 consensus_queue.cc:237] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.227008 10335 tablet_bootstrap.cc:492] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:00.227607 10335 ts_tablet_manager.cc:1403] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.049s	user 0.008s	sys 0.013s
I20260709 13:04:00.228947  9961 catalog_manager.cc:5697] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:00.230542 10335 raft_consensus.cc:359] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.231288 10335 raft_consensus.cc:385] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.231555 10335 raft_consensus.cc:740] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.232499 10537 consensus_queue.cc:1048] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:00.235409 10338 tablet_bootstrap.cc:492] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:00.235924 10338 ts_tablet_manager.cc:1403] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.003s
I20260709 13:04:00.235178 10537 consensus_queue.cc:1048] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:00.238066 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1aaf32c658c483eb6a4a3858745515f" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:00.232182 10335 consensus_queue.cc:260] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.241712 10335 ts_tablet_manager.cc:1434] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.001s
W20260709 13:04:00.242079 10178 leader_election.cc:343] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.242827 10335 tablet_bootstrap.cc:492] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:04:00.242609 10178 leader_election.cc:304] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:00.244544 10459 raft_consensus.cc:2749] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:04:00.249119 10105 consensus_peers.cc:597] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b -> Peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Couldn't send request to peer 8c3d3c9022c94705ac4ee834e1dc49d0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:00.250710 10459 raft_consensus.cc:493] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.251375 10459 raft_consensus.cc:515] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.253640 10459 leader_election.cc:290] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:00.244719  9960 catalog_manager.cc:5697] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:00.249969 10335 tablet_bootstrap.cc:654] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.255832 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d3e10044ce408294e1b86ce960b84a" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:00.259332 10137 raft_consensus.cc:2468] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:04:00.260330 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d3e10044ce408294e1b86ce960b84a" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:04:00.265071 10179 leader_election.cc:343] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:00.258569 10643 log.cc:927] Time spent T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:04:00.266170 10179 leader_election.cc:304] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:00.249295 10338 raft_consensus.cc:359] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.267413 10338 raft_consensus.cc:385] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.267745 10338 raft_consensus.cc:740] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.268492 10338 consensus_queue.cc:260] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.260259 10331 ts_tablet_manager.cc:1434] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.000s
I20260709 13:04:00.271703 10331 tablet_bootstrap.cc:492] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:00.275214 10338 ts_tablet_manager.cc:1434] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 13:04:00.276331 10338 tablet_bootstrap.cc:492] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:00.282930 10626 raft_consensus.cc:2804] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:00.284826  9962 catalog_manager.cc:5697] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:00.289178 10626 raft_consensus.cc:493] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.290076 10626 raft_consensus.cc:3060] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.285014 10333 tablet_bootstrap.cc:492] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:04:00.310508 10515 consensus_queue.cc:1048] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:00.325384  9962 catalog_manager.cc:5697] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:00.349678 10335 tablet_bootstrap.cc:492] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:00.350253 10335 ts_tablet_manager.cc:1403] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.108s	user 0.026s	sys 0.049s
I20260709 13:04:00.353888 10331 tablet_bootstrap.cc:654] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.358846 10338 tablet_bootstrap.cc:654] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.360194 10335 raft_consensus.cc:359] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.361037 10335 raft_consensus.cc:385] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.361349 10335 raft_consensus.cc:740] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.329851 10333 tablet_bootstrap.cc:654] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.391739 10335 consensus_queue.cc:260] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.393764 10335 ts_tablet_manager.cc:1434] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.001s
I20260709 13:04:00.394732 10335 tablet_bootstrap.cc:492] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:04:00.400772 10335 tablet_bootstrap.cc:654] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.306461 10626 raft_consensus.cc:515] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.431836 10335 tablet_bootstrap.cc:492] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:00.432336 10335 ts_tablet_manager.cc:1403] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.038s	user 0.005s	sys 0.019s
I20260709 13:04:00.435194 10213 raft_consensus.cc:1275] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:00.437789 10620 consensus_queue.cc:1048] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:00.445377 10635 raft_consensus.cc:493] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.445964 10635 raft_consensus.cc:515] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.451260 10635 leader_election.cc:290] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:00.455024 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13fc2f8cffc54c41b5bdde0305fe868c" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
W20260709 13:04:00.456553 10104 leader_election.cc:343] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.456971 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13fc2f8cffc54c41b5bdde0305fe868c" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:00.460096 10626 leader_election.cc:290] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
W20260709 13:04:00.460981 10103 leader_election.cc:343] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.461570 10103 leader_election.cc:304] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:00.461606 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d3e10044ce408294e1b86ce960b84a" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:00.460273 10338 tablet_bootstrap.cc:492] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:00.462563 10635 raft_consensus.cc:2749] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:00.463259 10338 ts_tablet_manager.cc:1403] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.187s	user 0.018s	sys 0.026s
I20260709 13:04:00.462442 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d3e10044ce408294e1b86ce960b84a" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
W20260709 13:04:00.467842 10179 leader_election.cc:343] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.467700 10139 raft_consensus.cc:3060] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.471024 10338 raft_consensus.cc:359] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:00.471511 10285 raft_consensus.cc:1275] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:00.471774 10338 raft_consensus.cc:385] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.472088 10338 raft_consensus.cc:740] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.472862 10338 consensus_queue.cc:260] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:00.474154 10331 tablet_bootstrap.cc:492] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:04:00.474227 10620 consensus_queue.cc:1048] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:00.474748 10331 ts_tablet_manager.cc:1403] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.203s	user 0.020s	sys 0.038s
I20260709 13:04:00.480957 10338 ts_tablet_manager.cc:1434] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.001s
I20260709 13:04:00.483927 10333 tablet_bootstrap.cc:492] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:00.484849 10333 ts_tablet_manager.cc:1403] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.200s	user 0.007s	sys 0.007s
I20260709 13:04:00.486711 10335 raft_consensus.cc:359] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.487520 10335 raft_consensus.cc:385] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.487773 10335 raft_consensus.cc:740] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.482092 10338 tablet_bootstrap.cc:492] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:00.490404 10139 raft_consensus.cc:2468] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:04:00.492127 10179 leader_election.cc:304] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:00.492538 10331 raft_consensus.cc:359] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.493340 10331 raft_consensus.cc:385] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.493680 10331 raft_consensus.cc:740] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.493402 10626 raft_consensus.cc:2804] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:00.494314 10626 raft_consensus.cc:697] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:04:00.495235 10626 consensus_queue.cc:237] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.497979 10331 consensus_queue.cc:260] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.499097 10335 consensus_queue.cc:260] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.501210 10331 ts_tablet_manager.cc:1434] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 13:04:00.490471 10333 raft_consensus.cc:359] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.502080 10333 raft_consensus.cc:385] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.502341 10333 raft_consensus.cc:740] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.503212 10331 tablet_bootstrap.cc:492] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:00.503185 10333 consensus_queue.cc:260] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
W20260709 13:04:00.501031 10011 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:00.510649 10338 tablet_bootstrap.cc:654] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.513698 10335 ts_tablet_manager.cc:1434] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.000s
I20260709 13:04:00.514524 10331 tablet_bootstrap.cc:654] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.514818 10335 tablet_bootstrap.cc:492] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:04:00.519397 10333 ts_tablet_manager.cc:1434] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.002s
I20260709 13:04:00.520591 10333 tablet_bootstrap.cc:492] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:04:00.526283 10335 tablet_bootstrap.cc:654] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.533315 10594 raft_consensus.cc:493] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.533915 10594 raft_consensus.cc:515] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.534368 10338 tablet_bootstrap.cc:492] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:00.534889 10338 ts_tablet_manager.cc:1403] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.002s
I20260709 13:04:00.537781 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af4aeb2cefe7417e9c4364dd33b00847" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:00.538264 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af4aeb2cefe7417e9c4364dd33b00847" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:00.540827 10253 leader_election.cc:304] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:00.542439 10338 raft_consensus.cc:359] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.543212 10338 raft_consensus.cc:385] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.543145 10544 raft_consensus.cc:2749] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:00.544418 10594 leader_election.cc:290] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:00.552692 10620 raft_consensus.cc:493] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.553521 10620 raft_consensus.cc:515] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
W20260709 13:04:00.555019 10105 consensus_peers.cc:597] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b -> Peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Couldn't send request to peer 8c3d3c9022c94705ac4ee834e1dc49d0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:00.543658 10338 raft_consensus.cc:740] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.556381 10594 raft_consensus.cc:493] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.556424 10338 consensus_queue.cc:260] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.556985 10594 raft_consensus.cc:515] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.559079 10338 ts_tablet_manager.cc:1434] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:04:00.559926 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6c597501a7d4e89a7a60fe4a56fcef8" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:00.560230 10338 tablet_bootstrap.cc:492] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
W20260709 13:04:00.561962 10253 leader_election.cc:343] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.563484 10594 leader_election.cc:290] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:00.563637 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb355c77dc43e78812c2b2a2df92c9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:00.562204 10620 leader_election.cc:290] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:00.565768 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb355c77dc43e78812c2b2a2df92c9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:00.566748 10139 raft_consensus.cc:2468] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:04:00.566792 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6c597501a7d4e89a7a60fe4a56fcef8" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
W20260709 13:04:00.568271 10251 leader_election.cc:343] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.568744 10251 leader_election.cc:304] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:00.569878 10544 raft_consensus.cc:2749] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:04:00.565728 10028 leader_election.cc:343] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.550928  9960 catalog_manager.cc:5697] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:00.575224 10030 leader_election.cc:304] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:04:00.576100 10620 raft_consensus.cc:2804] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:00.576483 10620 raft_consensus.cc:493] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.576828 10620 raft_consensus.cc:3060] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.581553 10338 tablet_bootstrap.cc:654] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.584065 10331 tablet_bootstrap.cc:492] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:04:00.584682 10331 ts_tablet_manager.cc:1403] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.082s	user 0.007s	sys 0.005s
I20260709 13:04:00.594316 10335 tablet_bootstrap.cc:492] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:00.594249 10331 raft_consensus.cc:359] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:00.594910 10335 ts_tablet_manager.cc:1403] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.080s	user 0.023s	sys 0.025s
I20260709 13:04:00.595075 10331 raft_consensus.cc:385] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.595125 10333 tablet_bootstrap.cc:654] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.595435 10331 raft_consensus.cc:740] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.597584 10594 raft_consensus.cc:493] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.597875 10335 raft_consensus.cc:359] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.598138 10594 raft_consensus.cc:515] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:00.598708 10335 raft_consensus.cc:385] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.599031 10335 raft_consensus.cc:740] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.598902 10331 consensus_queue.cc:260] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:00.599920 10335 consensus_queue.cc:260] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.601167 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1aaf32c658c483eb6a4a3858745515f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:00.602128 10213 raft_consensus.cc:2468] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:04:00.602409 10331 ts_tablet_manager.cc:1434] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:04:00.602511 10335 ts_tablet_manager.cc:1434] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:00.598011 10620 raft_consensus.cc:515] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.603518 10331 tablet_bootstrap.cc:492] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:00.603380 10251 leader_election.cc:304] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:00.604323 10636 raft_consensus.cc:2804] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:00.604688 10636 raft_consensus.cc:493] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.605023 10636 raft_consensus.cc:3060] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.604703 10594 leader_election.cc:290] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:00.626325 10335 tablet_bootstrap.cc:492] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:04:00.635845 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb355c77dc43e78812c2b2a2df92c9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:00.636772 10139 raft_consensus.cc:3060] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.644855 10620 leader_election.cc:290] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:00.646545 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb355c77dc43e78812c2b2a2df92c9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:00.648134 10331 tablet_bootstrap.cc:654] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:00.648532 10028 leader_election.cc:343] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.625124 10636 raft_consensus.cc:515] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:00.652916 10636 leader_election.cc:290] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:00.657526 10620 raft_consensus.cc:493] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.658437 10620 raft_consensus.cc:515] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.659287 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1aaf32c658c483eb6a4a3858745515f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:00.659884 10213 raft_consensus.cc:3060] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.661247 10620 leader_election.cc:290] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:00.661998 10335 tablet_bootstrap.cc:654] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.669621 10338 tablet_bootstrap.cc:492] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:00.670306 10338 ts_tablet_manager.cc:1403] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.110s	user 0.019s	sys 0.024s
I20260709 13:04:00.673349 10338 raft_consensus.cc:359] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.674094 10338 raft_consensus.cc:385] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.674381 10338 raft_consensus.cc:740] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.676848 10338 consensus_queue.cc:260] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.677356 10213 raft_consensus.cc:2468] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:04:00.679077 10338 ts_tablet_manager.cc:1434] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:00.679726 10624 consensus_queue.cc:1048] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:00.680390 10251 leader_election.cc:304] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:00.680977 10335 tablet_bootstrap.cc:492] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:00.681535 10335 ts_tablet_manager.cc:1403] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.055s	user 0.009s	sys 0.005s
I20260709 13:04:00.681694 10544 raft_consensus.cc:2804] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:00.682226 10544 raft_consensus.cc:697] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:00.683131 10544 consensus_queue.cc:237] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:00.684392 10335 raft_consensus.cc:359] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.685106 10335 raft_consensus.cc:385] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.685402 10335 raft_consensus.cc:740] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
W20260709 13:04:00.685818 10251 consensus_peers.cc:597] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:00.686177 10335 consensus_queue.cc:260] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.688444 10335 ts_tablet_manager.cc:1434] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 13:04:00.689285 10139 raft_consensus.cc:2468] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:04:00.690490 10338 tablet_bootstrap.cc:492] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:00.696630 10335 tablet_bootstrap.cc:492] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:04:00.691479 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1aaf32c658c483eb6a4a3858745515f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:00.699360 10064 raft_consensus.cc:2468] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:04:00.692271 10331 tablet_bootstrap.cc:492] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:04:00.694589 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1aaf32c658c483eb6a4a3858745515f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:04:00.701153 10331 ts_tablet_manager.cc:1403] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.098s	user 0.016s	sys 0.000s
I20260709 13:04:00.701201 10063 raft_consensus.cc:3060] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.703922 10335 tablet_bootstrap.cc:654] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.705133 10331 raft_consensus.cc:359] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.706722 10331 raft_consensus.cc:385] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.707152 10331 raft_consensus.cc:740] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.708045 10331 consensus_queue.cc:260] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.710819 10331 ts_tablet_manager.cc:1434] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:04:00.715093 10331 tablet_bootstrap.cc:492] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:00.709026 10635 raft_consensus.cc:493] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.715822 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027075b60dc34541a40249671ad2fcc9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:00.715724 10635 raft_consensus.cc:515] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.716820 10335 tablet_bootstrap.cc:492] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:00.717351 10335 ts_tablet_manager.cc:1403] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 13:04:00.718019 10635 leader_election.cc:290] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:00.720234 10335 raft_consensus.cc:359] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.720907 10335 raft_consensus.cc:385] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.721167 10335 raft_consensus.cc:740] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.722231 10338 tablet_bootstrap.cc:654] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.723196 10335 consensus_queue.cc:260] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.724852 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027075b60dc34541a40249671ad2fcc9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:00.725737 10139 raft_consensus.cc:2468] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:04:00.722625 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6c597501a7d4e89a7a60fe4a56fcef8" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:00.728569 10335 ts_tablet_manager.cc:1434] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:04:00.729691 10335 tablet_bootstrap.cc:492] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:04:00.736109 10331 tablet_bootstrap.cc:654] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.741609 10620 consensus_queue.cc:1048] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:00.746930 10063 raft_consensus.cc:2468] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:04:00.748750 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6c597501a7d4e89a7a60fe4a56fcef8" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:04:00.750515 10103 leader_election.cc:343] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.750541 10288 raft_consensus.cc:2468] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:04:00.756690 10105 leader_election.cc:304] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:04:00.758983 10335 tablet_bootstrap.cc:654] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.762326 10288 raft_consensus.cc:3060] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.763592 10411 raft_consensus.cc:2804] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:00.765437 10411 raft_consensus.cc:493] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.765864 10411 raft_consensus.cc:3060] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.774305 10635 consensus_queue.cc:1048] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:00.741248 10285 raft_consensus.cc:2468] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:04:00.742399 10030 leader_election.cc:304] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:04:00.785597 10030 leader_election.cc:304] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:00.791153 10411 raft_consensus.cc:515] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.821177 10411 leader_election.cc:290] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:00.791890 10621 raft_consensus.cc:2804] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:00.792217 10620 raft_consensus.cc:2804] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:00.791523 10553 consensus_queue.cc:1048] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 13:04:00.825400 10636 consensus_queue.cc:1048] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:04:00.825822 10621 raft_consensus.cc:493] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.851658  9960 catalog_manager.cc:5697] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:00.844724 10620 raft_consensus.cc:697] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:00.853562 10621 raft_consensus.cc:3060] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.864167 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6c597501a7d4e89a7a60fe4a56fcef8" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:00.854264 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6c597501a7d4e89a7a60fe4a56fcef8" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:00.867516 10338 tablet_bootstrap.cc:492] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:00.871551 10338 ts_tablet_manager.cc:1403] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.181s	user 0.018s	sys 0.048s
I20260709 13:04:00.869647 10285 raft_consensus.cc:3060] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.876936 10620 consensus_queue.cc:237] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.889976 10335 tablet_bootstrap.cc:492] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:00.890568 10335 ts_tablet_manager.cc:1403] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.161s	user 0.019s	sys 0.011s
I20260709 13:04:00.891458 10333 tablet_bootstrap.cc:492] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:00.894343 10333 ts_tablet_manager.cc:1403] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.374s	user 0.037s	sys 0.072s
I20260709 13:04:00.897647 10636 raft_consensus.cc:493] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.898243 10636 raft_consensus.cc:515] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.900629 10636 leader_election.cc:290] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:00.901412 10459 raft_consensus.cc:493] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:00.907812 10103 leader_election.cc:343] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:00.915107 10338 raft_consensus.cc:359] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.915952 10338 raft_consensus.cc:385] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.916239 10338 raft_consensus.cc:740] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.918114 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940beae71dca43e58d3879c233f87dbb" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:00.907081 10459 raft_consensus.cc:515] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.893328 10335 raft_consensus.cc:359] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.926397 10335 raft_consensus.cc:385] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.926849 10335 raft_consensus.cc:740] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.927786 10335 consensus_queue.cc:260] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:00.916889 10338 consensus_queue.cc:260] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.940436 10459 leader_election.cc:290] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:00.942312 10621 raft_consensus.cc:515] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:00.938746 10333 raft_consensus.cc:359] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.944291 10338 ts_tablet_manager.cc:1434] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.000s
I20260709 13:04:00.944095 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13fc2f8cffc54c41b5bdde0305fe868c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:00.945495 10338 tablet_bootstrap.cc:492] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:00.945845 10137 raft_consensus.cc:2468] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:04:00.946971 10621 leader_election.cc:290] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:00.947428 10179 leader_election.cc:304] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:00.948017 10285 raft_consensus.cc:2468] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:04:00.943794 10333 raft_consensus.cc:385] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.959913 10333 raft_consensus.cc:740] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:04:00.960968 10333 consensus_queue.cc:260] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.948534 10459 raft_consensus.cc:2804] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:00.963017 10459 raft_consensus.cc:493] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:00.963721 10459 raft_consensus.cc:3060] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:00.964303 10678 consensus_queue.cc:1048] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:00.942838 10331 tablet_bootstrap.cc:492] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
W20260709 13:04:00.955188 10179 consensus_peers.cc:597] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 -> Peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Couldn't send request to peer 8c3d3c9022c94705ac4ee834e1dc49d0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:00.962265 10105 leader_election.cc:304] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:04:00.948520 10335 ts_tablet_manager.cc:1434] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.000s
I20260709 13:04:00.973776 10331 ts_tablet_manager.cc:1403] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.259s	user 0.022s	sys 0.009s
I20260709 13:04:00.974649 10338 tablet_bootstrap.cc:654] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:00.974305 10333 ts_tablet_manager.cc:1434] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.079s	user 0.007s	sys 0.000s
I20260709 13:04:00.975950 10333 tablet_bootstrap.cc:492] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:04:00.976327 10411 raft_consensus.cc:2804] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:00.990665 10411 raft_consensus.cc:697] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:00.990917 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940beae71dca43e58d3879c233f87dbb" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:00.991568 10411 consensus_queue.cc:237] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.992398 10331 raft_consensus.cc:359] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:00.993166 10331 raft_consensus.cc:385] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.978148 10635 raft_consensus.cc:493] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:00.978583 10335 tablet_bootstrap.cc:492] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:04:00.981137 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13fc2f8cffc54c41b5bdde0305fe868c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:00.985014 10338 tablet_bootstrap.cc:492] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:00.994526 10635 raft_consensus.cc:515] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.995009 10064 raft_consensus.cc:2468] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:04:00.995239 10338 ts_tablet_manager.cc:1403] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.050s	user 0.008s	sys 0.004s
I20260709 13:04:00.996841 10635 leader_election.cc:290] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:00.998059 10338 raft_consensus.cc:359] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:00.998827 10338 raft_consensus.cc:385] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:00.999099 10338 raft_consensus.cc:740] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.004005 10335 tablet_bootstrap.cc:654] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.004922 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027075b60dc34541a40249671ad2fcc9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:01.004992 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6314d5916384527a155e2c956145d98" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:01.005650 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027075b60dc34541a40249671ad2fcc9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:01.005862 10213 raft_consensus.cc:2468] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:04:00.987018 10459 raft_consensus.cc:515] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:01.006326 10333 tablet_bootstrap.cc:654] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.006302 10137 raft_consensus.cc:3060] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.005704 10331 raft_consensus.cc:740] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.007117 10103 leader_election.cc:304] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:01.007495 10331 consensus_queue.cc:260] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:01.008018 10618 raft_consensus.cc:2804] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:01.008183 10459 leader_election.cc:290] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:01.008433 10618 raft_consensus.cc:493] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.008878 10618 raft_consensus.cc:3060] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.009781 10331 ts_tablet_manager.cc:1434] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:04:01.010842 10331 tablet_bootstrap.cc:492] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:01.010735 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13fc2f8cffc54c41b5bdde0305fe868c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:01.011567 10138 raft_consensus.cc:3060] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.018600 10137 raft_consensus.cc:2468] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:04:01.020578 10338 consensus_queue.cc:260] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:01.023020 10338 ts_tablet_manager.cc:1434] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:04:01.024114 10338 tablet_bootstrap.cc:492] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:01.030768 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6314d5916384527a155e2c956145d98" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:01.034943 10288 raft_consensus.cc:3060] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.035573 10252 leader_election.cc:304] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
W20260709 13:04:01.036198 10105 leader_election.cc:343] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:01.036908 10636 raft_consensus.cc:2749] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:01.063979 10618 raft_consensus.cc:515] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:01.079006 10288 raft_consensus.cc:2468] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:04:01.054306 10544 raft_consensus.cc:493] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.081722 10635 raft_consensus.cc:493] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:01.096235 10030 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:33711, user_credentials={real_user=slave}} blocked reactor thread for 55131.5us
I20260709 13:04:01.096952 10030 leader_election.cc:304] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:01.098650 10553 raft_consensus.cc:2804] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:01.099184 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13fc2f8cffc54c41b5bdde0305fe868c" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:04:01.099936 10065 raft_consensus.cc:3060] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.086030 10544 raft_consensus.cc:515] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.099298 10553 raft_consensus.cc:697] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:01.111238 10544 leader_election.cc:290] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:01.121373 10333 tablet_bootstrap.cc:492] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:01.119491  9960 catalog_manager.cc:5697] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:01.122004 10333 ts_tablet_manager.cc:1403] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.146s	user 0.031s	sys 0.042s
I20260709 13:04:01.124569 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6314d5916384527a155e2c956145d98" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:01.125327 10213 raft_consensus.cc:3060] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.098048 10635 raft_consensus.cc:515] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:01.131080 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c5f39847af4013aa501ce097f58b87" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:01.131898 10063 raft_consensus.cc:2468] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:04:01.134640 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c5f39847af4013aa501ce097f58b87" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:01.135366 10140 raft_consensus.cc:2468] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:04:01.136592 10331 tablet_bootstrap.cc:654] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.138651 10252 leader_election.cc:304] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 
I20260709 13:04:01.139772 10553 consensus_queue.cc:237] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.151772 10065 raft_consensus.cc:2468] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:04:01.158238  9963 catalog_manager.cc:5697] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:01.175136 10213 raft_consensus.cc:2468] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:04:01.141875 10636 raft_consensus.cc:2804] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:01.204933 10338 tablet_bootstrap.cc:654] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:01.193912 10682 log.cc:927] Time spent T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:04:01.205561 10103 leader_election.cc:304] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:01.206806 10693 raft_consensus.cc:2804] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:01.207428 10693 raft_consensus.cc:697] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:01.190595 10636 raft_consensus.cc:493] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.220834 10333 raft_consensus.cc:359] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:01.223491 10333 raft_consensus.cc:385] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:01.215605 10683 log.cc:927] Time spent T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:04:01.226018 10333 raft_consensus.cc:740] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.208487 10693 consensus_queue.cc:237] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:01.227628 10333 consensus_queue.cc:260] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:01.228957 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3384978cd09b40f29b899f22ded46604" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:01.226143 10138 raft_consensus.cc:2468] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:04:01.198030 10178 leader_election.cc:304] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
W20260709 13:04:01.232539 10685 log.cc:927] Time spent T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:04:01.233917 10459 raft_consensus.cc:2804] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:04:01.234184 10104 leader_election.cc:343] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:01.234460 10459 raft_consensus.cc:697] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:04:01.235248 10459 consensus_queue.cc:237] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:01.213692 10636 raft_consensus.cc:3060] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.270097 10331 tablet_bootstrap.cc:492] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:04:01.270892 10331 ts_tablet_manager.cc:1403] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.260s	user 0.030s	sys 0.018s
I20260709 13:04:01.271301 10333 ts_tablet_manager.cc:1434] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.149s	user 0.009s	sys 0.000s
I20260709 13:04:01.285331 10618 leader_election.cc:290] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:01.287835 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6314d5916384527a155e2c956145d98" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:01.304373 10333 tablet_bootstrap.cc:492] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:04:01.307981 10689 consensus_queue.cc:1048] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:04:01.314579 10283 raft_consensus.cc:3060] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:01.323606 10105 leader_election.cc:343] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:01.339630 10636 raft_consensus.cc:515] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.344069 10636 leader_election.cc:290] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:01.356196 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c5f39847af4013aa501ce097f58b87" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:04:01.356352 10695 consensus_queue.cc:1048] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:01.336654 10331 raft_consensus.cc:359] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.360631 10331 raft_consensus.cc:385] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.360976 10331 raft_consensus.cc:740] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.360455 10062 raft_consensus.cc:3060] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.358939 10335 tablet_bootstrap.cc:492] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:01.362008 10335 ts_tablet_manager.cc:1403] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.384s	user 0.021s	sys 0.007s
I20260709 13:04:01.361686 10331 consensus_queue.cc:260] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.362421 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c5f39847af4013aa501ce097f58b87" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:01.362933 10333 tablet_bootstrap.cc:654] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.363059 10137 raft_consensus.cc:3060] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.364912 10695 raft_consensus.cc:493] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.365443 10695 raft_consensus.cc:515] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:01.367631 10695 leader_election.cc:290] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:01.368314 10335 raft_consensus.cc:359] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.369035 10335 raft_consensus.cc:385] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.369329 10335 raft_consensus.cc:740] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.369539 10062 raft_consensus.cc:2468] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:04:01.370040 10335 consensus_queue.cc:260] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.371978 10331 ts_tablet_manager.cc:1434] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.101s	user 0.008s	sys 0.000s
I20260709 13:04:01.372959 10335 ts_tablet_manager.cc:1434] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:04:01.374118 10335 tablet_bootstrap.cc:492] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:04:01.375188 10252 leader_election.cc:304] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 
I20260709 13:04:01.376471 10338 tablet_bootstrap.cc:492] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:01.376998 10338 ts_tablet_manager.cc:1403] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.353s	user 0.020s	sys 0.012s
I20260709 13:04:01.378139 10688 raft_consensus.cc:2804] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:01.378701 10688 raft_consensus.cc:697] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:01.379220 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d3e10044ce408294e1b86ce960b84a" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:01.379595 10688 consensus_queue.cc:237] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.379228 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d3e10044ce408294e1b86ce960b84a" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:01.385411 10333 tablet_bootstrap.cc:492] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:01.386036 10333 ts_tablet_manager.cc:1403] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.091s	user 0.013s	sys 0.000s
I20260709 13:04:01.389858 10331 tablet_bootstrap.cc:492] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:01.395702 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3384978cd09b40f29b899f22ded46604" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:01.396847 10251 leader_election.cc:304] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:01.383782 10635 leader_election.cc:290] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
W20260709 13:04:01.400481 10103 leader_election.cc:343] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:01.400696 10338 raft_consensus.cc:359] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:01.401033 10103 leader_election.cc:304] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:01.401115 10694 raft_consensus.cc:2749] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:01.401521 10338 raft_consensus.cc:385] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.401908 10338 raft_consensus.cc:740] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.402645 10338 consensus_queue.cc:260] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:01.406054 10687 raft_consensus.cc:493] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.406663 10687 raft_consensus.cc:515] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.411096 10333 raft_consensus.cc:359] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:01.411855 10333 raft_consensus.cc:385] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.412143 10333 raft_consensus.cc:740] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.413072 10331 tablet_bootstrap.cc:654] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.419780 10333 consensus_queue.cc:260] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:01.409137 10687 leader_election.cc:290] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:01.422148 10333 ts_tablet_manager.cc:1434] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.036s	user 0.005s	sys 0.001s
I20260709 13:04:01.427637 10333 tablet_bootstrap.cc:492] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:04:01.427369 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:01.428280 10138 raft_consensus.cc:2468] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:04:01.423385 10335 tablet_bootstrap.cc:654] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.429795 10688 consensus_queue.cc:1048] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:01.431928 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:01.432924 10635 raft_consensus.cc:2749] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:01.436949  9961 catalog_manager.cc:5697] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:01.440429 10103 consensus_peers.cc:597] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.444615 10331 tablet_bootstrap.cc:492] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:04:01.444640 10338 ts_tablet_manager.cc:1434] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 13:04:01.445338 10331 ts_tablet_manager.cc:1403] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.056s	user 0.013s	sys 0.003s
I20260709 13:04:01.451303 10678 raft_consensus.cc:493] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.451901 10678 raft_consensus.cc:515] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:01.453883 10689 raft_consensus.cc:493] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.454228 10678 leader_election.cc:290] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:01.454394 10338 tablet_bootstrap.cc:492] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:01.454473 10689 raft_consensus.cc:515] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:01.461050 10335 tablet_bootstrap.cc:492] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:01.462124 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc9831f2c60444db097f11766636c56" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:01.463132 10335 ts_tablet_manager.cc:1403] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.088s	user 0.008s	sys 0.004s
I20260709 13:04:01.454455 10030 leader_election.cc:304] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:01.456607  9960 catalog_manager.cc:5697] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:01.465468 10678 raft_consensus.cc:2804] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:01.465688 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5893ec967cb241af8709bba918f3429a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:01.465917 10678 raft_consensus.cc:493] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.466339 10678 raft_consensus.cc:3060] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.466437 10212 raft_consensus.cc:2468] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:04:01.470139 10028 leader_election.cc:304] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:01.470872 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5893ec967cb241af8709bba918f3429a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:01.471127 10687 raft_consensus.cc:2804] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:01.471628 10687 raft_consensus.cc:493] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.471997 10687 raft_consensus.cc:3060] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.458945  9963 catalog_manager.cc:5697] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:01.475005 10335 raft_consensus.cc:359] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:01.481709 10335 raft_consensus.cc:385] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.482560 10335 raft_consensus.cc:740] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.450129 10137 raft_consensus.cc:2468] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:04:01.470153 10689 leader_election.cc:290] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:01.453284 10333 tablet_bootstrap.cc:654] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.478137 10338 tablet_bootstrap.cc:654] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:01.489904 10030 leader_election.cc:343] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:01.484745 10335 consensus_queue.cc:260] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
W20260709 13:04:01.491030 10030 leader_election.cc:343] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:01.495203 10335 ts_tablet_manager.cc:1434] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.002s
I20260709 13:04:01.496354 10335 tablet_bootstrap.cc:492] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:04:01.508018 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdc9831f2c60444db097f11766636c56" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
W20260709 13:04:01.508965 10028 consensus_peers.cc:597] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:01.514415  9962 catalog_manager.cc:5697] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:01.516021 10178 leader_election.cc:304] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:01.516909 10689 raft_consensus.cc:2749] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:01.519723 10553 consensus_queue.cc:1048] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:01.527334 10288 raft_consensus.cc:3060] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.526203 10331 raft_consensus.cc:359] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:01.529094 10331 raft_consensus.cc:385] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.530707 10331 raft_consensus.cc:740] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.531446 10331 consensus_queue.cc:260] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:01.534871 10331 ts_tablet_manager.cc:1434] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.089s	user 0.006s	sys 0.000s
I20260709 13:04:01.513006 10678 raft_consensus.cc:515] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.543412 10678 leader_election.cc:290] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:01.545447 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:01.537761 10331 tablet_bootstrap.cc:492] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:01.546324 10139 raft_consensus.cc:3060] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.496856 10687 raft_consensus.cc:515] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:01.544945 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:01.602849 10687 leader_election.cc:290] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:01.603554 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5893ec967cb241af8709bba918f3429a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:01.604247 10211 raft_consensus.cc:3060] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.608543 10139 raft_consensus.cc:2468] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:04:01.613679 10030 leader_election.cc:304] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:01.610042 10695 raft_consensus.cc:493] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.615895 10695 raft_consensus.cc:515] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:01.616914 10678 raft_consensus.cc:2804] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:01.617508 10678 raft_consensus.cc:697] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:01.617815 10695 leader_election.cc:290] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:01.618360 10678 consensus_queue.cc:237] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.622787 10333 tablet_bootstrap.cc:492] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:01.623347 10333 ts_tablet_manager.cc:1403] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.196s	user 0.015s	sys 0.006s
I20260709 13:04:01.624526 10635 consensus_queue.cc:1048] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:01.625798 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6314d5916384527a155e2c956145d98" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:01.626681 10214 raft_consensus.cc:2393] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in current term 1: Already voted for candidate bfa051e18d0549eabe79dfcc7610d99b in this term.
I20260709 13:04:01.626152 10333 raft_consensus.cc:359] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:01.627893 10333 raft_consensus.cc:385] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.628161 10333 raft_consensus.cc:740] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.629037 10333 consensus_queue.cc:260] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
W20260709 13:04:01.631129 10030 leader_election.cc:343] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:01.635257 10335 tablet_bootstrap.cc:654] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.636523 10333 ts_tablet_manager.cc:1434] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.002s
I20260709 13:04:01.640643 10338 tablet_bootstrap.cc:492] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:01.641491 10338 ts_tablet_manager.cc:1403] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.195s	user 0.018s	sys 0.000s
W20260709 13:04:01.641857 10011 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:01.626065 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5893ec967cb241af8709bba918f3429a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:01.648877 10333 tablet_bootstrap.cc:492] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
W20260709 13:04:01.651717 10030 leader_election.cc:343] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:01.655718 10331 tablet_bootstrap.cc:654] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.650342 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6314d5916384527a155e2c956145d98" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:01.671653 10253 leader_election.cc:304] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:01.674152 10636 raft_consensus.cc:3060] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.676153 10338 raft_consensus.cc:359] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.686805 10338 raft_consensus.cc:385] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.687160 10338 raft_consensus.cc:740] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.690464 10338 consensus_queue.cc:260] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.689695 10333 tablet_bootstrap.cc:654] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.695832 10338 ts_tablet_manager.cc:1434] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.054s	user 0.008s	sys 0.000s
I20260709 13:04:01.696625  9962 catalog_manager.cc:5697] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:01.698036 10687 consensus_queue.cc:1048] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:01.700911 10636 raft_consensus.cc:2749] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:01.710155 10687 consensus_queue.cc:1048] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:01.716603 10338 tablet_bootstrap.cc:492] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:01.718255 10537 consensus_queue.cc:1048] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:01.719117 10689 consensus_queue.cc:1048] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:01.742547 10335 tablet_bootstrap.cc:492] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:01.743134 10335 ts_tablet_manager.cc:1403] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.247s	user 0.013s	sys 0.004s
I20260709 13:04:01.754793 10335 raft_consensus.cc:359] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:01.756347 10335 raft_consensus.cc:385] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.756599 10335 raft_consensus.cc:740] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.757246 10335 consensus_queue.cc:260] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:01.760157 10335 ts_tablet_manager.cc:1434] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:04:01.761188 10335 tablet_bootstrap.cc:492] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:04:01.764060 10211 raft_consensus.cc:2468] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:04:01.765235 10333 tablet_bootstrap.cc:492] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:01.765826 10333 ts_tablet_manager.cc:1403] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.122s	user 0.007s	sys 0.006s
I20260709 13:04:01.766971 10331 tablet_bootstrap.cc:492] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:04:01.767401 10028 leader_election.cc:304] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:01.768776 10678 raft_consensus.cc:2804] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:01.772646 10333 raft_consensus.cc:359] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:01.773413 10333 raft_consensus.cc:385] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.773720 10333 raft_consensus.cc:740] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.774489 10338 tablet_bootstrap.cc:654] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.774428 10333 consensus_queue.cc:260] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:01.775347 10635 consensus_queue.cc:1048] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:01.776222 10678 raft_consensus.cc:697] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:01.777184 10678 consensus_queue.cc:237] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:01.779598 10335 tablet_bootstrap.cc:654] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.781502 10333 ts_tablet_manager.cc:1434] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:04:01.782555 10333 tablet_bootstrap.cc:492] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:04:01.796183 10214 raft_consensus.cc:3060] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.811239 10331 ts_tablet_manager.cc:1403] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.274s	user 0.006s	sys 0.027s
I20260709 13:04:01.806285 10693 consensus_queue.cc:1048] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:04:01.801281 10333 tablet_bootstrap.cc:654] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.831075 10331 raft_consensus.cc:359] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:01.858851 10338 tablet_bootstrap.cc:492] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:01.861826 10338 ts_tablet_manager.cc:1403] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.152s	user 0.013s	sys 0.005s
I20260709 13:04:01.864022 10331 raft_consensus.cc:385] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.869942 10331 raft_consensus.cc:740] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.870054 10624 raft_consensus.cc:493] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.871913 10338 raft_consensus.cc:359] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:01.872357 10624 raft_consensus.cc:515] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:01.872839 10338 raft_consensus.cc:385] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.873153 10338 raft_consensus.cc:740] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.873878 10338 consensus_queue.cc:260] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:01.874648 10624 leader_election.cc:290] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:01.877650 10338 ts_tablet_manager.cc:1434] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:04:01.887645 10331 consensus_queue.cc:260] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:01.893697 10333 tablet_bootstrap.cc:492] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:01.895126 10333 ts_tablet_manager.cc:1403] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.113s	user 0.019s	sys 0.009s
I20260709 13:04:01.895751 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3384978cd09b40f29b899f22ded46604" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:01.896442 10139 raft_consensus.cc:2468] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:04:01.904656 10331 ts_tablet_manager.cc:1434] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.093s	user 0.006s	sys 0.001s
I20260709 13:04:01.906234 10338 tablet_bootstrap.cc:492] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:01.906404 10331 tablet_bootstrap.cc:492] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:01.895730 10689 raft_consensus.cc:493] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.908738 10689 raft_consensus.cc:515] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:01.919065 10338 tablet_bootstrap.cc:654] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.900566 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3384978cd09b40f29b899f22ded46604" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:01.921728 10211 raft_consensus.cc:2468] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:04:01.923127 10028 leader_election.cc:304] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:01.924060 10687 raft_consensus.cc:2804] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:01.924142 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb355c77dc43e78812c2b2a2df92c9" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:01.925663 10689 leader_election.cc:290] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:01.924491 10687 raft_consensus.cc:493] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:01.927670 10687 raft_consensus.cc:3060] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:01.927537 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb355c77dc43e78812c2b2a2df92c9" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:01.929577 10331 tablet_bootstrap.cc:654] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:01.929478 10335 tablet_bootstrap.cc:492] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:01.934259 10335 ts_tablet_manager.cc:1403] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.173s	user 0.038s	sys 0.032s
I20260709 13:04:01.936859 10688 consensus_queue.cc:1048] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:01.942875 10618 raft_consensus.cc:493] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:01.943462 10618 raft_consensus.cc:515] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:01.944468 10688 consensus_queue.cc:1048] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:01.945649 10618 leader_election.cc:290] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:01.946586 10333 raft_consensus.cc:359] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:01.948415 10333 raft_consensus.cc:385] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.949170 10333 raft_consensus.cc:740] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.955360 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4d469562c14664860f49baa277f1d9" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:04:01.957137 10105 leader_election.cc:343] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:01.956306 10179 leader_election.cc:304] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:01.960706 10335 raft_consensus.cc:359] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.962387 10335 raft_consensus.cc:385] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:01.952873 10333 consensus_queue.cc:260] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:01.962776 10335 raft_consensus.cc:740] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:01.979007 10335 consensus_queue.cc:260] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:01.973819 10537 raft_consensus.cc:2749] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:01.975006 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4d469562c14664860f49baa277f1d9" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:02.024613 10333 ts_tablet_manager.cc:1434] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.129s	user 0.010s	sys 0.004s
I20260709 13:04:02.036769 10338 tablet_bootstrap.cc:492] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:02.037293 10338 ts_tablet_manager.cc:1403] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.131s	user 0.028s	sys 0.023s
I20260709 13:04:02.038231 10333 tablet_bootstrap.cc:492] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:04:02.042598 10687 raft_consensus.cc:515] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:02.045069 10687 leader_election.cc:290] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
W20260709 13:04:02.049805 10104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 45921.4us
I20260709 13:04:02.051608 10335 ts_tablet_manager.cc:1434] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.117s	user 0.006s	sys 0.005s
W20260709 13:04:02.052218 10103 leader_election.cc:343] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:02.052731 10103 leader_election.cc:304] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:04:02.054546 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3384978cd09b40f29b899f22ded46604" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:02.055192 10140 raft_consensus.cc:3060] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:02.052696 10335 tablet_bootstrap.cc:492] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:04:02.058887  9961 catalog_manager.cc:5697] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:02.058895 10635 raft_consensus.cc:2749] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:02.079082 10331 tablet_bootstrap.cc:492] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:04:02.079695 10331 ts_tablet_manager.cc:1403] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.174s	user 0.009s	sys 0.039s
I20260709 13:04:02.082638 10140 raft_consensus.cc:2468] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:04:02.096869 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3384978cd09b40f29b899f22ded46604" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:02.101028 10338 raft_consensus.cc:359] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:02.106770 10338 raft_consensus.cc:385] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:02.107048 10338 raft_consensus.cc:740] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:02.102768 10621 consensus_queue.cc:1048] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:04:02.111177 10212 raft_consensus.cc:3060] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:02.101580 10331 raft_consensus.cc:359] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:02.112003 10331 raft_consensus.cc:385] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:02.112347 10331 raft_consensus.cc:740] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:02.110812 10338 consensus_queue.cc:260] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:02.100135 10030 leader_election.cc:304] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:02.100521 10688 raft_consensus.cc:493] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.119879 10688 raft_consensus.cc:515] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:02.122936 10331 consensus_queue.cc:260] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:02.127645 10338 ts_tablet_manager.cc:1434] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.090s	user 0.005s	sys 0.006s
I20260709 13:04:02.122973 10621 raft_consensus.cc:2804] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:02.129920 10333 tablet_bootstrap.cc:654] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:02.129102 10621 raft_consensus.cc:697] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:02.130180 10695 raft_consensus.cc:493] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.130712 10695 raft_consensus.cc:515] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:02.131114 10621 consensus_queue.cc:237] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:02.133983 10695 leader_election.cc:290] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:02.135840 10688 leader_election.cc:290] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:02.140919 10331 ts_tablet_manager.cc:1434] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.061s	user 0.009s	sys 0.001s
I20260709 13:04:02.140425 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:02.169724 10140 raft_consensus.cc:2393] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in current term 1: Already voted for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in this term.
I20260709 13:04:02.165365 10693 raft_consensus.cc:493] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.173460 10678 raft_consensus.cc:493] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.174023 10678 raft_consensus.cc:515] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:02.175439 10213 raft_consensus.cc:3060] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:02.177083 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5893ec967cb241af8709bba918f3429a" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:02.177995 10211 raft_consensus.cc:2393] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in current term 1: Already voted for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in this term.
I20260709 13:04:02.180955 10693 raft_consensus.cc:515] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:02.183245 10693 leader_election.cc:290] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:02.185395 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bc0e52b9424f6dab141d4c1e28023b" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:02.186676 10678 leader_election.cc:290] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:02.185541 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bc0e52b9424f6dab141d4c1e28023b" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:02.187532 10062 raft_consensus.cc:2468] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:04:02.189528 10104 leader_election.cc:304] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:02.190033 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bc0e52b9424f6dab141d4c1e28023b" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:02.192679 10411 raft_consensus.cc:2804] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:02.193046 10212 raft_consensus.cc:2468] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:04:02.239997 10688 raft_consensus.cc:493] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.197402 10331 tablet_bootstrap.cc:492] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:02.253191 10688 raft_consensus.cc:515] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:02.222084 10691 raft_consensus.cc:493] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:02.254364 10691 raft_consensus.cc:515] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:02.193255 10411 raft_consensus.cc:493] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:02.255484 10411 raft_consensus.cc:3060] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:02.299010 10331 tablet_bootstrap.cc:654] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:02.202715 10028 leader_election.cc:343] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:02.329774 10028 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 127120us
W20260709 13:04:02.331501 10252 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 236171us
I20260709 13:04:02.331552 10331 tablet_bootstrap.cc:492] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:04:02.334244 10331 ts_tablet_manager.cc:1403] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.137s	user 0.015s	sys 0.006s
I20260709 13:04:02.335839 10678 consensus_queue.cc:1048] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:02.255597 10688 leader_election.cc:290] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:02.260259 10691 leader_election.cc:290] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:02.200582 10335 tablet_bootstrap.cc:654] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:02.263810 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:02.316990 10333 tablet_bootstrap.cc:492] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:02.341627 10333 ts_tablet_manager.cc:1403] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.308s	user 0.013s	sys 0.010s
W20260709 13:04:02.342278 10251 leader_election.cc:343] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:02.342935 10411 raft_consensus.cc:515] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:02.349018 10411 leader_election.cc:290] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:02.350333 10331 raft_consensus.cc:359] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:02.373359 10331 raft_consensus.cc:385] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:02.373894 10331 raft_consensus.cc:740] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:02.374754 10331 consensus_queue.cc:260] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:02.379758 10331 ts_tablet_manager.cc:1434] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.004s
I20260709 13:04:02.344463 10333 raft_consensus.cc:359] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:02.380888 10333 raft_consensus.cc:385] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:02.381098 10331 tablet_bootstrap.cc:492] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:02.381220 10333 raft_consensus.cc:740] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
W20260709 13:04:02.194172 10103 leader_election.cc:343] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:02.345432 10104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 156062us
I20260709 13:04:02.384958 10333 consensus_queue.cc:260] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.018886 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e94ff7589e401ba16e6019e0a4d6eb" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.019589 10062 raft_consensus.cc:2468] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:04:03.020974 10333 ts_tablet_manager.cc:1434] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.679s	user 0.006s	sys 0.000s
W20260709 13:04:03.021498 10333 ts_tablet_manager.cc:1467] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577: Tablet startup took 679ms
W20260709 13:04:03.021842 10333 ts_tablet_manager.cc:1469] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577: Trace:
0709 13:03:54.839846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:03:54.840013 (+   167us) service_pool.cc:224] Handling call
0709 13:03:54.848970 (+  8957us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 13:03:54.849087 (+   117us) ts_tablet_manager.cc:682] Creating new metadata...
0709 13:03:54.926470 (+ 77383us) tablet_metadata.cc:665] Metadata flushed
0709 13:03:55.022321 (+ 95851us) tablet_replica.cc:206] Creating consensus instance
0709 13:03:55.022941 (+   620us) inbound_call.cc:177] Queueing success response
0709 13:04:02.028315 (+7005374us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:04:02.341957 (+313642us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:04:02.342429 (+   472us) tablet_replica.cc:261] Starting instrumentation
0709 13:04:02.342945 (+   516us) tablet_replica.cc:280] Starting consensus
Metrics: {"raft.queue_time_us":148,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"tablet-open.queue_time_us":7005489}
I20260709 13:04:03.024407 10333 tablet_bootstrap.cc:492] T a8bc0e52b9424f6dab141d4c1e28023b P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:04:02.388489 10537 raft_consensus.cc:493] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.026607 10537 raft_consensus.cc:515] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:02.389215 10331 tablet_bootstrap.cc:654] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:03.028855 10537 leader_election.cc:290] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:02.407919 10335 tablet_bootstrap.cc:492] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:03.029925 10335 ts_tablet_manager.cc:1403] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.977s	user 0.012s	sys 0.004s
I20260709 13:04:03.032819 10335 raft_consensus.cc:359] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:03.040889 10335 raft_consensus.cc:385] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:04:02.408149 10103 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 214019us
I20260709 13:04:03.034937 10704 raft_consensus.cc:493] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c3d3c9022c94705ac4ee834e1dc49d0)
I20260709 13:04:02.836388 10338 tablet_bootstrap.cc:492] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:03.044224 10704 raft_consensus.cc:515] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.049937 10704 leader_election.cc:290] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:03.050395 10210 raft_consensus.cc:3060] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.051541 10335 raft_consensus.cc:740] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:03.056651 10335 consensus_queue.cc:260] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:03.059126 10335 ts_tablet_manager.cc:1434] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.001s
I20260709 13:04:03.070570 10335 tablet_bootstrap.cc:492] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
I20260709 13:04:03.074415  9963 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46024 (request call id 88) took 1069 ms. Trace:
I20260709 13:04:03.074936  9963 rpcz_store.cc:276] 0709 13:04:02.027249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.028366 (+  1117us) service_pool.cc:224] Handling call
0709 13:04:03.074316 (+1045950us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32378,"reported_tablets":2}
I20260709 13:04:03.067121 10338 tablet_bootstrap.cc:654] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:03.078565 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e94ff7589e401ba16e6019e0a4d6eb" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.062813 10104 leader_election.cc:304] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:03.082172 10750 raft_consensus.cc:2804] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:03.082683 10750 raft_consensus.cc:493] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.083094 10750 raft_consensus.cc:3060] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.086781 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bc0e52b9424f6dab141d4c1e28023b" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:04:03.087720 10065 raft_consensus.cc:3060] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.092414 10338 tablet_bootstrap.cc:492] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:03.092751 10333 tablet_bootstrap.cc:654] T a8bc0e52b9424f6dab141d4c1e28023b P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:03.093003 10338 ts_tablet_manager.cc:1403] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.257s	user 0.008s	sys 0.003s
I20260709 13:04:03.094498 10745 raft_consensus.cc:493] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.112972 10745 raft_consensus.cc:515] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.117017 10704 raft_consensus.cc:493] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c3d3c9022c94705ac4ee834e1dc49d0)
I20260709 13:04:03.117867 10704 raft_consensus.cc:515] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.119343 10745 leader_election.cc:290] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:03.121225 10065 raft_consensus.cc:2468] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:04:03.135159 10749 raft_consensus.cc:493] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcf45a2176234db5957dc38e32554577)
I20260709 13:04:03.135833 10749 raft_consensus.cc:515] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.108798 10338 raft_consensus.cc:359] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:03.109372 10284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6c597501a7d4e89a7a60fe4a56fcef8" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.110061 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bc0e52b9424f6dab141d4c1e28023b" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:03.138527 10749 leader_election.cc:290] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:03.146447 10285 raft_consensus.cc:3060] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.161633 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9b66ba4b81428f896a77bd88bc7e4d" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.162461 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd01deadf87945439c630e949d9c2e7b" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.165802 10758 raft_consensus.cc:493] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcf45a2176234db5957dc38e32554577)
I20260709 13:04:03.267791 10758 raft_consensus.cc:515] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.103675 10750 raft_consensus.cc:515] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:03.121287 10704 leader_election.cc:290] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:03.271198 10750 leader_election.cc:290] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:03.271658 10704 raft_consensus.cc:493] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.272446 10764 raft_consensus.cc:493] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcf45a2176234db5957dc38e32554577)
I20260709 13:04:03.181623 10135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8bc0e52b9424f6dab141d4c1e28023b" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:03.273337 10135 raft_consensus.cc:2393] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in current term 1: Already voted for candidate bfa051e18d0549eabe79dfcc7610d99b in this term.
I20260709 13:04:03.273708 10758 leader_election.cc:290] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:03.194960 10746 raft_consensus.cc:493] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bfa051e18d0549eabe79dfcc7610d99b)
I20260709 13:04:03.274405 10746 raft_consensus.cc:515] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:03.275312 10686 raft_consensus.cc:493] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:03.275866 10686 raft_consensus.cc:515] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.276777 10746 leader_election.cc:290] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.218086 10748 raft_consensus.cc:493] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcf45a2176234db5957dc38e32554577)
I20260709 13:04:03.277637 10760 raft_consensus.cc:493] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c3d3c9022c94705ac4ee834e1dc49d0)
I20260709 13:04:03.278002 10748 raft_consensus.cc:515] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.278192 10760 raft_consensus.cc:515] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:03.280908 10748 leader_election.cc:290] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.219419 10104 leader_election.cc:304] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
W20260709 13:04:03.281929 10104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 62655.8us
I20260709 13:04:03.284511 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e94ff7589e401ba16e6019e0a4d6eb" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:04:03.284649 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065434c541d4f10a459f9003a463d86" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.286803 10748 raft_consensus.cc:2804] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:03.287354 10748 raft_consensus.cc:697] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:03.285235 10065 raft_consensus.cc:3060] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.290699 10760 leader_election.cc:290] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:03.293732 10748 consensus_queue.cc:237] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
W20260709 13:04:03.297607 10028 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 49128.4us
I20260709 13:04:03.298938 10686 leader_election.cc:290] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.243564 10745 raft_consensus.cc:493] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:03.252615 10754 raft_consensus.cc:493] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.309226 10745 raft_consensus.cc:515] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:03.309371 10754 raft_consensus.cc:515] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:03.311676 10745 leader_election.cc:290] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:03.311836 10754 leader_election.cc:290] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:03.312616 10746 raft_consensus.cc:493] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bfa051e18d0549eabe79dfcc7610d99b)
I20260709 13:04:03.313344 10746 raft_consensus.cc:515] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:03.272194 10704 raft_consensus.cc:515] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.174798 10759 raft_consensus.cc:493] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcf45a2176234db5957dc38e32554577)
I20260709 13:04:03.316154 10759 raft_consensus.cc:515] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.318604 10759 leader_election.cc:290] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:03.319603 10771 raft_consensus.cc:493] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcf45a2176234db5957dc38e32554577)
I20260709 13:04:03.320081 10771 raft_consensus.cc:515] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.321352 10764 raft_consensus.cc:515] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.322279 10771 leader_election.cc:290] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.323585 10764 leader_election.cc:290] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.323993 10746 leader_election.cc:290] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.317330 10704 leader_election.cc:290] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:03.325873 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5893ec967cb241af8709bba918f3429a" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.326292  9962 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:46016 (request call id 88) took 1212 ms. Trace:
I20260709 13:04:03.301931  9961 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:45994 (request call id 71) took 1376 ms. Trace:
I20260709 13:04:03.326745  9962 rpcz_store.cc:276] 0709 13:04:02.113619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:02.113785 (+   166us) service_pool.cc:224] Handling call
0709 13:04:03.326223 (+1212438us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1069724,"reported_tablets":3}
I20260709 13:04:03.301791 10760 raft_consensus.cc:493] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c3d3c9022c94705ac4ee834e1dc49d0)
I20260709 13:04:03.240273 10761 raft_consensus.cc:493] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8c3d3c9022c94705ac4ee834e1dc49d0)
I20260709 13:04:03.328819 10761 raft_consensus.cc:515] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:03.331549 10761 leader_election.cc:290] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:03.327247  9961 rpcz_store.cc:276] 0709 13:04:01.925258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:04:01.929759 (+  4501us) service_pool.cc:224] Handling call
0709 13:04:03.301829 (+1372070us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:04:02.077497 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:04:02.078045 (+   548us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:04:02.078080 (+    35us) write_op.cc:435] Acquired schema lock
0709 13:04:02.078095 (+    15us) tablet.cc:662] Decoding operations
0709 13:04:02.078440 (+   345us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:04:02.078496 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 13:04:02.078521 (+    25us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:04:02.078707 (+   186us) tablet.cc:701] Row locks acquired
0709 13:04:02.078719 (+    12us) write_op.cc:260] PREPARE: finished
0709 13:04:02.078856 (+   137us) write_op.cc:270] Start()
0709 13:04:02.078957 (+   101us) write_op.cc:276] Timestamp: P: 1783602242078817 usec, L: 0
0709 13:04:02.078974 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 13:04:02.079542 (+   568us) log.cc:844] Serialized 540 byte log entry
0709 13:04:02.081407 (+  1865us) op_driver.cc:464] REPLICATION: finished
0709 13:04:02.081840 (+   433us) write_op.cc:301] APPLY: starting
0709 13:04:02.081919 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:04:02.082012 (+    93us) tablet.cc:1370] finished BulkCheckPresence
0709 13:04:02.082101 (+    89us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:04:02.082292 (+   191us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:04:02.082468 (+   176us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:04:02.082510 (+    42us) write_op.cc:312] APPLY: finished
0709 13:04:02.082709 (+   199us) log.cc:844] Serialized 22 byte log entry
0709 13:04:02.082870 (+   161us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:02.082957 (+    87us) write_op.cc:454] Released schema lock
0709 13:04:02.083000 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:04:02.083323 (+   323us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:04:02.083349 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1206350,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":239,"apply.run_cpu_time_us":1450,"apply.run_wall_time_us":1450,"num_ops":1,"prepare.queue_time_us":128,"prepare.run_cpu_time_us":2344,"prepare.run_wall_time_us":2339,"replication_time_us":2318}]]}
I20260709 13:04:03.297688 10338 raft_consensus.cc:385] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:03.326025 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.334209 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0cced7e11946f09a13c6adcf95833b" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.334704 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.335350 10338 raft_consensus.cc:740] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:03.327780 10760 raft_consensus.cc:515] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:03.338891 10760 leader_election.cc:290] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:03.338825 10338 consensus_queue.cc:260] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:03.339375 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.341539 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.342530 10063 raft_consensus.cc:2468] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:04:03.343297 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2679daef0c4447c5823833ed4577441c" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.308661 10335 tablet_bootstrap.cc:654] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:03.345754 10062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96acb559acc745428add2c682cf83cfc" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.346745 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102b42f582ce49458990caefcc8d451c" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.348268 10338 ts_tablet_manager.cc:1434] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.255s	user 0.013s	sys 0.000s
W20260709 13:04:03.348870 10104 leader_election.cc:343] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:03.326939 10065 raft_consensus.cc:2468] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:04:03.351044 10285 raft_consensus.cc:2468] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:04:03.359040 10104 leader_election.cc:304] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:03.360306 10748 raft_consensus.cc:2804] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:03.360854 10179 leader_election.cc:304] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:03.361732 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd01deadf87945439c630e949d9c2e7b" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.363037 10766 raft_consensus.cc:2804] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:03.363581 10766 raft_consensus.cc:493] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.364069 10766 raft_consensus.cc:3060] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.365790 10331 tablet_bootstrap.cc:492] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:04:03.366389 10331 ts_tablet_manager.cc:1403] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.986s	user 0.039s	sys 0.085s
I20260709 13:04:03.367573 10338 tablet_bootstrap.cc:492] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:03.360889 10748 raft_consensus.cc:697] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:03.371469 10748 consensus_queue.cc:237] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:03.373828 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee9b66ba4b81428f896a77bd88bc7e4d" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
W20260709 13:04:03.376165 10103 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 122241us
W20260709 13:04:03.384662 10103 leader_election.cc:343] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:03.387558 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0065434c541d4f10a459f9003a463d86" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.378172 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4f4fa6d20f4f1ca0b1befcc3a605b5" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.380897 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.391475 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0cced7e11946f09a13c6adcf95833b" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.393368 10750 raft_consensus.cc:493] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcf45a2176234db5957dc38e32554577)
I20260709 13:04:03.394477 10750 raft_consensus.cc:515] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.413654 10762 raft_consensus.cc:493] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcf45a2176234db5957dc38e32554577)
I20260709 13:04:03.413950 10750 leader_election.cc:290] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.415601 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.416501 10064 raft_consensus.cc:2468] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:04:03.413672 10331 raft_consensus.cc:359] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.420635 10331 raft_consensus.cc:385] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:03.420928 10331 raft_consensus.cc:740] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:03.422276 10030 leader_election.cc:304] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:03.422796 10331 consensus_queue.cc:260] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.410616 10746 raft_consensus.cc:493] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bfa051e18d0549eabe79dfcc7610d99b)
I20260709 13:04:03.426891 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "102b42f582ce49458990caefcc8d451c" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:03.414350 10762 raft_consensus.cc:515] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.436316 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4f4fa6d20f4f1ca0b1befcc3a605b5" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.423730 10750 raft_consensus.cc:493] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcf45a2176234db5957dc38e32554577)
I20260709 13:04:03.425024 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "220a3db850dd4a5280bf7c8e9519d9d2" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.425565 10686 raft_consensus.cc:493] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcf45a2176234db5957dc38e32554577)
I20260709 13:04:03.430312 10759 raft_consensus.cc:2749] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:03.428211 10754 raft_consensus.cc:493] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcf45a2176234db5957dc38e32554577)
I20260709 13:04:03.422015 10769 raft_consensus.cc:493] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bcf45a2176234db5957dc38e32554577)
I20260709 13:04:03.451669 10252 leader_election.cc:304] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.445645 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.454674 10252 leader_election.cc:304] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:03.462239 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7939918ac27c421f99a6747c1ea4a994" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.464738 10331 ts_tablet_manager.cc:1434] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.098s	user 0.010s	sys 0.006s
I20260709 13:04:03.439781 10283 raft_consensus.cc:3060] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.429864 10746 raft_consensus.cc:515] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:03.440197 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6c597501a7d4e89a7a60fe4a56fcef8" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:03.440804 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2679daef0c4447c5823833ed4577441c" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:03.442565 10762 leader_election.cc:290] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.427577 10103 leader_election.cc:304] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.441159 10284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde8fc103c3d453abd4d298b1beffd74" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:04:03.472214 10011 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:03.474032 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0709ff33d24996ae638187a8e7c57f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.474550 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9785f60ee640d686f599267cf005fb" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.486135 10338 tablet_bootstrap.cc:654] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:03.494184 10335 tablet_bootstrap.cc:492] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:03.494866 10335 ts_tablet_manager.cc:1403] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.425s	user 0.015s	sys 0.010s
I20260709 13:04:03.497874 10331 tablet_bootstrap.cc:492] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:03.475260 10747 raft_consensus.cc:2749] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:03.468273 10286 raft_consensus.cc:2468] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:04:03.460633 10750 raft_consensus.cc:515] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:03.459795 10754 raft_consensus.cc:515] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.451726 10769 raft_consensus.cc:515] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.505792 10136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be75bbbdef504a5793ba0b07feaff6f8" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:03.460812 10686 raft_consensus.cc:515] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.496162 10762 raft_consensus.cc:2749] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.491286 10135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:03.497447 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.494060 10282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16541bea7ded48538adada244223f2c9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.522820 10766 raft_consensus.cc:515] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.509233 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be75bbbdef504a5793ba0b07feaff6f8" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.515583 10134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7939918ac27c421f99a6747c1ea4a994" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:03.524331 10686 leader_election.cc:290] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.525107 10766 leader_election.cc:290] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:03.517663 10769 leader_election.cc:290] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:03.518481 10335 raft_consensus.cc:359] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.501442 10758 consensus_queue.cc:1048] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 13:04:03.525446 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16541bea7ded48538adada244223f2c9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:03.541662 10747 raft_consensus.cc:493] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.543985 10747 raft_consensus.cc:515] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:03.544368 10140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "220a3db850dd4a5280bf7c8e9519d9d2" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:03.545694 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde8fc103c3d453abd4d298b1beffd74" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:03.547797 10179 leader_election.cc:304] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:03.548221 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84c07107a48406d9fd585e9982d3069" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.525689 10214 raft_consensus.cc:2393] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in current term 1: Already voted for candidate bcf45a2176234db5957dc38e32554577 in this term.
I20260709 13:04:03.549847 10745 raft_consensus.cc:2749] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:03.526914 10746 leader_election.cc:290] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.529042 10750 leader_election.cc:290] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
W20260709 13:04:03.507514 10103 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 85825.7us
I20260709 13:04:03.555131 10103 leader_election.cc:304] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.535713 10688 raft_consensus.cc:2749] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:03.535964 10775 consensus_queue.cc:1048] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:03.535760 10028 leader_election.cc:304] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.559010 10775 raft_consensus.cc:2749] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.559587 10028 leader_election.cc:304] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.560097 10762 raft_consensus.cc:2749] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.560575 10758 raft_consensus.cc:2749] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.515746 10754 leader_election.cc:290] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.539333 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.540654 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c9785f60ee640d686f599267cf005fb" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.561147 10028 leader_election.cc:304] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.563256 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13fc2f8cffc54c41b5bdde0305fe868c" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.564086 10771 raft_consensus.cc:2749] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.542222 10335 raft_consensus.cc:385] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:03.566195 10335 raft_consensus.cc:740] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:03.567092 10335 consensus_queue.cc:260] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.569046 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13fc2f8cffc54c41b5bdde0305fe868c" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.571666 10064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.572576 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f0709ff33d24996ae638187a8e7c57f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.574092 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf60e84958749f9826c9e8de7d266d9" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.578356 10747 leader_election.cc:290] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.584549 10335 ts_tablet_manager.cc:1434] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.089s	user 0.006s	sys 0.001s
I20260709 13:04:03.585736 10335 tablet_bootstrap.cc:492] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b: Bootstrap starting.
W20260709 13:04:03.587327 10105 leader_election.cc:343] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:03.589195 10103 leader_election.cc:304] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.591622 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5cf0e36da48d0946769d2bb27c981" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.594765 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.597653 10762 raft_consensus.cc:2749] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.599148 10103 leader_election.cc:304] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
W20260709 13:04:03.606914 10178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 77297.5us
I20260709 13:04:03.614444 10331 tablet_bootstrap.cc:654] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:03.618254 10252 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 156223us
I20260709 13:04:03.619046 10252 leader_election.cc:304] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.619817 10749 raft_consensus.cc:2804] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:03.622478 10749 raft_consensus.cc:493] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.622923 10749 raft_consensus.cc:3060] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.625485 10621 raft_consensus.cc:493] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.626267 10621 raft_consensus.cc:515] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.629115 10621 leader_election.cc:290] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.630129 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fa8e2d405c485dbf19dff403d2328f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
W20260709 13:04:03.631960 10028 leader_election.cc:343] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:03.634508 10338 tablet_bootstrap.cc:492] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
W20260709 13:04:03.634783 10251 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 140061us
I20260709 13:04:03.635665 10030 leader_election.cc:304] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.637212 10030 leader_election.cc:304] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.635145 10338 ts_tablet_manager.cc:1403] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.286s	user 0.013s	sys 0.005s
I20260709 13:04:03.639931 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b84c07107a48406d9fd585e9982d3069" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.641101 10769 raft_consensus.cc:2804] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:03.641503 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf60e84958749f9826c9e8de7d266d9" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.641578 10769 raft_consensus.cc:493] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.642912 10686 raft_consensus.cc:2749] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.644001 10769 raft_consensus.cc:3060] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.644487 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4d469562c14664860f49baa277f1d9" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.641307 10775 raft_consensus.cc:2749] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.648751 10621 consensus_queue.cc:1048] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.041s
I20260709 13:04:03.671283 10762 raft_consensus.cc:493] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:03.710028 10338 raft_consensus.cc:359] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:03.711010 10338 raft_consensus.cc:385] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:03.711274 10338 raft_consensus.cc:740] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:03.711963 10338 consensus_queue.cc:260] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:03.687103 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6314d5916384527a155e2c956145d98" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.694523 10178 leader_election.cc:304] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
W20260709 13:04:03.687812 10103 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 74862.4us
I20260709 13:04:03.716001 10251 leader_election.cc:304] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
W20260709 13:04:03.719936 10252 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 65648.3us
I20260709 13:04:03.726845 10338 ts_tablet_manager.cc:1434] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.001s
I20260709 13:04:03.736397 10753 raft_consensus.cc:2749] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.713126 10762 raft_consensus.cc:515] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.739954 10762 leader_election.cc:290] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.740559 10335 tablet_bootstrap.cc:654] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:03.741370 10134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fa8e2d405c485dbf19dff403d2328f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
W20260709 13:04:03.748247 10251 leader_election.cc:343] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:03.748639 10688 raft_consensus.cc:2749] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.750921 10135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6314d5916384527a155e2c956145d98" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:03.751215 10333 tablet_bootstrap.cc:492] T a8bc0e52b9424f6dab141d4c1e28023b P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:03.761750 10333 ts_tablet_manager.cc:1403] T a8bc0e52b9424f6dab141d4c1e28023b P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.738s	user 0.019s	sys 0.006s
I20260709 13:04:03.751636 10136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4d469562c14664860f49baa277f1d9" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:03.762349 10136 raft_consensus.cc:2468] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:04:03.748582 10134 raft_consensus.cc:2468] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:04:03.733036 10769 raft_consensus.cc:515] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.764644 10333 raft_consensus.cc:359] T a8bc0e52b9424f6dab141d4c1e28023b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.756894 10762 raft_consensus.cc:493] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:03.754127 10749 raft_consensus.cc:515] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.766024 10762 raft_consensus.cc:515] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:03.756405 10748 raft_consensus.cc:493] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:03.765470 10769 leader_election.cc:290] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
W20260709 13:04:03.767237 10252 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 45451.9us
I20260709 13:04:03.767256 10333 raft_consensus.cc:385] T a8bc0e52b9424f6dab141d4c1e28023b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:03.755609 10103 leader_election.cc:304] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.771792 10103 leader_election.cc:304] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.778949 10765 raft_consensus.cc:2749] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.780295 10060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fa8e2d405c485dbf19dff403d2328f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.776777 10754 raft_consensus.cc:2749] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.781070 10060 raft_consensus.cc:2468] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 0.
I20260709 13:04:03.782612 10104 leader_election.cc:304] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:03.783836 10754 raft_consensus.cc:2804] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:03.784318 10754 raft_consensus.cc:493] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.784658 10754 raft_consensus.cc:3060] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.773084 10748 raft_consensus.cc:515] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:03.798749 10754 raft_consensus.cc:515] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.802203 10060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "220a3db850dd4a5280bf7c8e9519d9d2" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.807580 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fa8e2d405c485dbf19dff403d2328f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:03.805478 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:03.808624 10213 raft_consensus.cc:2393] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in current term 1: Already voted for candidate bcf45a2176234db5957dc38e32554577 in this term.
I20260709 13:04:03.812635 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "220a3db850dd4a5280bf7c8e9519d9d2" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
W20260709 13:04:03.812825 10103 leader_election.cc:343] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:03.834343 10030 leader_election.cc:304] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:03.835669 10030 leader_election.cc:304] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:03.813462 10060 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fa8e2d405c485dbf19dff403d2328f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:04:03.818504 10754 leader_election.cc:290] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.819411 10762 leader_election.cc:290] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:03.820179 10331 tablet_bootstrap.cc:492] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:04:03.820627 10065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:03.821704 10749 leader_election.cc:290] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:03.770498 10333 raft_consensus.cc:740] T a8bc0e52b9424f6dab141d4c1e28023b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
W20260709 13:04:03.826994 10252 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 56499.1us
I20260709 13:04:03.843887 10252 leader_election.cc:304] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.801015 10748 leader_election.cc:290] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.838598 10030 leader_election.cc:304] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:03.846308 10333 consensus_queue.cc:260] T a8bc0e52b9424f6dab141d4c1e28023b P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.847376 10030 leader_election.cc:304] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:03.840306 10331 ts_tablet_manager.cc:1403] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.353s	user 0.014s	sys 0.009s
I20260709 13:04:03.845659  9963 catalog_manager.cc:5697] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } }
I20260709 13:04:03.859686 10030 leader_election.cc:304] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:03.860234 10333 ts_tablet_manager.cc:1434] T a8bc0e52b9424f6dab141d4c1e28023b P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.098s	user 0.007s	sys 0.001s
I20260709 13:04:03.844043 10769 raft_consensus.cc:2749] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.844954 10060 raft_consensus.cc:3060] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.845146 10771 raft_consensus.cc:2749] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.865831 10338 tablet_bootstrap.cc:492] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:03.851028 10678 raft_consensus.cc:2749] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.852809 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:03.873900 10283 raft_consensus.cc:2393] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in current term 1: Already voted for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in this term.
I20260709 13:04:03.875345 10621 consensus_queue.cc:1048] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:03.882030 10253 leader_election.cc:304] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:03.858700 10759 raft_consensus.cc:2749] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.889592 10253 leader_election.cc:304] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:03.868944 10760 raft_consensus.cc:2749] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.897502 10253 leader_election.cc:304] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:04:03.872495 10749 raft_consensus.cc:2749] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:03.867760 10178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 188750us
I20260709 13:04:03.909873 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96acb559acc745428add2c682cf83cfc" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.905027 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63e94ff7589e401ba16e6019e0a4d6eb" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:03.910876 10288 raft_consensus.cc:3060] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.920571 10746 raft_consensus.cc:2804] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:03.921178 10746 raft_consensus.cc:493] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:03.921567 10746 raft_consensus.cc:3060] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.928890 10288 raft_consensus.cc:2468] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:04:03.932137 10786 raft_consensus.cc:2749] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.932299 10686 consensus_queue.cc:1048] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:03.933036 10794 raft_consensus.cc:2749] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.901295 10103 leader_election.cc:304] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
W20260709 13:04:03.935575 10103 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 48138.6us
I20260709 13:04:03.936988 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:03.938176 10288 raft_consensus.cc:2393] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bcf45a2176234db5957dc38e32554577 in current term 1: Already voted for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in this term.
I20260709 13:04:03.942782 10750 raft_consensus.cc:2749] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:03.957050 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fa8e2d405c485dbf19dff403d2328f" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:03.948593 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5cf0e36da48d0946769d2bb27c981" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.957535 10060 raft_consensus.cc:2468] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:04:03.967276 10104 leader_election.cc:304] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:03.969795 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:03.970165 10748 raft_consensus.cc:2804] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:03.970717 10748 raft_consensus.cc:697] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:03.971590 10748 consensus_queue.cc:237] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:03.976598 10338 tablet_bootstrap.cc:654] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:03.981197 10214 raft_consensus.cc:3060] T a8bc0e52b9424f6dab141d4c1e28023b P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:03.968874 10331 raft_consensus.cc:359] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.946856 10335 tablet_bootstrap.cc:492] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b: No bootstrap required, opened a new log
I20260709 13:04:03.989151 10746 raft_consensus.cc:515] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:03.992544 10331 raft_consensus.cc:385] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:03.998747 10333 tablet_bootstrap.cc:492] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:04:04.006798 10331 raft_consensus.cc:740] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:04.024809 10331 consensus_queue.cc:260] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:03.997920 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
W20260709 13:04:04.028237 10105 leader_election.cc:343] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:04.028451 10210 raft_consensus.cc:2393] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in current term 1: Already voted for candidate bcf45a2176234db5957dc38e32554577 in this term.
I20260709 13:04:04.029165 10105 leader_election.cc:304] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:04.030488 10805 raft_consensus.cc:2749] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:04.002735 10335 ts_tablet_manager.cc:1403] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b: Time spent bootstrapping tablet: real 0.417s	user 0.039s	sys 0.017s
W20260709 13:04:04.000957 10103 leader_election.cc:343] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:04.035822 10105 leader_election.cc:304] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:04:04.036793 10805 raft_consensus.cc:2749] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:04.036936 10030 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:33711, user_credentials={real_user=slave}} blocked reactor thread for 170192us
I20260709 13:04:04.039682 10030 leader_election.cc:304] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:04:04.040589 10621 raft_consensus.cc:2764] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:04:04.050239 10331 ts_tablet_manager.cc:1434] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.202s	user 0.005s	sys 0.002s
I20260709 13:04:04.023945 10134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4d469562c14664860f49baa277f1d9" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:04.051640 10134 raft_consensus.cc:3060] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:04.053031 10746 leader_election.cc:290] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:03.993008  9959 catalog_manager.cc:5697] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:04.002224 10801 log.cc:927] Time spent T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 13:04:04.060734 10030 leader_election.cc:304] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:04:04.063731 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4d469562c14664860f49baa277f1d9" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
W20260709 13:04:04.065749 10251 leader_election.cc:343] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:04.066952 10331 tablet_bootstrap.cc:492] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:04.072669 10333 tablet_bootstrap.cc:654] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:04.074159  9959 catalog_manager.cc:5697] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:04.074183 10807 raft_consensus.cc:2749] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:04.043746 10105 leader_election.cc:304] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:04.076020 10030 leader_election.cc:304] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:04:04.085812 10807 raft_consensus.cc:2749] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:04.099507 10134 raft_consensus.cc:2468] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:04:04.109027 10335 raft_consensus.cc:359] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:04.109906 10335 raft_consensus.cc:385] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:04.110180 10335 raft_consensus.cc:740] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Initialized, Role: FOLLOWER
I20260709 13:04:04.110883 10335 consensus_queue.cc:260] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:04.112643 10253 leader_election.cc:304] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: bcf45a2176234db5957dc38e32554577
I20260709 13:04:04.114377 10803 raft_consensus.cc:2804] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:04.114439 10765 consensus_queue.cc:1048] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:04.114979 10803 raft_consensus.cc:697] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:04.123363 10335 ts_tablet_manager.cc:1434] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b: Time spent starting tablet: real 0.089s	user 0.007s	sys 0.000s
I20260709 13:04:04.125941 10803 consensus_queue.cc:237] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:04.126298 10754 consensus_queue.cc:1048] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:04.146759 10058 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
W20260709 13:04:04.147245 10309 debug-util.cc:398] Leaking SignalData structure 0x7b08002eb6c0 after lost signal to thread 434
I20260709 13:04:04.147745 10058 raft_consensus.cc:2393] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in current term 1: Already voted for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in this term.
W20260709 13:04:04.148589 10309 debug-util.cc:398] Leaking SignalData structure 0x7b08000aaee0 after lost signal to thread 9997
I20260709 13:04:04.150910 10768 consensus_queue.cc:1048] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 13:04:04.152156 10061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:04:04.153100 10061 raft_consensus.cc:2393] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bcf45a2176234db5957dc38e32554577 in current term 1: Already voted for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in this term.
W20260709 13:04:04.180168 10309 debug-util.cc:398] Leaking SignalData structure 0x7b08000cde40 after lost signal to thread 10089
W20260709 13:04:04.192216 10309 debug-util.cc:398] Leaking SignalData structure 0x7b0800320aa0 after lost signal to thread 10164
W20260709 13:04:04.194067 10309 debug-util.cc:398] Leaking SignalData structure 0x7b080032b240 after lost signal to thread 10238
I20260709 13:04:04.204543 10331 tablet_bootstrap.cc:654] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:04.214855 10309 debug-util.cc:398] Leaking SignalData structure 0x7b0800250ca0 after lost signal to thread 10312
I20260709 13:04:04.222220 10338 tablet_bootstrap.cc:492] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:04.224268 10338 ts_tablet_manager.cc:1403] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.365s	user 0.031s	sys 0.010s
I20260709 13:04:04.230602 10252 leader_election.cc:304] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:04.222586 10805 raft_consensus.cc:2749] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:04.237200 10782 raft_consensus.cc:2749] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:04.287397 10333 tablet_bootstrap.cc:492] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:04.287988 10333 ts_tablet_manager.cc:1403] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.289s	user 0.011s	sys 0.019s
I20260709 13:04:04.300110 10333 raft_consensus.cc:359] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:04.300876 10333 raft_consensus.cc:385] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:04.301170 10333 raft_consensus.cc:740] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:04:04.266654 10178 leader_election.cc:304] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
W20260709 13:04:04.323264 10178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 63473.5us
I20260709 13:04:04.324095 10766 raft_consensus.cc:2749] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:04.313031 10338 raft_consensus.cc:359] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:04.345973 10338 raft_consensus.cc:385] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:04.346515 10338 raft_consensus.cc:740] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:04.347347 10338 consensus_queue.cc:260] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:04.327004 10333 consensus_queue.cc:260] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:04.362972 10333 ts_tablet_manager.cc:1434] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.075s	user 0.002s	sys 0.005s
I20260709 13:04:04.364038 10333 tablet_bootstrap.cc:492] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:04:04.364151 10338 ts_tablet_manager.cc:1434] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.139s	user 0.006s	sys 0.000s
I20260709 13:04:04.367363 10803 raft_consensus.cc:493] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.369812 10803 raft_consensus.cc:515] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:04.371022 10338 tablet_bootstrap.cc:492] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:04.372434 10803 leader_election.cc:290] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:04.368983 10760 raft_consensus.cc:493] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.374199 10760 raft_consensus.cc:515] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:04.374313 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d87b05b458741adb45ae4a1753447d0" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:04.376544 10760 leader_election.cc:290] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:04.377220 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa60f11d5c674f3180f2616f97adf40f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
W20260709 13:04:04.379097 10028 leader_election.cc:343] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:04.380959 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa60f11d5c674f3180f2616f97adf40f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:04:04.382349 10251 leader_election.cc:343] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:04.383074 10063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d87b05b458741adb45ae4a1753447d0" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
W20260709 13:04:04.384559 10030 leader_election.cc:343] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:04.390388 10252 leader_election.cc:343] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:04.391000 10331 tablet_bootstrap.cc:492] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:04:04.391582 10331 ts_tablet_manager.cc:1403] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.325s	user 0.024s	sys 0.064s
I20260709 13:04:04.391026 10252 leader_election.cc:304] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:04.408294 10338 tablet_bootstrap.cc:654] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:04.384987 10030 leader_election.cc:304] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:04:04.396381 10331 raft_consensus.cc:359] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:04.409282 10331 raft_consensus.cc:385] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:04.410039 10331 raft_consensus.cc:740] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:04.409474 10761 raft_consensus.cc:2749] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:04.411530 10331 consensus_queue.cc:260] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:04.414826 10795 raft_consensus.cc:2749] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:04.424165 10331 ts_tablet_manager.cc:1434] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 13:04:04.425869 10331 tablet_bootstrap.cc:492] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:04.427322 10333 tablet_bootstrap.cc:654] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:04.478111 10768 raft_consensus.cc:493] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.494670 10768 raft_consensus.cc:515] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:04.499006 10331 tablet_bootstrap.cc:654] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:04.514149 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88ba89b2d1849848da78b9b3747e66e" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:04.511937 10807 raft_consensus.cc:493] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:04.515314 10807 raft_consensus.cc:515] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:04.517442 10807 leader_election.cc:290] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
W20260709 13:04:04.520466 10105 leader_election.cc:343] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:04.520748 10805 consensus_queue.cc:1048] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:04.524036 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88ba89b2d1849848da78b9b3747e66e" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:04.527356 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96acb559acc745428add2c682cf83cfc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:04.528137 10137 raft_consensus.cc:2468] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 0.
I20260709 13:04:04.530580 10030 leader_election.cc:304] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:04.510583 10768 leader_election.cc:290] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
W20260709 13:04:04.538434 10103 leader_election.cc:343] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:04.538959 10103 leader_election.cc:304] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:04:04.539474 10759 raft_consensus.cc:2804] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:04.539919 10759 raft_consensus.cc:493] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:04.540264 10759 raft_consensus.cc:3060] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:04.540606 10103 consensus_peers.cc:597] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:04.540787 10802 raft_consensus.cc:2749] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:04.538975 10282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96acb559acc745428add2c682cf83cfc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:04:04.557953 10030 leader_election.cc:343] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:04.603929 10759 raft_consensus.cc:515] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:04.627388 10338 tablet_bootstrap.cc:492] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:04.627974 10338 ts_tablet_manager.cc:1403] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.257s	user 0.030s	sys 0.025s
I20260709 13:04:04.628957 10803 consensus_queue.cc:1048] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:04:04.633267  9961 catalog_manager.cc:5697] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:04.644088 10282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96acb559acc745428add2c682cf83cfc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:04.647480 10759 leader_election.cc:290] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:04.650717 10331 tablet_bootstrap.cc:492] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331: No bootstrap required, opened a new log
I20260709 13:04:04.651298 10331 ts_tablet_manager.cc:1403] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.226s	user 0.015s	sys 0.011s
I20260709 13:04:04.653309 10338 raft_consensus.cc:359] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:04.658134 10331 raft_consensus.cc:359] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:04.671151 10331 raft_consensus.cc:385] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:04.658562 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96acb559acc745428add2c682cf83cfc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:04.671411 10331 raft_consensus.cc:740] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:04.668555 10763 raft_consensus.cc:493] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:04:04.670502 10030 leader_election.cc:343] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:04.671795 10338 raft_consensus.cc:385] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:04.672976 10331 consensus_queue.cc:260] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:04.675675 10137 raft_consensus.cc:3060] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:04.677366 10331 ts_tablet_manager.cc:1434] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 13:04:04.675683 10338 raft_consensus.cc:740] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:04.683701  9962 catalog_manager.cc:5697] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 1 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:04.689857 10338 consensus_queue.cc:260] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:04.682142 10763 raft_consensus.cc:515] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
W20260709 13:04:04.700107 10011 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:04.703136 10136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4d469562c14664860f49baa277f1d9" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:04.704188 10211 raft_consensus.cc:3060] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:04.710601 10338 ts_tablet_manager.cc:1434] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.000s
I20260709 13:04:04.711849 10338 tablet_bootstrap.cc:492] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:04.701066 10763 leader_election.cc:290] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:04.733994 10333 tablet_bootstrap.cc:492] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:04.734594 10333 ts_tablet_manager.cc:1403] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.371s	user 0.009s	sys 0.019s
I20260709 13:04:04.737886 10749 consensus_queue.cc:1048] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:04.738793 10338 tablet_bootstrap.cc:654] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:04.757721 10333 raft_consensus.cc:359] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:04.758800 10333 raft_consensus.cc:385] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:04.759172 10333 raft_consensus.cc:740] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:04:04.755378 10282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e4d469562c14664860f49baa277f1d9" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:04.770582 10333 consensus_queue.cc:260] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:04.793557 10179 leader_election.cc:304] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:04.795405 10753 raft_consensus.cc:2749] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:04.800484 10137 raft_consensus.cc:2468] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:04:04.806115 10333 ts_tablet_manager.cc:1434] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.001s
I20260709 13:04:04.811678 10333 tablet_bootstrap.cc:492] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:04:04.815428 10030 leader_election.cc:304] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:04.831183 10807 raft_consensus.cc:2804] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:04.831816 10807 raft_consensus.cc:697] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:04.832791 10807 consensus_queue.cc:237] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:04.855172 10338 tablet_bootstrap.cc:492] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:04.855854 10338 ts_tablet_manager.cc:1403] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.144s	user 0.027s	sys 0.028s
I20260709 13:04:04.859090 10338 raft_consensus.cc:359] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:04.860023 10338 raft_consensus.cc:385] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:04.860424 10338 raft_consensus.cc:740] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:04.861711 10338 consensus_queue.cc:260] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:04.869453 10338 ts_tablet_manager.cc:1434] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:04:04.880299 10338 tablet_bootstrap.cc:492] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0: Bootstrap starting.
I20260709 13:04:04.898910 10333 tablet_bootstrap.cc:654] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:04.921212 10338 tablet_bootstrap.cc:654] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:04.923990 10333 tablet_bootstrap.cc:492] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:04.924696 10333 ts_tablet_manager.cc:1403] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.113s	user 0.014s	sys 0.000s
I20260709 13:04:04.928136 10333 raft_consensus.cc:359] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:04.932741 10333 raft_consensus.cc:385] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:04.933203 10333 raft_consensus.cc:740] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:04:04.934165 10333 consensus_queue.cc:260] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:04.936688 10333 ts_tablet_manager.cc:1434] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.002s
I20260709 13:04:04.941072 10333 tablet_bootstrap.cc:492] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:04:04.948609 10333 tablet_bootstrap.cc:654] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:04.959646 10338 tablet_bootstrap.cc:492] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0: No bootstrap required, opened a new log
I20260709 13:04:04.960217 10338 ts_tablet_manager.cc:1403] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent bootstrapping tablet: real 0.080s	user 0.015s	sys 0.001s
I20260709 13:04:04.963606 10338 raft_consensus.cc:359] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:04.964447 10338 raft_consensus.cc:385] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:04.964787 10338 raft_consensus.cc:740] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Initialized, Role: FOLLOWER
I20260709 13:04:04.965852 10338 consensus_queue.cc:260] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:04.968555 10338 ts_tablet_manager.cc:1434] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:04:04.976892 10333 tablet_bootstrap.cc:492] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:04.977681 10333 ts_tablet_manager.cc:1403] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.009s
I20260709 13:04:04.980785 10333 raft_consensus.cc:359] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:04.981653 10333 raft_consensus.cc:385] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:04.981985 10333 raft_consensus.cc:740] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:04:04.982738 10333 consensus_queue.cc:260] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:04.984337 10333 ts_tablet_manager.cc:1434] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.002s
I20260709 13:04:04.985301 10333 tablet_bootstrap.cc:492] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577: Bootstrap starting.
I20260709 13:04:04.991169 10333 tablet_bootstrap.cc:654] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:04.982160  9959 catalog_manager.cc:5697] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:05.002534 10763 raft_consensus.cc:493] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.003239 10763 raft_consensus.cc:515] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:05.006186 10763 leader_election.cc:290] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:05.007658 10061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d87b05b458741adb45ae4a1753447d0" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:05.008924 10061 raft_consensus.cc:2468] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:04:05.010540 10178 leader_election.cc:304] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:05.012465 10785 raft_consensus.cc:2804] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:05.012969 10785 raft_consensus.cc:493] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.013386 10785 raft_consensus.cc:3060] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:05.021245 10282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d87b05b458741adb45ae4a1753447d0" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:05.022078 10282 raft_consensus.cc:2468] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:04:05.032730 10333 tablet_bootstrap.cc:492] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577: No bootstrap required, opened a new log
I20260709 13:04:05.033319 10333 ts_tablet_manager.cc:1403] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.014s
I20260709 13:04:05.036293 10333 raft_consensus.cc:359] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:05.036955 10333 raft_consensus.cc:385] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:05.037220 10333 raft_consensus.cc:740] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Initialized, Role: FOLLOWER
I20260709 13:04:05.037933 10333 consensus_queue.cc:260] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:05.040333 10333 ts_tablet_manager.cc:1434] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:04:05.050256 10785 raft_consensus.cc:515] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:05.052938 10282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d87b05b458741adb45ae4a1753447d0" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:05.053663 10282 raft_consensus.cc:3060] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:05.054446 10061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d87b05b458741adb45ae4a1753447d0" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:04:05.055058 10061 raft_consensus.cc:3060] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:05.066882 10282 raft_consensus.cc:2468] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:04:05.069746 10179 leader_election.cc:304] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:05.076279 10763 raft_consensus.cc:2804] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:05.077015 10763 raft_consensus.cc:697] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:04:05.078045 10763 consensus_queue.cc:237] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:05.088068 10061 raft_consensus.cc:2468] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:04:05.089380 10785 leader_election.cc:290] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:05.107407  9962 catalog_manager.cc:5697] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:05.122685 10763 raft_consensus.cc:493] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.123315 10763 raft_consensus.cc:515] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:05.125537 10763 leader_election.cc:290] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:05.126554 10061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fa8e2d405c485dbf19dff403d2328f" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:05.128271 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60fa8e2d405c485dbf19dff403d2328f" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:05.133939 10179 leader_election.cc:304] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:05.134745 10763 raft_consensus.cc:2749] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:05.152519 10803 raft_consensus.cc:493] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.153241 10803 raft_consensus.cc:515] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:05.155839 10803 leader_election.cc:290] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:05.156503 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa60f11d5c674f3180f2616f97adf40f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:05.156950 10061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa60f11d5c674f3180f2616f97adf40f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:05.157159 10214 raft_consensus.cc:2468] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:04:05.157724 10061 raft_consensus.cc:2468] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 0.
I20260709 13:04:05.158763 10251 leader_election.cc:304] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:05.160629 10803 raft_consensus.cc:2804] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:05.161314 10803 raft_consensus.cc:493] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.161857 10803 raft_consensus.cc:3060] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:05.172227 10803 raft_consensus.cc:515] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:05.175745 10803 leader_election.cc:290] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:05.175967 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa60f11d5c674f3180f2616f97adf40f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:05.176743 10214 raft_consensus.cc:3060] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:05.178356 10061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa60f11d5c674f3180f2616f97adf40f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
I20260709 13:04:05.182365 10061 raft_consensus.cc:3060] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:05.189127 10214 raft_consensus.cc:2468] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:04:05.190553 10251 leader_election.cc:304] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:05.191375 10803 raft_consensus.cc:2804] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:05.192762 10803 raft_consensus.cc:697] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:05.193714 10803 consensus_queue.cc:237] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:05.194689 10061 raft_consensus.cc:2468] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:04:05.207187  9962 catalog_manager.cc:5697] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 1 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:05.223294 10785 raft_consensus.cc:493] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.223979 10785 raft_consensus.cc:515] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:05.226455 10785 leader_election.cc:290] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:05.228320 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88ba89b2d1849848da78b9b3747e66e" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:05.229581 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88ba89b2d1849848da78b9b3747e66e" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:05.231037 10286 raft_consensus.cc:2468] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:04:05.230516 10131 raft_consensus.cc:2468] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 0.
I20260709 13:04:05.232332 10179 leader_election.cc:304] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:05.233165 10785 raft_consensus.cc:2804] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:05.233605 10785 raft_consensus.cc:493] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.233968 10785 raft_consensus.cc:3060] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:05.244172 10803 raft_consensus.cc:493] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.243920 10785 raft_consensus.cc:515] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:05.244796 10803 raft_consensus.cc:515] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:05.246704 10785 leader_election.cc:290] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:05.247457 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88ba89b2d1849848da78b9b3747e66e" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:05.248140 10131 raft_consensus.cc:3060] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:05.248235 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b88ba89b2d1849848da78b9b3747e66e" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:05.248939 10286 raft_consensus.cc:3060] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:05.250450 10803 leader_election.cc:290] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:05.250659 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96acb559acc745428add2c682cf83cfc" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:05.251412 10137 raft_consensus.cc:2393] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in current term 1: Already voted for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in this term.
I20260709 13:04:05.258096 10286 raft_consensus.cc:2468] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:04:05.258394 10131 raft_consensus.cc:2468] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:04:05.259783 10179 leader_election.cc:304] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:05.260607 10785 raft_consensus.cc:2804] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:05.261164 10785 raft_consensus.cc:697] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:04:05.262291 10061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96acb559acc745428add2c682cf83cfc" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" is_pre_election: true
I20260709 13:04:05.262112 10785 consensus_queue.cc:237] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:05.263985 10252 leader_election.cc:304] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:05.264962 10803 raft_consensus.cc:3060] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:05.284307  9962 catalog_manager.cc:5697] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 0 to 1, leader changed from <none> to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:05.294564 10803 raft_consensus.cc:2749] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:05.296514 10214 raft_consensus.cc:3060] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:05.333348 10771 consensus_queue.cc:1048] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:05.351614 10771 consensus_queue.cc:1048] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:05.486887 10785 consensus_queue.cc:1048] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:05.500708 10785 consensus_queue.cc:1048] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:05.619205 10749 consensus_queue.cc:1048] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:05.631531 10749 consensus_queue.cc:1048] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:04:05.732378 10011 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:05.802690 10796 consensus_queue.cc:1048] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:05.824716 10753 consensus_queue.cc:1048] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:05.918066 10844 raft_consensus.cc:493] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:05.918605 10844 raft_consensus.cc:515] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:05.920739 10844 leader_election.cc:290] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:05.921530 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:05.922163 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:05.922435 10211 raft_consensus.cc:2468] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:04:05.922834 10286 raft_consensus.cc:2468] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 1.
I20260709 13:04:05.923897 10030 leader_election.cc:304] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 
I20260709 13:04:05.924880 10844 raft_consensus.cc:2804] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:05.925314 10844 raft_consensus.cc:493] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:05.925736 10844 raft_consensus.cc:3060] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:05.938694 10844 raft_consensus.cc:515] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:05.940567 10844 leader_election.cc:290] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:05.941257 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:05.941922 10211 raft_consensus.cc:3060] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:05.942134 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:05.942708 10286 raft_consensus.cc:3060] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:05.952647 10286 raft_consensus.cc:2468] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:05.953909 10030 leader_election.cc:304] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 
I20260709 13:04:05.954972 10844 raft_consensus.cc:2804] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:05.955466 10844 raft_consensus.cc:697] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:05.956316 10844 consensus_queue.cc:237] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:05.969933 10211 raft_consensus.cc:2468] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:05.990665  9962 catalog_manager.cc:5697] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 0 to 2, leader changed from <none> to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:06.147840   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:04:06.169546 10178 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:04:06.184389 10178 consensus_peers.cc:597] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.200924 10104 consensus_peers.cc:597] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.203572 10252 consensus_peers.cc:597] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.205750 10178 consensus_peers.cc:597] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.220947   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:06.222322   431 tablet_replica.cc:333] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.223389   431 raft_consensus.cc:2243] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.224004   431 raft_consensus.cc:2272] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.227731   431 tablet_replica.cc:333] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.228681   431 raft_consensus.cc:2243] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.229246   431 raft_consensus.cc:2272] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.232000   431 tablet_replica.cc:333] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.232775   431 raft_consensus.cc:2243] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.233245   431 raft_consensus.cc:2272] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.235697   431 tablet_replica.cc:333] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.236497   431 raft_consensus.cc:2243] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.237027   431 raft_consensus.cc:2272] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.239825   431 tablet_replica.cc:333] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.240702   431 raft_consensus.cc:2243] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.241716   431 raft_consensus.cc:2272] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.244120   431 tablet_replica.cc:333] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.244920   431 raft_consensus.cc:2243] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.245888   431 raft_consensus.cc:2272] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.248173   431 tablet_replica.cc:333] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.249001   431 raft_consensus.cc:2243] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.249577   431 raft_consensus.cc:2272] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.252472   431 tablet_replica.cc:333] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.253281   431 raft_consensus.cc:2243] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.254289   431 raft_consensus.cc:2272] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.256794   431 tablet_replica.cc:333] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
W20260709 13:04:06.258965 10252 consensus_peers.cc:597] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.262562   431 raft_consensus.cc:2243] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.264011   431 raft_consensus.cc:2272] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.268389 10104 consensus_peers.cc:597] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.269208   431 tablet_replica.cc:333] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.270964   431 raft_consensus.cc:2243] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.271493   431 raft_consensus.cc:2272] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.274120   431 tablet_replica.cc:333] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.275008   431 raft_consensus.cc:2243] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.275568   431 raft_consensus.cc:2272] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.278439   431 tablet_replica.cc:333] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.279383   431 raft_consensus.cc:2243] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.279933   431 raft_consensus.cc:2272] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.280484 10104 consensus_peers.cc:597] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.282543   431 tablet_replica.cc:333] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.283609   431 raft_consensus.cc:2243] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:06.283841 10104 consensus_peers.cc:597] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.285916 10252 consensus_peers.cc:597] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.286329   431 raft_consensus.cc:2272] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.289569   431 tablet_replica.cc:333] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.290491   431 raft_consensus.cc:2243] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.291049   431 raft_consensus.cc:2272] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.293614   431 tablet_replica.cc:333] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.294469   431 raft_consensus.cc:2243] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.294989   431 raft_consensus.cc:2272] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.297583   431 tablet_replica.cc:333] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.298451   431 raft_consensus.cc:2243] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.299016   431 raft_consensus.cc:2272] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.302053   431 tablet_replica.cc:333] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.302909   431 raft_consensus.cc:2243] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.303465   431 raft_consensus.cc:2272] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.306329   431 tablet_replica.cc:333] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.307248   431 raft_consensus.cc:2243] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.308303   431 raft_consensus.cc:2272] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.311153   431 tablet_replica.cc:333] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.312119   431 raft_consensus.cc:2243] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.313318   431 raft_consensus.cc:2272] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.316148   431 tablet_replica.cc:333] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.317020   431 raft_consensus.cc:2243] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.317628   431 raft_consensus.cc:2272] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.320524   431 tablet_replica.cc:333] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.321461   431 raft_consensus.cc:2243] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.322069   431 raft_consensus.cc:2272] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.324748   431 tablet_replica.cc:333] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.325611   431 raft_consensus.cc:2243] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.326676   431 raft_consensus.cc:2272] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.329699   431 tablet_replica.cc:333] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.330636   431 raft_consensus.cc:2243] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.331748   431 raft_consensus.cc:2272] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.334620   431 tablet_replica.cc:333] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.335484   431 raft_consensus.cc:2243] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.336514   431 raft_consensus.cc:2272] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.339469   431 tablet_replica.cc:333] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.340319   431 raft_consensus.cc:2243] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.340907   431 raft_consensus.cc:2272] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.343731   431 tablet_replica.cc:333] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.344575   431 raft_consensus.cc:2243] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.345110   431 raft_consensus.cc:2272] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.347738   431 tablet_replica.cc:333] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.348572   431 raft_consensus.cc:2243] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.349212   431 raft_consensus.cc:2272] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.352152   431 tablet_replica.cc:333] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.352998   431 raft_consensus.cc:2243] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.353608   431 raft_consensus.cc:2272] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.354678 10252 consensus_peers.cc:597] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.356326   431 tablet_replica.cc:333] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.357326   431 raft_consensus.cc:2243] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.357928   431 raft_consensus.cc:2272] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.361243   431 tablet_replica.cc:333] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.362151   431 raft_consensus.cc:2243] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.363065   431 pending_rounds.cc:70] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: Trying to abort 1 pending ops.
I20260709 13:04:06.363324   431 pending_rounds.cc:77] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: Aborting op as it isn't in flight: id { term: 2 index: 1 } timestamp: 7305634799465271296 op_type: NO_OP noop_request { }
I20260709 13:04:06.363711   431 raft_consensus.cc:2889] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:06.364075   431 raft_consensus.cc:2272] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.367177   431 tablet_replica.cc:333] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.368206   431 raft_consensus.cc:2243] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.369400   431 raft_consensus.cc:2272] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.372450   431 tablet_replica.cc:333] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.373394   431 raft_consensus.cc:2243] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.374014   431 raft_consensus.cc:2272] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.375439 10104 consensus_peers.cc:597] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.375721 10252 consensus_peers.cc:597] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.376897   431 tablet_replica.cc:333] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.377976   431 raft_consensus.cc:2243] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.378563   431 raft_consensus.cc:2272] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.381405   431 tablet_replica.cc:333] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.382264   431 raft_consensus.cc:2243] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.383232   431 raft_consensus.cc:2272] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.386044   431 tablet_replica.cc:333] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.386947   431 raft_consensus.cc:2243] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:06.385731 10104 consensus_peers.cc:597] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.387547   431 raft_consensus.cc:2272] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.389647 10178 consensus_peers.cc:597] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.390743   431 tablet_replica.cc:333] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
W20260709 13:04:06.391579 10252 consensus_peers.cc:597] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.391731   431 raft_consensus.cc:2243] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.392753   431 raft_consensus.cc:2272] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.395216   431 tablet_replica.cc:333] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.395951   431 raft_consensus.cc:2243] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.396482   431 raft_consensus.cc:2272] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.399108   431 tablet_replica.cc:333] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.399801   431 raft_consensus.cc:2243] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.400317   431 raft_consensus.cc:2272] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.402885   431 tablet_replica.cc:333] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.403596   431 raft_consensus.cc:2243] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.404592   431 raft_consensus.cc:2272] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.407431   431 tablet_replica.cc:333] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.408147   431 raft_consensus.cc:2243] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.408624   431 raft_consensus.cc:2272] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.411204   431 tablet_replica.cc:333] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.411937   431 raft_consensus.cc:2243] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.412474   431 raft_consensus.cc:2272] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.413206 10178 consensus_peers.cc:597] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.415045   431 tablet_replica.cc:333] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.415875   431 raft_consensus.cc:2243] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:06.416781   431 raft_consensus.cc:2272] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.419471   431 tablet_replica.cc:333] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.420222   431 raft_consensus.cc:2243] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.420749   431 raft_consensus.cc:2272] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.423043   431 tablet_replica.cc:333] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.423576   431 raft_consensus.cc:2243] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.424023   431 raft_consensus.cc:2272] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:06.426064   431 tablet_replica.cc:333] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331: stopping tablet replica
I20260709 13:04:06.426668   431 raft_consensus.cc:2243] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:06.427170   431 raft_consensus.cc:2272] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:06.436371 10252 consensus_peers.cc:597] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.452965 10178 consensus_peers.cc:597] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.469630 10178 consensus_peers.cc:597] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.480593 10178 consensus_peers.cc:597] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.487900 10252 consensus_peers.cc:597] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.499147   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:04:06.517604 10178 consensus_peers.cc:597] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.535499 10104 consensus_peers.cc:597] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.549300 10178 consensus_peers.cc:597] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.560539 10104 consensus_peers.cc:597] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.578399 10252 consensus_peers.cc:597] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.615284 10178 consensus_peers.cc:597] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.619081   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:06.619917 10252 consensus_peers.cc:597] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.636202 10104 consensus_peers.cc:597] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:06.640026 10866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:06.641616 10867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:06.655236 10869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:06.656349   431 server_base.cc:1061] running on GCE node
I20260709 13:04:06.658761   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:06.659076   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:06.659314   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602246659294 us; error 0 us; skew 500 ppm
I20260709 13:04:06.660178   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:06.666541   431 webserver.cc:533] Webserver started at http://127.0.107.193:35629/ using document root <none> and password file <none>
I20260709 13:04:06.667414   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:06.667702   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:04:06.675352   431 fs_manager.cc:714] Time spent opening directory manager: real 0.006s	user 0.001s	sys 0.007s
W20260709 13:04:06.684907 10178 consensus_peers.cc:597] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:06.686779 10874 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:06.687857   431 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:04:06.688236   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-0-root
uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
format_stamp: "Formatted at 2026-07-09 13:03:52 on dist-test-slave-jk05"
I20260709 13:04:06.688795   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783601935286233-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:06.723886   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:06.725832   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:06.728022   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:06.750264 10881 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 13:04:07.260469 10805 raft_consensus.cc:493] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.261220 10805 raft_consensus.cc:515] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:07.263684 10805 leader_election.cc:290] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:07.266162 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:07.269163   431 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:04:07.269596   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.536s	user 0.017s	sys 0.007s
I20260709 13:04:07.269965   431 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:04:07.281857 10881 tablet_bootstrap.cc:492] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:07.283484 10885 raft_consensus.cc:493] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.284221 10885 raft_consensus.cc:515] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:07.287946 10885 leader_election.cc:290] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:07.290370 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0b03477ff4d1ab861dc093edbe68a" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
W20260709 13:04:07.299083 10104 leader_election.cc:336] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:07.299701 10104 leader_election.cc:304] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
W20260709 13:04:07.302534 10178 leader_election.cc:336] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:07.303185 10178 leader_election.cc:304] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:07.304777 10887 raft_consensus.cc:2749] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:07.316120 10768 raft_consensus.cc:2749] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:07.377969 10864 raft_consensus.cc:493] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.378611 10864 raft_consensus.cc:515] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:07.387151 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb355c77dc43e78812c2b2a2df92c9" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:07.396436 10882 raft_consensus.cc:493] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.397190 10882 raft_consensus.cc:515] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:07.400164 10882 leader_election.cc:290] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:07.402323 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96acb559acc745428add2c682cf83cfc" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
W20260709 13:04:07.452334 10104 leader_election.cc:336] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:07.452867 10104 leader_election.cc:304] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:07.455386 10768 raft_consensus.cc:2749] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:07.462356 10864 leader_election.cc:290] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:07.476392 10885 raft_consensus.cc:493] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.477131 10885 raft_consensus.cc:515] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:07.479851 10885 leader_election.cc:290] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:07.514684 10768 raft_consensus.cc:493] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.515295 10768 raft_consensus.cc:515] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:07.538461 10283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027075b60dc34541a40249671ad2fcc9" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:07.544200 10768 leader_election.cc:290] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:07.550508 10864 raft_consensus.cc:493] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.551115 10864 raft_consensus.cc:515] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
W20260709 13:04:07.554706 10178 leader_election.cc:336] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:07.558784 10768 raft_consensus.cc:493] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
W20260709 13:04:07.566437 10103 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 57834.8us
I20260709 13:04:07.573757 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb355c77dc43e78812c2b2a2df92c9" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:07.574713 10139 raft_consensus.cc:2468] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:04:07.579952 10805 raft_consensus.cc:493] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.637775 10805 raft_consensus.cc:515] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:07.640188 10805 leader_election.cc:290] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:07.590334 10688 raft_consensus.cc:493] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.708148 10688 raft_consensus.cc:515] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:07.711982 10688 leader_election.cc:290] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:07.600469 10882 raft_consensus.cc:493] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.715196 10882 raft_consensus.cc:515] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:07.716455 10881 tablet_bootstrap.cc:492] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:07.717589 10881 tablet_bootstrap.cc:492] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:07.717675 10882 leader_election.cc:290] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:07.718379 10881 ts_tablet_manager.cc:1403] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.437s	user 0.185s	sys 0.183s
W20260709 13:04:07.627115 10252 leader_election.cc:336] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:07.720026 10252 leader_election.cc:304] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
W20260709 13:04:07.721005 10252 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 93917.8us
I20260709 13:04:07.559484 10768 raft_consensus.cc:515] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:07.724336 10768 leader_election.cc:290] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:07.629330 10896 raft_consensus.cc:493] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.632802 10282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96acb559acc745428add2c682cf83cfc" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:07.725325 10282 raft_consensus.cc:2468] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:04:07.721217 10881 raft_consensus.cc:359] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:07.727679 10881 raft_consensus.cc:740] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:07.726603 10896 raft_consensus.cc:515] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:07.728612 10881 consensus_queue.cc:260] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:07.731345 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5893ec967cb241af8709bba918f3429a" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:07.732003 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f9858e8dc4fd488d562cac2d54aa4" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:07.583027 10892 raft_consensus.cc:493] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.642558 10886 raft_consensus.cc:493] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.733096 10131 raft_consensus.cc:2468] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:04:07.733655 10886 raft_consensus.cc:515] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:07.735952 10886 leader_election.cc:290] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:07.643080 10796 raft_consensus.cc:493] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.736543 10253 leader_election.cc:304] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:07.736661 10796 raft_consensus.cc:515] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:07.651199 10885 raft_consensus.cc:493] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.738209 10885 raft_consensus.cc:515] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:07.654141 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3384978cd09b40f29b899f22ded46604" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:07.664525 10897 raft_consensus.cc:493] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.739760 10897 raft_consensus.cc:515] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:07.739297 10796 leader_election.cc:290] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:07.553247 10864 leader_election.cc:290] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:07.675228 10179 leader_election.cc:304] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:07.741889 10897 leader_election.cc:290] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:07.742139 10796 raft_consensus.cc:2804] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:07.743752 10796 raft_consensus.cc:493] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.741873 10892 raft_consensus.cc:515] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:07.744287 10796 raft_consensus.cc:3060] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.748013 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a68082d0468d49ada47812f6c8e3f106" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:07.749255 10139 raft_consensus.cc:2468] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
W20260709 13:04:07.750487 10179 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:33711, user_credentials={real_user=slave}} blocked reactor thread for 75348.3us
I20260709 13:04:07.753075 10896 leader_election.cc:290] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:07.742614 10885 leader_election.cc:290] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:07.756054 10892 leader_election.cc:290] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:07.758127 10688 raft_consensus.cc:2749] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:07.758903 10688 raft_consensus.cc:493] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.759416 10688 raft_consensus.cc:515] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
W20260709 13:04:07.761787 10104 leader_election.cc:336] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:07.771476 10104 leader_election.cc:304] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:07.772176 10898 raft_consensus.cc:493] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:07.772920 10895 raft_consensus.cc:493] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.772739 10898 raft_consensus.cc:515] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:07.773494 10895 raft_consensus.cc:515] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:07.775254 10898 leader_election.cc:290] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:07.775696 10895 leader_election.cc:290] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:07.771641 10253 leader_election.cc:304] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:07.779762 10282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a68082d0468d49ada47812f6c8e3f106" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:07.780408 10898 raft_consensus.cc:2804] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:07.781045 10898 raft_consensus.cc:493] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.781730 10898 raft_consensus.cc:3060] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.781937 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:07.782732 10214 raft_consensus.cc:2468] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 3.
I20260709 13:04:07.785470 10251 leader_election.cc:304] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:07.786146 10881 ts_tablet_manager.cc:1434] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.067s	user 0.027s	sys 0.023s
I20260709 13:04:07.786890 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:07.766163 10897 raft_consensus.cc:493] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.787715 10897 raft_consensus.cc:515] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
W20260709 13:04:07.788232 10178 leader_election.cc:336] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:07.789003 10178 leader_election.cc:336] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:07.789526 10178 leader_election.cc:336] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:07.790006 10178 leader_election.cc:336] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:07.790053 10897 leader_election.cc:290] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:07.788296 10896 raft_consensus.cc:2804] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:04:07.791244 10896 raft_consensus.cc:493] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.791671 10896 raft_consensus.cc:3060] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:07.766737 10899 raft_consensus.cc:2804] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:07.794075 10899 raft_consensus.cc:493] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.755684 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0cced7e11946f09a13c6adcf95833b" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:07.794449 10897 raft_consensus.cc:493] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.794821 10211 raft_consensus.cc:2468] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:04:07.794605 10898 raft_consensus.cc:515] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:07.767398 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027075b60dc34541a40249671ad2fcc9" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:07.787206 10881 tablet_bootstrap.cc:492] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:07.796151 10139 raft_consensus.cc:2468] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:04:07.797339 10282 raft_consensus.cc:2393] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bfa051e18d0549eabe79dfcc7610d99b in current term 2: Already voted for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in this term.
I20260709 13:04:07.797926 10898 leader_election.cc:290] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:07.795243 10897 raft_consensus.cc:515] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:07.800194 10210 raft_consensus.cc:2468] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 2.
I20260709 13:04:07.801039 10897 leader_election.cc:290] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:07.801642 10251 leader_election.cc:304] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:07.794516 10899 raft_consensus.cc:3060] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.809784 10885 raft_consensus.cc:493] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.810493 10885 raft_consensus.cc:515] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:07.809834 10898 raft_consensus.cc:2804] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:07.812517 10898 raft_consensus.cc:493] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:07.812930 10885 leader_election.cc:290] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:07.813167 10898 raft_consensus.cc:3060] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:07.813580 10882 raft_consensus.cc:493] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.764320 10688 leader_election.cc:290] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
W20260709 13:04:07.814986 10179 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:33711, user_credentials={real_user=slave}} blocked reactor thread for 61068.6us
I20260709 13:04:07.814481 10882 raft_consensus.cc:515] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:07.819701 10881 tablet_bootstrap.cc:492] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:07.820139 10882 leader_election.cc:290] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:07.820813 10881 tablet_bootstrap.cc:492] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:07.821657 10881 ts_tablet_manager.cc:1403] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.035s	user 0.022s	sys 0.002s
I20260709 13:04:07.821300 10796 raft_consensus.cc:515] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:07.823709 10796 leader_election.cc:290] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:07.824543 10881 raft_consensus.cc:359] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:07.825376 10881 raft_consensus.cc:740] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:07.826480 10881 consensus_queue.cc:260] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:07.827554 10899 raft_consensus.cc:515] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:07.829017 10881 ts_tablet_manager.cc:1434] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:04:07.830020 10881 tablet_bootstrap.cc:492] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
W20260709 13:04:07.830811 10252 leader_election.cc:336] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:07.831665 10252 leader_election.cc:336] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:07.832536 10252 leader_election.cc:336] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:07.832974 10252 leader_election.cc:304] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:07.833806 10882 raft_consensus.cc:2749] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:04:07.833977 10252 leader_election.cc:336] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:07.832631 10896 raft_consensus.cc:515] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:07.836920 10899 leader_election.cc:290] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:07.839022 10896 leader_election.cc:290] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:07.836738 10898 raft_consensus.cc:515] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:07.875687   431 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:04:07.876690   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.607s	user 0.373s	sys 0.004s
I20260709 13:04:07.881333 10103 leader_election.cc:304] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
W20260709 13:04:07.884994 10104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 123055us
W20260709 13:04:07.886147 10104 leader_election.cc:336] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:07.890622 10105 leader_election.cc:304] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:07.902643 10768 raft_consensus.cc:493] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.906236 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:07.907864 10210 raft_consensus.cc:3060] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:07.904196 10768 raft_consensus.cc:515] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
W20260709 13:04:07.906695 10252 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 71974.9us
I20260709 13:04:07.910094 10864 raft_consensus.cc:2804] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:07.910806 10864 raft_consensus.cc:493] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.911264 10864 raft_consensus.cc:3060] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.913973 10890 raft_consensus.cc:2749] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:07.915858 10768 leader_election.cc:290] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:07.918129 10891 raft_consensus.cc:2804] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:07.918951 10891 raft_consensus.cc:493] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.919575 10891 raft_consensus.cc:3060] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.926285 10688 raft_consensus.cc:493] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.927436 10688 raft_consensus.cc:515] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:07.910279 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd372c208b444c4e9e4e02ee51c19601" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:07.930471 10211 raft_consensus.cc:3060] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:07.897199 10178 leader_election.cc:336] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:07.936177 10178 leader_election.cc:336] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:07.934392 10688 leader_election.cc:290] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:07.900532 10898 leader_election.cc:290] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:07.944623 10213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:07.945924 10213 raft_consensus.cc:2468] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
I20260709 13:04:07.961369 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "220a3db850dd4a5280bf7c8e9519d9d2" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:07.963750 10214 raft_consensus.cc:2468] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:04:07.980722 10253 leader_election.cc:304] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:07.981140 10894 raft_consensus.cc:493] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.982103 10894 raft_consensus.cc:515] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:07.986428 10898 raft_consensus.cc:2804] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:07.986977 10898 raft_consensus.cc:493] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:07.987411 10898 raft_consensus.cc:3060] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:07.989979 10282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4c2e323c7ec414eb53a9c1622b81dcc" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:07.991469 10282 raft_consensus.cc:2393] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bcf45a2176234db5957dc38e32554577 in current term 4: Already voted for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in this term.
I20260709 13:04:07.991801 10284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5893ec967cb241af8709bba918f3429a" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:07.992779 10284 raft_consensus.cc:2468] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:04:07.985116 10894 leader_election.cc:290] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:07.998567 10209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c29bf223e8224f72926bd39f6dcb3f08" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:07.999821 10209 raft_consensus.cc:2468] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
W20260709 13:04:08.001375 10104 leader_election.cc:336] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.006206 10104 leader_election.cc:304] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:08.007736 10131 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a68082d0468d49ada47812f6c8e3f106" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:08.008930 10131 raft_consensus.cc:3060] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.011725 10137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "737f9858e8dc4fd488d562cac2d54aa4" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:08.012991 10137 raft_consensus.cc:3060] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:08.018473 10252 leader_election.cc:336] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:08.019886 10252 leader_election.cc:336] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:08.023240 10178 leader_election.cc:336] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.016038 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
W20260709 13:04:08.023941 10178 leader_election.cc:336] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.024433 10139 raft_consensus.cc:2468] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 1.
W20260709 13:04:08.027877 10105 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.196:42219, user_credentials={real_user=slave}} blocked reactor thread for 137648us
I20260709 13:04:08.004853 10286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:08.034438 10253 leader_election.cc:304] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:08.035813 10103 leader_election.cc:304] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:08.036798 10768 raft_consensus.cc:2749] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.037497 10103 leader_election.cc:304] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:08.037695 10768 raft_consensus.cc:2804] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.038307 10768 raft_consensus.cc:493] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:08.038818 10896 raft_consensus.cc:2804] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.039368 10896 raft_consensus.cc:493] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:08.039546 10904 raft_consensus.cc:2804] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.039847 10896 raft_consensus.cc:3060] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.040091 10904 raft_consensus.cc:493] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:08.040859 10904 raft_consensus.cc:3060] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.038823 10768 raft_consensus.cc:3060] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.029649 10286 raft_consensus.cc:2468] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:04:08.002985 10251 leader_election.cc:304] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:08.046051 10211 raft_consensus.cc:2468] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 3.
I20260709 13:04:08.046921 10894 raft_consensus.cc:493] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:08.048859 10894 raft_consensus.cc:515] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:08.051906 10894 leader_election.cc:290] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
W20260709 13:04:08.060161 10251 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 57426.9us
I20260709 13:04:08.060577 10898 raft_consensus.cc:515] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:08.061654 10688 raft_consensus.cc:2804] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.062448 10688 raft_consensus.cc:493] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:08.062945 10688 raft_consensus.cc:3060] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:08.064337 10104 leader_election.cc:336] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.068351 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0b03477ff4d1ab861dc093edbe68a" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:08.069388 10214 raft_consensus.cc:2468] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 1.
I20260709 13:04:08.071110 10103 leader_election.cc:304] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
W20260709 13:04:08.072613 10252 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 52809.9us
I20260709 13:04:08.072880 10906 raft_consensus.cc:2804] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.073386 10906 raft_consensus.cc:493] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:08.073948 10906 raft_consensus.cc:3060] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.084547 10881 tablet_bootstrap.cc:492] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:08.085743 10251 leader_election.cc:304] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:08.086536 10136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0cced7e11946f09a13c6adcf95833b" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:08.088099 10895 raft_consensus.cc:2804] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:08.088765 10895 raft_consensus.cc:697] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:08.089267 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c29bf223e8224f72926bd39f6dcb3f08" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:08.090159 10134 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3384978cd09b40f29b899f22ded46604" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:08.090003 10895 consensus_queue.cc:237] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:08.091028 10134 raft_consensus.cc:2468] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 1.
I20260709 13:04:08.091181 10133 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ffb355c77dc43e78812c2b2a2df92c9" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:08.091986 10133 raft_consensus.cc:3060] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.086834 10135 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "220a3db850dd4a5280bf7c8e9519d9d2" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:08.053170 10891 raft_consensus.cc:515] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:08.099057 10210 raft_consensus.cc:2468] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 4.
I20260709 13:04:08.099354 10891 leader_election.cc:290] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:08.100795 10251 leader_election.cc:304] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:08.101926 10899 raft_consensus.cc:2804] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:08.102562 10899 raft_consensus.cc:697] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 4 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:08.103816 10899 consensus_queue.cc:237] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:08.064996 10104 leader_election.cc:304] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:08.110608 10179 leader_election.cc:304] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:08.111830 10885 raft_consensus.cc:2749] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:08.112061 10104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 47765.5us
W20260709 13:04:08.113566 10104 leader_election.cc:336] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.115270 10881 tablet_bootstrap.cc:492] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:08.116503 10881 ts_tablet_manager.cc:1403] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.287s	user 0.020s	sys 0.025s
I20260709 13:04:08.118309 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027075b60dc34541a40249671ad2fcc9" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:08.118853 10133 raft_consensus.cc:2468] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 2.
I20260709 13:04:08.119184 10138 raft_consensus.cc:3060] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.114089 10864 raft_consensus.cc:515] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:08.122818 10136 raft_consensus.cc:2393] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bcf45a2176234db5957dc38e32554577 in current term 2: Already voted for candidate bfa051e18d0549eabe79dfcc7610d99b in this term.
I20260709 13:04:08.123660 10864 leader_election.cc:290] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:08.125337 10896 raft_consensus.cc:515] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:08.120915 10768 raft_consensus.cc:515] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:08.128762 10135 raft_consensus.cc:2393] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bcf45a2176234db5957dc38e32554577 in current term 2: Already voted for candidate bfa051e18d0549eabe79dfcc7610d99b in this term.
I20260709 13:04:08.129377 10137 raft_consensus.cc:2468] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 2.
I20260709 13:04:08.131690 10768 leader_election.cc:290] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:08.136618 10253 leader_election.cc:304] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:08.141794 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "220a3db850dd4a5280bf7c8e9519d9d2" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:08.142573 10211 raft_consensus.cc:3060] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.143002 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0cced7e11946f09a13c6adcf95833b" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:08.143874 10210 raft_consensus.cc:3060] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.144484 10904 raft_consensus.cc:515] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:08.141269 10131 raft_consensus.cc:2468] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 2.
I20260709 13:04:08.146162 10136 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:08.146879 10136 raft_consensus.cc:3060] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.146826 10894 raft_consensus.cc:2749] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.142560 10895 raft_consensus.cc:2804] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.150215 10895 raft_consensus.cc:697] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:08.151412 10895 consensus_queue.cc:237] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:08.140997 10688 raft_consensus.cc:515] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:08.157816 10881 raft_consensus.cc:359] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:08.158648 10881 raft_consensus.cc:740] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:08.116652 10898 leader_election.cc:290] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:08.163091 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96acb559acc745428add2c682cf83cfc" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:08.164188 10287 raft_consensus.cc:3060] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.165342 10688 leader_election.cc:290] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:08.138725 10896 leader_election.cc:290] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:08.141229 10179 leader_election.cc:304] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
W20260709 13:04:08.172609 10252 leader_election.cc:336] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.155349 10211 raft_consensus.cc:2468] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 2.
I20260709 13:04:08.159940 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:08.170367 10132 raft_consensus.cc:2393] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bcf45a2176234db5957dc38e32554577 in current term 2: Already voted for candidate bfa051e18d0549eabe79dfcc7610d99b in this term.
I20260709 13:04:08.173631 10214 raft_consensus.cc:3060] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.174577 10885 raft_consensus.cc:2804] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.179361 10885 raft_consensus.cc:493] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:08.179997 10885 raft_consensus.cc:3060] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.181059 10179 leader_election.cc:304] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:08.183952 10881 consensus_queue.cc:260] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:08.190214 10904 leader_election.cc:290] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
W20260709 13:04:08.191040 10104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 77496us
W20260709 13:04:08.191800 10104 leader_election.cc:336] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.194880 10179 leader_election.cc:304] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:08.196471 10881 ts_tablet_manager.cc:1434] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.080s	user 0.006s	sys 0.004s
I20260709 13:04:08.196501 10103 leader_election.cc:304] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:08.204888 10904 raft_consensus.cc:2804] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.205265 10897 raft_consensus.cc:2804] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:08.197300 10287 raft_consensus.cc:2468] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 2.
I20260709 13:04:08.201905 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c29bf223e8224f72926bd39f6dcb3f08" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:08.207986 10211 raft_consensus.cc:3060] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.193156 10136 raft_consensus.cc:2468] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 2.
W20260709 13:04:08.205175 10252 leader_election.cc:336] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.205502 10904 raft_consensus.cc:697] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:08.211555 10214 raft_consensus.cc:2468] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 2.
I20260709 13:04:08.205936 10897 raft_consensus.cc:493] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:08.216818 10897 raft_consensus.cc:3060] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.217008 10796 raft_consensus.cc:2764] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 13:04:08.219810 10251 leader_election.cc:304] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:08.214834 10906 raft_consensus.cc:515] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
W20260709 13:04:08.228281 10104 leader_election.cc:336] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.240763 10210 raft_consensus.cc:2468] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 2.
I20260709 13:04:08.223464 10896 raft_consensus.cc:2804] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.241813 10896 raft_consensus.cc:697] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:08.242830 10896 consensus_queue.cc:237] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:08.244355 10253 leader_election.cc:304] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:08.245339 10688 raft_consensus.cc:2804] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.246116 10688 raft_consensus.cc:697] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:08.237102 10885 raft_consensus.cc:515] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:08.247722 10688 consensus_queue.cc:237] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
W20260709 13:04:08.245805 10252 leader_election.cc:336] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:08.245379 10253 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:33711, user_credentials={real_user=slave}} blocked reactor thread for 68296.2us
I20260709 13:04:08.251386 10138 raft_consensus.cc:2468] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 2.
W20260709 13:04:08.251575 10252 leader_election.cc:336] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.245090 10904 consensus_queue.cc:237] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:08.252450 10103 leader_election.cc:304] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:08.259518 10211 raft_consensus.cc:2468] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 2.
I20260709 13:04:08.260849 10103 leader_election.cc:304] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:08.252106 10885 leader_election.cc:290] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:08.263047 10105 leader_election.cc:304] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:08.265856 10890 raft_consensus.cc:2804] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.266469 10890 raft_consensus.cc:697] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:08.267578 10890 consensus_queue.cc:237] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:08.271696 10253 leader_election.cc:304] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:08.260639 10768 raft_consensus.cc:2804] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:04:08.277477 10252 leader_election.cc:336] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.277711 10882 raft_consensus.cc:2804] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.278231 10881 tablet_bootstrap.cc:492] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:08.279549 10253 leader_election.cc:304] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:08.278251 10768 raft_consensus.cc:697] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:08.280395 10891 raft_consensus.cc:2804] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.262830 10179 leader_election.cc:304] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
W20260709 13:04:08.284644 10178 leader_election.cc:336] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.285184 10899 raft_consensus.cc:2804] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.285625 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0b03477ff4d1ab861dc093edbe68a" candidate_uuid: "bfa051e18d0549eabe79dfcc7610d99b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:08.290342 10882 raft_consensus.cc:697] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:08.282552 10906 leader_election.cc:290] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:08.297751 10210 raft_consensus.cc:3060] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.298334 10891 raft_consensus.cc:697] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:08.298391 10882 consensus_queue.cc:237] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:08.299301 10891 consensus_queue.cc:237] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
W20260709 13:04:08.291786 10104 leader_election.cc:336] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:08.302304 10104 leader_election.cc:336] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:08.302981 10104 leader_election.cc:336] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:08.303681 10104 leader_election.cc:336] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:08.304916 10104 leader_election.cc:336] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.302099 10899 raft_consensus.cc:697] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
W20260709 13:04:08.317497 10179 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:33711, user_credentials={real_user=slave}} blocked reactor thread for 54754.7us
I20260709 13:04:08.327538 10899 consensus_queue.cc:237] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:08.331148 10210 raft_consensus.cc:2468] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa051e18d0549eabe79dfcc7610d99b in term 2.
I20260709 13:04:08.315061 10897 raft_consensus.cc:515] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:08.339771 10179 leader_election.cc:304] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:08.348459 10911 raft_consensus.cc:2804] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.343406 10103 leader_election.cc:304] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:08.349565 10911 raft_consensus.cc:697] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 2 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:04:08.345300 10768 consensus_queue.cc:237] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:08.358585 10796 raft_consensus.cc:2749] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.351567 10906 raft_consensus.cc:2804] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.360991 10906 raft_consensus.cc:697] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Becoming Leader. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:08.361774 10897 leader_election.cc:290] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 election: Requested vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:08.370405 10906 consensus_queue.cc:237] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
W20260709 13:04:08.371479 10252 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 89618.4us
W20260709 13:04:08.372220 10252 leader_election.cc:336] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.367767  9962 catalog_manager.cc:5697] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 2 to 3, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 3 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:08.389024 10104 leader_election.cc:336] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.391991 10911 consensus_queue.cc:237] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
W20260709 13:04:08.396858 10912 log.cc:927] Time spent T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 13:04:08.398828 10179 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:33711, user_credentials={real_user=slave}} blocked reactor thread for 75749.1us
I20260709 13:04:08.400682 10179 leader_election.cc:304] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
W20260709 13:04:08.402908 10178 leader_election.cc:336] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.411623 10885 raft_consensus.cc:2749] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.412240 10179 leader_election.cc:304] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:08.413208 10796 raft_consensus.cc:2749] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:08.423452  9963 catalog_manager.cc:5697] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 1 to 2, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 2 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:08.443317 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3384978cd09b40f29b899f22ded46604" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:08.443915 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5893ec967cb241af8709bba918f3429a" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:08.444801 10285 raft_consensus.cc:3060] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:08.444046 10132 raft_consensus.cc:3060] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:08.469537 10252 leader_election.cc:336] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:08.470882 10252 leader_election.cc:336] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:08.471433 10252 leader_election.cc:336] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:08.504726 10132 raft_consensus.cc:2468] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 2.
I20260709 13:04:08.519986 10285 raft_consensus.cc:2468] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 2.
I20260709 13:04:08.522678 10179 leader_election.cc:304] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:08.527347 10796 raft_consensus.cc:2804] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.529832 10796 raft_consensus.cc:697] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 2 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:04:08.531383 10796 consensus_queue.cc:237] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:08.535718 10179 leader_election.cc:304] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:08.577050 10911 raft_consensus.cc:2804] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:08.583461 10911 raft_consensus.cc:697] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 2 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
W20260709 13:04:08.582849 10252 consensus_peers.cc:597] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:08.580408 10210 raft_consensus.cc:1275] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Refusing update from remote peer 8c3d3c9022c94705ac4ee834e1dc49d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 1. (index mismatch)
I20260709 13:04:08.586710 10911 consensus_queue.cc:237] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:08.587533 10899 consensus_queue.cc:1048] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:08.591895 10881 tablet_bootstrap.cc:492] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:08.593725 10881 tablet_bootstrap.cc:492] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:08.597247 10881 ts_tablet_manager.cc:1403] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.343s	user 0.040s	sys 0.022s
W20260709 13:04:08.607890 10252 consensus_peers.cc:597] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:08.625787 10131 raft_consensus.cc:1275] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer 8c3d3c9022c94705ac4ee834e1dc49d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.631522 10896 consensus_queue.cc:1048] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:08.644954 10252 consensus_peers.cc:597] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:08.684832 10132 raft_consensus.cc:1275] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer 8c3d3c9022c94705ac4ee834e1dc49d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:08.687047 10252 consensus_peers.cc:597] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:08.695319 10881 raft_consensus.cc:359] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:08.696202 10881 raft_consensus.cc:740] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
W20260709 13:04:08.696955 10252 consensus_peers.cc:597] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:08.697834 10881 consensus_queue.cc:260] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:08.698982 10896 consensus_queue.cc:1048] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:08.724471 10881 ts_tablet_manager.cc:1434] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.127s	user 0.005s	sys 0.009s
I20260709 13:04:08.736795 10214 raft_consensus.cc:1275] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 4 FOLLOWER]: Refusing update from remote peer 8c3d3c9022c94705ac4ee834e1dc49d0: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:04:08.738718 10252 consensus_peers.cc:597] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:08.739388 10896 consensus_queue.cc:1048] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:08.742036 10214 raft_consensus.cc:1275] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Refusing update from remote peer 8c3d3c9022c94705ac4ee834e1dc49d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.746364 10896 consensus_queue.cc:1048] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:08.751745 10881 tablet_bootstrap.cc:492] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
W20260709 13:04:08.768683 10252 consensus_peers.cc:597] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:08.819077 10252 consensus_peers.cc:597] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:08.825886 10252 consensus_peers.cc:597] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:08.828830 10287 raft_consensus.cc:1275] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Refusing update from remote peer bfa051e18d0549eabe79dfcc7610d99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.835142 10136 raft_consensus.cc:1275] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer 8c3d3c9022c94705ac4ee834e1dc49d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.853349 10906 consensus_queue.cc:1048] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:08.857251 10136 raft_consensus.cc:1275] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer 8c3d3c9022c94705ac4ee834e1dc49d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.857254 10881 tablet_bootstrap.cc:492] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:08.860195 10881 tablet_bootstrap.cc:492] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:08.861049 10881 ts_tablet_manager.cc:1403] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.130s	user 0.013s	sys 0.015s
I20260709 13:04:08.848846 10898 consensus_queue.cc:1048] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:04:08.864603 10881 raft_consensus.cc:359] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:08.871939 10882 consensus_queue.cc:1048] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:08.877091 10881 raft_consensus.cc:740] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
W20260709 13:04:08.875768 10104 consensus_peers.cc:597] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:08.878633 10881 consensus_queue.cc:260] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:08.881444 10881 ts_tablet_manager.cc:1434] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:04:08.882411 10881 tablet_bootstrap.cc:492] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:08.904278 10214 raft_consensus.cc:1275] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Refusing update from remote peer bfa051e18d0549eabe79dfcc7610d99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.906191 10864 consensus_queue.cc:1048] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:08.910815 10104 consensus_peers.cc:597] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:08.911454 10104 consensus_peers.cc:597] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:08.944274 10211 raft_consensus.cc:1275] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Refusing update from remote peer bfa051e18d0549eabe79dfcc7610d99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.943215 10210 raft_consensus.cc:1275] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Refusing update from remote peer bfa051e18d0549eabe79dfcc7610d99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.949869  9959 catalog_manager.cc:5697] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 1 to 2, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:08.954245  9959 catalog_manager.cc:5697] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 1 to 2, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:08.956697 10178 consensus_peers.cc:597] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:08.960978 10864 consensus_queue.cc:1048] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:04:08.961778 10210 raft_consensus.cc:1275] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Refusing update from remote peer bfa051e18d0549eabe79dfcc7610d99b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:08.960851 10890 consensus_queue.cc:1048] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:04:08.989086 10904 consensus_queue.cc:1048] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:04:09.018934 10131 raft_consensus.cc:1275] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer bcf45a2176234db5957dc38e32554577: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:04:09.030925 10178 consensus_peers.cc:597] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:09.031653 10178 consensus_peers.cc:597] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.031543 10252 consensus_peers.cc:597] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.034601 10252 consensus_peers.cc:597] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.044245 10941 raft_consensus.cc:493] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.045138 10941 raft_consensus.cc:515] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
W20260709 13:04:09.049364 10104 consensus_peers.cc:597] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:09.049877 10104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 51171.7us
I20260709 13:04:08.958515  9959 catalog_manager.cc:5697] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 1 to 2, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:09.053331 10881 tablet_bootstrap.cc:492] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:04:09.050441 10104 consensus_peers.cc:597] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.065507 10287 raft_consensus.cc:1275] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Refusing update from remote peer bcf45a2176234db5957dc38e32554577: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:09.056403  9959 catalog_manager.cc:5697] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 3 to 4, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 4 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } }
I20260709 13:04:09.076619  9959 catalog_manager.cc:5697] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 1 to 2, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } }
I20260709 13:04:09.088893 10881 tablet_bootstrap.cc:492] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:09.094523 10886 consensus_queue.cc:1048] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:09.096813 10104 consensus_peers.cc:597] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.097800 10941 leader_election.cc:290] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:09.062836 10924 consensus_queue.cc:1048] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.058s
W20260709 13:04:09.107913 10104 consensus_peers.cc:597] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:09.136955 10881 ts_tablet_manager.cc:1403] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.255s	user 0.024s	sys 0.004s
W20260709 13:04:09.136137 10104 consensus_peers.cc:597] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.104311 10252 consensus_peers.cc:597] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.160099  9963 catalog_manager.cc:5697] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 1 to 2, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:09.172346 10104 consensus_peers.cc:597] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.175091 10178 consensus_peers.cc:597] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.180900 10178 consensus_peers.cc:597] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.183125 10178 consensus_peers.cc:597] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.191788 10178 consensus_peers.cc:597] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:09.192358 10178 consensus_peers.cc:597] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.195052  9963 catalog_manager.cc:5697] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 1 to 2, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:09.197717  9963 catalog_manager.cc:5697] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 1 to 2, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:09.199726  9963 catalog_manager.cc:5697] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 1 to 2, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:09.203019 10881 raft_consensus.cc:359] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:09.203783 10881 raft_consensus.cc:740] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:09.204483 10881 consensus_queue.cc:260] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
W20260709 13:04:09.187469 10104 consensus_peers.cc:597] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.214424 10104 consensus_peers.cc:597] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.217803 10252 consensus_peers.cc:597] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.232298 10881 ts_tablet_manager.cc:1434] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.095s	user 0.006s	sys 0.000s
I20260709 13:04:09.233405 10881 tablet_bootstrap.cc:492] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
W20260709 13:04:09.230577 10178 consensus_peers.cc:597] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.243355 10178 consensus_peers.cc:597] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.244055 10178 consensus_peers.cc:597] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.237556 10136 raft_consensus.cc:1275] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer bcf45a2176234db5957dc38e32554577: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:09.267907 10886 consensus_queue.cc:1048] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:09.281613 10252 consensus_peers.cc:597] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.283433 10104 consensus_peers.cc:597] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.314572 10941 raft_consensus.cc:493] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.315212 10941 raft_consensus.cc:515] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:09.323138 10941 leader_election.cc:290] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:09.335135  9959 catalog_manager.cc:5697] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 1 to 2, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 2 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } }
W20260709 13:04:09.337121 10252 consensus_peers.cc:597] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.338313  9959 catalog_manager.cc:5697] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 1 to 2, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 2 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } }
I20260709 13:04:09.343286 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027075b60dc34541a40249671ad2fcc9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:09.342304  9959 catalog_manager.cc:5697] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 1 to 2, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 2 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } }
I20260709 13:04:09.355082 10282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
W20260709 13:04:09.370329 10178 consensus_peers.cc:597] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.382737  9962 catalog_manager.cc:5697] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b reported cstate change: term changed from 1 to 2, leader changed from 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193) to bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194). New cstate: current_term: 2 leader_uuid: "bfa051e18d0549eabe79dfcc7610d99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:09.379295 10941 raft_consensus.cc:493] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.384189 10941 raft_consensus.cc:515] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:09.385643 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027075b60dc34541a40249671ad2fcc9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:09.386394 10941 leader_election.cc:290] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:09.388321 10877 leader_election.cc:304] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:09.391880 10941 raft_consensus.cc:2749] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:09.393966 10132 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01f3be65cb0c478194f88fb404dd329f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
W20260709 13:04:09.403690 10104 consensus_peers.cc:597] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:09.427325 10252 consensus_peers.cc:597] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.441882 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:09.449404 10881 tablet_bootstrap.cc:492] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:09.450455 10881 tablet_bootstrap.cc:492] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:09.450716 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01f3be65cb0c478194f88fb404dd329f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
W20260709 13:04:09.452410 10178 consensus_peers.cc:597] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.451197 10881 ts_tablet_manager.cc:1403] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.218s	user 0.020s	sys 0.006s
I20260709 13:04:09.454466 10875 leader_election.cc:304] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:04:09.455540 10941 raft_consensus.cc:2749] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:09.456912 10875 leader_election.cc:304] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:09.458079 10941 raft_consensus.cc:2749] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:09.458813 10881 raft_consensus.cc:359] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:09.459554 10881 raft_consensus.cc:740] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:09.465701 10881 consensus_queue.cc:260] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
W20260709 13:04:09.471351 10252 consensus_peers.cc:597] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.474993 10881 ts_tablet_manager.cc:1434] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.002s
I20260709 13:04:09.476073 10881 tablet_bootstrap.cc:492] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
W20260709 13:04:09.481462 10104 consensus_peers.cc:597] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.561692 10881 tablet_bootstrap.cc:492] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:09.574419 10881 tablet_bootstrap.cc:492] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:09.575222 10881 ts_tablet_manager.cc:1403] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.099s	user 0.027s	sys 0.024s
I20260709 13:04:09.578822 10881 raft_consensus.cc:359] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:09.579635 10881 raft_consensus.cc:740] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:09.589960 10881 consensus_queue.cc:260] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:09.597249 10881 ts_tablet_manager.cc:1434] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 13:04:09.598384 10881 tablet_bootstrap.cc:492] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:09.648867 10881 tablet_bootstrap.cc:492] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:09.657637 10881 tablet_bootstrap.cc:492] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:09.658334 10881 ts_tablet_manager.cc:1403] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.005s
I20260709 13:04:09.662153 10881 raft_consensus.cc:359] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:09.663653 10881 raft_consensus.cc:740] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:09.666471 10881 consensus_queue.cc:260] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:09.669631 10881 ts_tablet_manager.cc:1434] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:04:09.670795 10881 tablet_bootstrap.cc:492] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
W20260709 13:04:09.696316 10178 consensus_peers.cc:597] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:09.719223 10881 tablet_bootstrap.cc:492] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:09.720697 10881 tablet_bootstrap.cc:492] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:09.721552 10881 ts_tablet_manager.cc:1403] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.051s	user 0.020s	sys 0.009s
I20260709 13:04:09.726531 10881 raft_consensus.cc:359] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:09.727319 10881 raft_consensus.cc:740] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:09.728251 10881 consensus_queue.cc:260] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:09.732025 10881 ts_tablet_manager.cc:1434] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:04:09.735952 10881 tablet_bootstrap.cc:492] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:09.767409 10881 tablet_bootstrap.cc:492] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:09.769055 10881 tablet_bootstrap.cc:492] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:09.769734 10881 ts_tablet_manager.cc:1403] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.034s	user 0.024s	sys 0.003s
I20260709 13:04:09.772609 10881 raft_consensus.cc:359] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:09.773267 10881 raft_consensus.cc:740] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:09.776991 10881 consensus_queue.cc:260] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:09.782833 10881 ts_tablet_manager.cc:1434] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:04:09.783803 10881 tablet_bootstrap.cc:492] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:09.845749 10881 tablet_bootstrap.cc:492] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:09.846752 10881 tablet_bootstrap.cc:492] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:09.847625 10881 ts_tablet_manager.cc:1403] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.064s	user 0.025s	sys 0.000s
I20260709 13:04:09.850378 10881 raft_consensus.cc:359] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:09.851130 10881 raft_consensus.cc:740] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:09.852176 10881 consensus_queue.cc:260] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:09.858630 10881 ts_tablet_manager.cc:1434] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:04:09.859627 10881 tablet_bootstrap.cc:492] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:09.882328 10941 raft_consensus.cc:493] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:09.882903 10941 raft_consensus.cc:515] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:09.885025 10941 leader_election.cc:290] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:09.887004 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d87b05b458741adb45ae4a1753447d0" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bcf45a2176234db5957dc38e32554577" is_pre_election: true
I20260709 13:04:09.888053 10282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d87b05b458741adb45ae4a1753447d0" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:09.891337 10875 leader_election.cc:304] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577
I20260709 13:04:09.902063 10941 raft_consensus.cc:2749] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:09.948205 10881 tablet_bootstrap.cc:492] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:09.949836 10881 tablet_bootstrap.cc:492] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:09.953122 10881 ts_tablet_manager.cc:1403] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.094s	user 0.021s	sys 0.004s
I20260709 13:04:09.957880 10881 raft_consensus.cc:359] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:09.958986 10881 raft_consensus.cc:740] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:09.962001 10881 consensus_queue.cc:260] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:09.965294 10881 ts_tablet_manager.cc:1434] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:04:09.966605 10881 tablet_bootstrap.cc:492] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:10.004563 10881 tablet_bootstrap.cc:492] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.006083 10881 tablet_bootstrap.cc:492] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:10.007149 10881 ts_tablet_manager.cc:1403] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.015s
I20260709 13:04:10.013937 10881 raft_consensus.cc:359] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.015152 10881 raft_consensus.cc:740] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.016229 10881 consensus_queue.cc:260] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.019685 10881 ts_tablet_manager.cc:1434] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:04:10.020985 10881 tablet_bootstrap.cc:492] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:10.069859 10881 tablet_bootstrap.cc:492] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.075760 10881 tablet_bootstrap.cc:492] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:10.076944 10881 ts_tablet_manager.cc:1403] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.056s	user 0.024s	sys 0.008s
I20260709 13:04:10.083964 10881 raft_consensus.cc:359] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.088152 10881 raft_consensus.cc:740] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.089521 10881 consensus_queue.cc:260] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.094295 10881 ts_tablet_manager.cc:1434] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 13:04:10.096328 10881 tablet_bootstrap.cc:492] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:10.145205   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:43611
I20260709 13:04:10.145295 11027 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:43611 every 8 connection(s)
I20260709 13:04:10.169268 10881 tablet_bootstrap.cc:492] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.171926 10881 tablet_bootstrap.cc:492] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:10.172679 10881 ts_tablet_manager.cc:1403] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.077s	user 0.022s	sys 0.042s
I20260709 13:04:10.178293 10881 raft_consensus.cc:359] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:10.179054 10881 raft_consensus.cc:740] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.179793 10881 consensus_queue.cc:260] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:10.182539 10881 ts_tablet_manager.cc:1434] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.004s
I20260709 13:04:10.183502 10881 tablet_bootstrap.cc:492] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:10.230545 10881 tablet_bootstrap.cc:492] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.236387 10881 tablet_bootstrap.cc:492] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:10.237119 10881 ts_tablet_manager.cc:1403] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.054s	user 0.014s	sys 0.012s
I20260709 13:04:10.240280 10881 raft_consensus.cc:359] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.241053 10881 raft_consensus.cc:740] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.241897 10881 consensus_queue.cc:260] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.249403 10881 ts_tablet_manager.cc:1434] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:04:10.250377 10881 tablet_bootstrap.cc:492] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:10.258172 10992 raft_consensus.cc:3060] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.279537 10881 tablet_bootstrap.cc:492] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.281137 10993 raft_consensus.cc:3060] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.282586 10881 tablet_bootstrap.cc:492] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:10.283560 10881 ts_tablet_manager.cc:1403] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.033s	user 0.016s	sys 0.002s
I20260709 13:04:10.287564 10881 raft_consensus.cc:359] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.288306 10881 raft_consensus.cc:740] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.289139 10881 consensus_queue.cc:260] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.292812 10881 ts_tablet_manager.cc:1434] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.002s
I20260709 13:04:10.293797 10881 tablet_bootstrap.cc:492] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:10.307858 11032 heartbeater.cc:344] Connected to a master server at 127.0.107.254:44803
I20260709 13:04:10.308449 11032 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:10.309587 11032 heartbeater.cc:507] Master 127.0.107.254:44803 requested a full tablet report, sending...
I20260709 13:04:10.339031 10881 tablet_bootstrap.cc:492] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.340231 10881 tablet_bootstrap.cc:492] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:10.341034 10881 ts_tablet_manager.cc:1403] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.047s	user 0.015s	sys 0.004s
I20260709 13:04:10.343977 10881 raft_consensus.cc:359] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:10.344749 10881 raft_consensus.cc:740] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.345567 10881 consensus_queue.cc:260] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:10.347090  9962 ts_manager.cc:194] Re-registered known tserver with Master: 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:10.348158 10881 ts_tablet_manager.cc:1434] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:04:10.349327 10881 tablet_bootstrap.cc:492] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:10.365939 10881 tablet_bootstrap.cc:492] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.367764 10881 tablet_bootstrap.cc:492] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:10.368498 10881 ts_tablet_manager.cc:1403] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:04:10.371344 10881 raft_consensus.cc:359] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.372134 10881 raft_consensus.cc:740] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.373124 10881 consensus_queue.cc:260] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.375536 10881 ts_tablet_manager.cc:1434] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:10.376499 10881 tablet_bootstrap.cc:492] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:10.386019 10992 raft_consensus.cc:3060] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.386523 10993 raft_consensus.cc:3060] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.390043  9962 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56522
I20260709 13:04:10.397559 11032 heartbeater.cc:499] Master 127.0.107.254:44803 was elected leader, sending a full tablet report...
I20260709 13:04:10.422973 10881 tablet_bootstrap.cc:492] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.424016 10881 tablet_bootstrap.cc:492] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:10.424685 10881 ts_tablet_manager.cc:1403] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.048s	user 0.011s	sys 0.004s
I20260709 13:04:10.430398 10881 raft_consensus.cc:359] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:10.432101 10881 raft_consensus.cc:740] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.432842 10881 consensus_queue.cc:260] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:10.435411 10881 ts_tablet_manager.cc:1434] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:04:10.439045 10881 tablet_bootstrap.cc:492] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
W20260709 13:04:10.568641 10179 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:33711, user_credentials={real_user=slave}} blocked reactor thread for 129198us
W20260709 13:04:10.573547 10105 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.196:42219, user_credentials={real_user=slave}} blocked reactor thread for 74297.8us
I20260709 13:04:10.642681 11039 raft_consensus.cc:493] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:10.643198 11039 raft_consensus.cc:515] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:10.646119 10992 raft_consensus.cc:3060] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.645315 11039 leader_election.cc:290] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:10.687251 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7939918ac27c421f99a6747c1ea4a994" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b" is_pre_election: true
I20260709 13:04:10.709242 10993 raft_consensus.cc:3060] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.738029 10282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7939918ac27c421f99a6747c1ea4a994" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:10.747016 10877 leader_election.cc:304] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331; no voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:10.748337 11039 raft_consensus.cc:2749] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:10.757663 10881 tablet_bootstrap.cc:492] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.763319 10881 tablet_bootstrap.cc:492] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:10.764024 10881 ts_tablet_manager.cc:1403] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.326s	user 0.028s	sys 0.027s
I20260709 13:04:10.767304 10881 raft_consensus.cc:359] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:10.768052 10881 raft_consensus.cc:740] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.768755 10881 consensus_queue.cc:260] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:10.771117 10881 ts_tablet_manager.cc:1434] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:04:10.771970 10881 tablet_bootstrap.cc:492] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:10.820628 10881 tablet_bootstrap.cc:492] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.821623 10881 tablet_bootstrap.cc:492] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:10.822243 10881 ts_tablet_manager.cc:1403] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.050s	user 0.022s	sys 0.011s
I20260709 13:04:10.824683 10881 raft_consensus.cc:359] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.825361 10881 raft_consensus.cc:740] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.826042 10881 consensus_queue.cc:260] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.829113 10881 ts_tablet_manager.cc:1434] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:10.829939 10881 tablet_bootstrap.cc:492] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:10.855118 10881 tablet_bootstrap.cc:492] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.856165 10881 tablet_bootstrap.cc:492] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:10.856907 10881 ts_tablet_manager.cc:1403] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.011s
I20260709 13:04:10.858649 10992 raft_consensus.cc:3060] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:10.861668 10881 raft_consensus.cc:359] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.862540 10881 raft_consensus.cc:740] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.891703 10881 consensus_queue.cc:260] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:10.897248 10881 ts_tablet_manager.cc:1434] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.040s	user 0.018s	sys 0.008s
I20260709 13:04:10.898121 10881 tablet_bootstrap.cc:492] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:10.935844 10881 tablet_bootstrap.cc:492] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:10.936916 10881 tablet_bootstrap.cc:492] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:10.937666 10881 ts_tablet_manager.cc:1403] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.009s
I20260709 13:04:10.945933 10881 raft_consensus.cc:359] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:10.946681 10881 raft_consensus.cc:740] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:10.947366 10881 consensus_queue.cc:260] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:10.949373 10881 ts_tablet_manager.cc:1434] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 13:04:10.950265 10881 tablet_bootstrap.cc:492] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:10.985780 10993 raft_consensus.cc:3060] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:11.022358 10252 consensus_peers.cc:597] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:11.034566 10881 tablet_bootstrap.cc:492] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.035723 10881 tablet_bootstrap.cc:492] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.036453 10881 ts_tablet_manager.cc:1403] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.086s	user 0.024s	sys 0.035s
I20260709 13:04:11.039295 10881 raft_consensus.cc:359] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:11.039990 10881 raft_consensus.cc:740] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.040742 10881 consensus_queue.cc:260] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:11.043393 10881 ts_tablet_manager.cc:1434] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:11.044250 10881 tablet_bootstrap.cc:492] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
W20260709 13:04:11.045864 10252 consensus_peers.cc:597] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:11.078290 10881 tablet_bootstrap.cc:492] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.079367 10881 tablet_bootstrap.cc:492] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.080060 10881 ts_tablet_manager.cc:1403] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.036s	user 0.023s	sys 0.008s
W20260709 13:04:11.080240 10252 consensus_peers.cc:597] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:11.083648 10881 raft_consensus.cc:359] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:11.084272 10881 raft_consensus.cc:740] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.085019 10881 consensus_queue.cc:260] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:11.086990 10881 ts_tablet_manager.cc:1434] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:04:11.088289 10881 tablet_bootstrap.cc:492] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.118427 10881 tablet_bootstrap.cc:492] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.119484 10881 tablet_bootstrap.cc:492] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.120235 10881 ts_tablet_manager.cc:1403] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.009s
W20260709 13:04:11.121748 10252 consensus_peers.cc:597] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:11.122985 10881 raft_consensus.cc:359] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.123852 10881 raft_consensus.cc:740] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.124706 10881 consensus_queue.cc:260] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.127089 10881 ts_tablet_manager.cc:1434] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:04:11.128341 10881 tablet_bootstrap.cc:492] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.150842 10881 tablet_bootstrap.cc:492] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.151880 10881 tablet_bootstrap.cc:492] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.152581 10881 ts_tablet_manager.cc:1403] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.010s
I20260709 13:04:11.155115 10881 raft_consensus.cc:359] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.155789 10881 raft_consensus.cc:740] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.156555 10881 consensus_queue.cc:260] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.158780 10881 ts_tablet_manager.cc:1434] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:04:11.159580 10881 tablet_bootstrap.cc:492] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.175275 10993 raft_consensus.cc:3060] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.198643 10881 tablet_bootstrap.cc:492] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.199684 10881 tablet_bootstrap.cc:492] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.200588 10881 ts_tablet_manager.cc:1403] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.004s
I20260709 13:04:11.208657 10881 raft_consensus.cc:359] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:11.209251 10881 raft_consensus.cc:740] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.209997 10881 consensus_queue.cc:260] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:11.211972 10881 ts_tablet_manager.cc:1434] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 13:04:11.213609 10881 tablet_bootstrap.cc:492] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.249727 10881 tablet_bootstrap.cc:492] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.251056 10881 tablet_bootstrap.cc:492] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.252002 10881 ts_tablet_manager.cc:1403] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.039s	user 0.019s	sys 0.010s
I20260709 13:04:11.255522 10881 raft_consensus.cc:359] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:11.256795 10881 raft_consensus.cc:740] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.257671 10881 consensus_queue.cc:260] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:11.259526 10881 ts_tablet_manager.cc:1434] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:11.260454 10881 tablet_bootstrap.cc:492] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.279500 10881 tablet_bootstrap.cc:492] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.280562 10881 tablet_bootstrap.cc:492] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.281248 10881 ts_tablet_manager.cc:1403] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.000s
I20260709 13:04:11.284224 10881 raft_consensus.cc:359] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:11.284906 10881 raft_consensus.cc:740] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.285640 10881 consensus_queue.cc:260] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:11.287458 10881 ts_tablet_manager.cc:1434] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:04:11.292855 10881 tablet_bootstrap.cc:492] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
W20260709 13:04:11.295785 10252 consensus_peers.cc:597] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:11.352612 10881 tablet_bootstrap.cc:492] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.353749 10881 tablet_bootstrap.cc:492] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.354403 10881 ts_tablet_manager.cc:1403] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.062s	user 0.039s	sys 0.014s
I20260709 13:04:11.356415 10881 raft_consensus.cc:359] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:11.356995 10881 raft_consensus.cc:740] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.357722 10881 consensus_queue.cc:260] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:11.359977 10881 ts_tablet_manager.cc:1434] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:04:11.361110 10881 tablet_bootstrap.cc:492] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.384002 10881 tablet_bootstrap.cc:492] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.385180 10881 tablet_bootstrap.cc:492] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.385914 10881 ts_tablet_manager.cc:1403] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.025s	user 0.021s	sys 0.000s
I20260709 13:04:11.388448 10881 raft_consensus.cc:359] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.389163 10881 raft_consensus.cc:740] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.389851 10881 consensus_queue.cc:260] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.392175 10881 ts_tablet_manager.cc:1434] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:04:11.392931 10881 tablet_bootstrap.cc:492] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.407477 10995 raft_consensus.cc:3060] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.421056 10881 tablet_bootstrap.cc:492] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.422160 10881 tablet_bootstrap.cc:492] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.422792 10881 ts_tablet_manager.cc:1403] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.030s	user 0.023s	sys 0.004s
I20260709 13:04:11.426190 10881 raft_consensus.cc:359] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.426891 10881 raft_consensus.cc:740] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.427634 10881 consensus_queue.cc:260] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.430184 10881 ts_tablet_manager.cc:1434] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 13:04:11.431800 10881 tablet_bootstrap.cc:492] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.434626 10993 raft_consensus.cc:3060] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.484331 10881 tablet_bootstrap.cc:492] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.485384 10881 tablet_bootstrap.cc:492] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
W20260709 13:04:11.491047 10178 consensus_peers.cc:597] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:11.492111 10881 ts_tablet_manager.cc:1403] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.061s	user 0.028s	sys 0.025s
I20260709 13:04:11.494678 10881 raft_consensus.cc:359] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.495323 10881 raft_consensus.cc:740] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.495942 10881 consensus_queue.cc:260] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.497824 10881 ts_tablet_manager.cc:1434] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:04:11.499251 10881 tablet_bootstrap.cc:492] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
W20260709 13:04:11.512580 10252 consensus_peers.cc:597] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:11.526548 10881 tablet_bootstrap.cc:492] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.527666 10881 tablet_bootstrap.cc:492] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.528391 10881 ts_tablet_manager.cc:1403] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.029s	user 0.005s	sys 0.022s
I20260709 13:04:11.532472 10881 raft_consensus.cc:359] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:11.533221 10881 raft_consensus.cc:740] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.534446 10881 consensus_queue.cc:260] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:11.536652 10881 ts_tablet_manager.cc:1434] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:11.537544 10881 tablet_bootstrap.cc:492] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
W20260709 13:04:11.556128 10178 consensus_peers.cc:597] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:11.576571 10881 tablet_bootstrap.cc:492] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.577646 10881 tablet_bootstrap.cc:492] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.578284 10881 ts_tablet_manager.cc:1403] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.041s	user 0.027s	sys 0.010s
I20260709 13:04:11.580396 10881 raft_consensus.cc:359] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:11.581008 10881 raft_consensus.cc:740] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.581708 10881 consensus_queue.cc:260] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:11.583233 10881 ts_tablet_manager.cc:1434] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:04:11.584091 10881 tablet_bootstrap.cc:492] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.633682 10881 tablet_bootstrap.cc:492] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.634990 10881 tablet_bootstrap.cc:492] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.635680 10881 ts_tablet_manager.cc:1403] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.052s	user 0.031s	sys 0.007s
I20260709 13:04:11.638211 10881 raft_consensus.cc:359] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.638727 10881 raft_consensus.cc:740] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.639345 10881 consensus_queue.cc:260] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.641279 10881 ts_tablet_manager.cc:1434] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:04:11.642936 10881 tablet_bootstrap.cc:492] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.680605 10881 tablet_bootstrap.cc:492] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.681689 10881 tablet_bootstrap.cc:492] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.682405 10881 ts_tablet_manager.cc:1403] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.040s	user 0.020s	sys 0.018s
I20260709 13:04:11.685010 10881 raft_consensus.cc:359] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.685707 10881 raft_consensus.cc:740] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.686368 10881 consensus_queue.cc:260] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.688496 10881 ts_tablet_manager.cc:1434] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:04:11.689397 10881 tablet_bootstrap.cc:492] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.696942 10995 raft_consensus.cc:3060] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:11.705821 10881 tablet_bootstrap.cc:492] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.707206 10881 tablet_bootstrap.cc:492] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.707949 10881 ts_tablet_manager.cc:1403] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.005s
I20260709 13:04:11.710857 10881 raft_consensus.cc:359] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.712024 10881 raft_consensus.cc:740] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.712886 10881 consensus_queue.cc:260] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.715495 10881 ts_tablet_manager.cc:1434] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:11.716424 10881 tablet_bootstrap.cc:492] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
W20260709 13:04:11.732327 10104 consensus_peers.cc:597] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:11.739833 10881 tablet_bootstrap.cc:492] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.740973 10881 tablet_bootstrap.cc:492] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.741820 10881 ts_tablet_manager.cc:1403] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.004s
I20260709 13:04:11.746464 10881 raft_consensus.cc:359] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:11.747260 10881 raft_consensus.cc:740] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.748132 10881 consensus_queue.cc:260] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:11.751405 10881 ts_tablet_manager.cc:1434] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:04:11.752272 10881 tablet_bootstrap.cc:492] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.868314 10881 tablet_bootstrap.cc:492] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.869374 10881 tablet_bootstrap.cc:492] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.870149 10881 ts_tablet_manager.cc:1403] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.118s	user 0.044s	sys 0.067s
I20260709 13:04:11.873126 10881 raft_consensus.cc:359] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:11.873910 10881 raft_consensus.cc:740] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.874630 10881 consensus_queue.cc:260] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:11.876052 10993 raft_consensus.cc:3060] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:11.876926 10881 ts_tablet_manager.cc:1434] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:11.878403 10881 tablet_bootstrap.cc:492] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.918702 10881 tablet_bootstrap.cc:492] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:04:11.919961 10881 tablet_bootstrap.cc:492] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.920843 10881 ts_tablet_manager.cc:1403] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.043s	user 0.036s	sys 0.004s
I20260709 13:04:11.923991 10881 raft_consensus.cc:359] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.925060 10881 raft_consensus.cc:740] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.926016 10881 consensus_queue.cc:260] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:11.928536 10881 ts_tablet_manager.cc:1434] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:11.929591 10881 tablet_bootstrap.cc:492] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap starting.
I20260709 13:04:11.961889 10881 tablet_bootstrap.cc:492] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:11.962760 10881 tablet_bootstrap.cc:492] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Bootstrap complete.
I20260709 13:04:11.963323 10881 ts_tablet_manager.cc:1403] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.010s
I20260709 13:04:11.965533 10881 raft_consensus.cc:359] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:11.966228 10881 raft_consensus.cc:740] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Initialized, Role: FOLLOWER
I20260709 13:04:11.966897 10881 consensus_queue.cc:260] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:11.969237 10881 ts_tablet_manager.cc:1434] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:04:12.079936 10993 raft_consensus.cc:3060] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:12.096503 10993 pending_rounds.cc:85] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: Aborting all ops after (but not including) 0
I20260709 13:04:12.147255 10993 pending_rounds.cc:107] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331: Aborting uncommitted NO_OP operation due to leader change: 2.1
I20260709 13:04:12.147704 10993 raft_consensus.cc:2889] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:04:12.161542 10104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 63877.9us
I20260709 13:04:12.219075 10993 raft_consensus.cc:3060] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:18.529531 10178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.193:43611, user_credentials={real_user=slave}} blocked reactor thread for 101392us
W20260709 13:04:19.483898 10235 debug-util.cc:398] Leaking SignalData structure 0x7b08001b1760 after lost signal to thread 434
W20260709 13:04:19.485080 10235 debug-util.cc:398] Leaking SignalData structure 0x7b08000c97e0 after lost signal to thread 9997
W20260709 13:04:19.486245 10235 debug-util.cc:398] Leaking SignalData structure 0x7b08000f14a0 after lost signal to thread 10164
W20260709 13:04:19.487210 10235 debug-util.cc:398] Leaking SignalData structure 0x7b08000e36c0 after lost signal to thread 10238
W20260709 13:04:19.488428 10235 debug-util.cc:398] Leaking SignalData structure 0x7b08003406c0 after lost signal to thread 10312
W20260709 13:04:19.489682 10235 debug-util.cc:398] Leaking SignalData structure 0x7b08003ff960 after lost signal to thread 11027
I20260709 13:04:20.361369   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:20.487458 10288 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17c5f39847af4013aa501ce097f58b87"
dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:44960
I20260709 13:04:20.488499 10288 raft_consensus.cc:606] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:20.492264 10288 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081"
dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:44960
I20260709 13:04:20.493182 10288 raft_consensus.cc:606] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:20.503723 10285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22b9c093052a4414be72162a905b9a1d"
dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
mode: GRACEFUL
new_leader_uuid: "bcf45a2176234db5957dc38e32554577"
 from {username='slave'} at 127.0.0.1:44960
I20260709 13:04:20.504369 10285 raft_consensus.cc:606] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Received request to transfer leadership to TS bcf45a2176234db5957dc38e32554577
I20260709 13:04:20.582124 10138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7c0cced7e11946f09a13c6adcf95833b"
dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:45054
I20260709 13:04:20.582779 10138 raft_consensus.cc:606] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:20.586812 10288 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a68082d0468d49ada47812f6c8e3f106"
dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:44960
I20260709 13:04:20.587471 10288 raft_consensus.cc:606] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:20.618475 10211 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9d3e10044ce408294e1b86ce960b84a"
dest_uuid: "bcf45a2176234db5957dc38e32554577"
mode: GRACEFUL
new_leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
 from {username='slave'} at 127.0.0.1:59300
I20260709 13:04:20.619050 10211 raft_consensus.cc:606] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Received request to transfer leadership to TS 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:20.621778 10288 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f"
dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:44960
I20260709 13:04:20.622362 10288 raft_consensus.cc:606] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:20.624050   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 13:04:20.625628   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:20.731410 11141 raft_consensus.cc:993] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:20.732141 11148 raft_consensus.cc:1081] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:20.734102 10987 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17c5f39847af4013aa501ce097f58b87"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56726
I20260709 13:04:20.734699 10987 raft_consensus.cc:493] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:20.735083 10987 raft_consensus.cc:3060] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:20.742617 10987 raft_consensus.cc:515] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:20.746025 10987 leader_election.cc:290] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:20.746637 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c5f39847af4013aa501ce097f58b87" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:20.747540 10288 raft_consensus.cc:3055] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:20.747880 10288 raft_consensus.cc:740] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:20.748584 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17c5f39847af4013aa501ce097f58b87" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:20.749095 10138 raft_consensus.cc:3060] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:20.748792 10288 consensus_queue.cc:260] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:20.750557 10288 raft_consensus.cc:3060] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:20.756583 10138 raft_consensus.cc:2468] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:20.758388 10877 leader_election.cc:304] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:20.760910 10288 raft_consensus.cc:2468] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:20.762452 11149 raft_consensus.cc:2804] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:20.766881 11149 raft_consensus.cc:697] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:20.768437 11149 consensus_queue.cc:237] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:20.806697  9963 catalog_manager.cc:5697] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 1 to 2, leader changed from 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:20.902653 11148 raft_consensus.cc:993] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:20.903276 11148 raft_consensus.cc:1081] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:20.905395 10987 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56726
I20260709 13:04:20.906191 10987 raft_consensus.cc:493] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:20.906694 10987 raft_consensus.cc:3060] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:20.918816 10987 raft_consensus.cc:515] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:20.921960 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:20.923002 10285 raft_consensus.cc:3055] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:20.923403 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1feaf9d0d9ca42c083b85e63a09d0081" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:20.923571 10987 leader_election.cc:290] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:20.924075 10138 raft_consensus.cc:3060] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:20.923398 10285 raft_consensus.cc:740] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:20.926558 10285 consensus_queue.cc:260] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:20.928035 10285 raft_consensus.cc:3060] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:20.934365 10138 raft_consensus.cc:2468] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 3.
I20260709 13:04:20.935774 10877 leader_election.cc:304] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:20.937039 11149 raft_consensus.cc:2804] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:20.937568 11149 raft_consensus.cc:697] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:20.938494 11149 consensus_queue.cc:237] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:20.941268 11141 raft_consensus.cc:993] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:20.942588 11141 raft_consensus.cc:1081] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:20.948963 11148 raft_consensus.cc:993] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0: : Instructing follower bcf45a2176234db5957dc38e32554577 to start an election
I20260709 13:04:20.949769 11148 raft_consensus.cc:1081] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Signalling peer bcf45a2176234db5957dc38e32554577 to start an election
I20260709 13:04:20.952484 10285 raft_consensus.cc:2468] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 3.
I20260709 13:04:20.957756 10211 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22b9c093052a4414be72162a905b9a1d"
dest_uuid: "bcf45a2176234db5957dc38e32554577"
 from {username='slave'} at 127.0.0.1:51176
I20260709 13:04:20.958369 10211 raft_consensus.cc:493] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:20.958763 10211 raft_consensus.cc:3060] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:20.959749 10987 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a68082d0468d49ada47812f6c8e3f106"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56726
I20260709 13:04:20.961328 10987 raft_consensus.cc:493] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:20.961962 10987 raft_consensus.cc:3060] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:20.962308  9963 catalog_manager.cc:5697] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 2 to 3, leader changed from 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:20.984539 10987 raft_consensus.cc:515] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:20.987339 10987 leader_election.cc:290] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:20.988165 11116 raft_consensus.cc:993] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577: : Instructing follower 8c3d3c9022c94705ac4ee834e1dc49d0 to start an election
I20260709 13:04:20.988700 11116 raft_consensus.cc:1081] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Signalling peer 8c3d3c9022c94705ac4ee834e1dc49d0 to start an election
I20260709 13:04:20.993240 10211 raft_consensus.cc:515] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:20.995191 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a68082d0468d49ada47812f6c8e3f106" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:20.995874 10288 raft_consensus.cc:3055] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:20.996186 10288 raft_consensus.cc:740] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:20.998711 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a68082d0468d49ada47812f6c8e3f106" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:20.999368 10139 raft_consensus.cc:3060] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:20.996989 10288 consensus_queue.cc:260] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:21.000931 10288 raft_consensus.cc:3060] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.002142 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b9c093052a4414be72162a905b9a1d" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:21.002791 10138 raft_consensus.cc:3060] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.006916 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22b9c093052a4414be72162a905b9a1d" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:21.007212 10287 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9d3e10044ce408294e1b86ce960b84a"
dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
 from {username='slave'} at 127.0.0.1:39532
I20260709 13:04:21.007668 10285 raft_consensus.cc:3055] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:21.007879 10287 raft_consensus.cc:493] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:21.008044 10285 raft_consensus.cc:740] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:21.008476 10287 raft_consensus.cc:3060] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.009076 10285 consensus_queue.cc:260] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:21.011391 10285 raft_consensus.cc:3060] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:20.996950 10211 leader_election.cc:290] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:21.025269 10139 raft_consensus.cc:2468] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 3.
I20260709 13:04:21.027562 11119 raft_consensus.cc:993] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:21.030205 11119 raft_consensus.cc:1081] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:21.030803 10877 leader_election.cc:304] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:21.033480 10987 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7c0cced7e11946f09a13c6adcf95833b"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56738
I20260709 13:04:21.034076 10987 raft_consensus.cc:493] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:21.034461 10987 raft_consensus.cc:3060] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.035396 11149 raft_consensus.cc:2804] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:21.049679 10288 raft_consensus.cc:2468] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 3.
I20260709 13:04:21.050549 10287 raft_consensus.cc:515] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:21.036867 10285 raft_consensus.cc:2468] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 2.
I20260709 13:04:21.046698 11149 raft_consensus.cc:697] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:21.046154 10138 raft_consensus.cc:2468] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 2.
I20260709 13:04:21.052470 11149 consensus_queue.cc:237] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:21.053576 10179 leader_election.cc:304] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:21.044113 10987 raft_consensus.cc:515] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:21.056380 11121 raft_consensus.cc:2804] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:21.058733 11121 raft_consensus.cc:697] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [term 2 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:04:21.060807 11121 consensus_queue.cc:237] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:21.062098 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d3e10044ce408294e1b86ce960b84a" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:21.062754 10210 raft_consensus.cc:3055] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:21.063024 10210 raft_consensus.cc:740] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:04:21.063822 10210 consensus_queue.cc:260] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:21.065169 10210 raft_consensus.cc:3060] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.073813 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0cced7e11946f09a13c6adcf95833b" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:21.058807 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9d3e10044ce408294e1b86ce960b84a" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:21.074455 10139 raft_consensus.cc:3055] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:21.074924 10138 raft_consensus.cc:3060] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.075042 10139 raft_consensus.cc:740] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:21.076244 10139 consensus_queue.cc:260] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:21.077692 10139 raft_consensus.cc:3060] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.074035 10287 leader_election.cc:290] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:21.080570 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7c0cced7e11946f09a13c6adcf95833b" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:21.080749 10987 leader_election.cc:290] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:21.081269 10211 raft_consensus.cc:3060] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.085247 10210 raft_consensus.cc:2468] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 2.
I20260709 13:04:21.087018 10251 leader_election.cc:304] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:21.088367 11143 raft_consensus.cc:2804] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:21.089108 11143 raft_consensus.cc:697] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:21.090523 11143 consensus_queue.cc:237] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:21.093791 10211 raft_consensus.cc:2468] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 3.
I20260709 13:04:21.095317 10875 leader_election.cc:304] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:21.096249 11149 raft_consensus.cc:2804] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:21.096765 11149 raft_consensus.cc:697] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:21.097662 11149 consensus_queue.cc:237] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:21.108493 10138 raft_consensus.cc:2468] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 2.
I20260709 13:04:21.109712 10139 raft_consensus.cc:2468] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 3.
I20260709 13:04:21.124042  9963 catalog_manager.cc:5697] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 2 to 3, leader changed from 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:21.127993  9959 catalog_manager.cc:5697] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 1 to 2, leader changed from 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196) to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:21.134063  9962 catalog_manager.cc:5697] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 1 to 2, leader changed from bcf45a2176234db5957dc38e32554577 (127.0.107.195) to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:21.146552 11141 raft_consensus.cc:993] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:21.147097 11141 raft_consensus.cc:1081] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:21.152890 10995 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56726
I20260709 13:04:21.153465 10995 raft_consensus.cc:493] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:21.153829 10995 raft_consensus.cc:3060] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.167315 10995 raft_consensus.cc:515] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:21.168830  9962 catalog_manager.cc:5697] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 2 to 3, leader changed from bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:21.179136 10139 raft_consensus.cc:1275] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:21.180763 10995 leader_election.cc:290] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:21.181807 11149 consensus_queue.cc:1048] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:21.182272 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:21.182893 10211 raft_consensus.cc:3060] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.186074 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:21.186749 10288 raft_consensus.cc:3055] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:21.187047 10288 raft_consensus.cc:740] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:21.187897 10288 consensus_queue.cc:260] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:21.189320 10288 raft_consensus.cc:3060] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:21.190294 10211 raft_consensus.cc:2468] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 3.
I20260709 13:04:21.191958 10875 leader_election.cc:304] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:21.193053 11149 raft_consensus.cc:2804] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:21.193661 11149 raft_consensus.cc:697] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:21.194537 11149 consensus_queue.cc:237] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:21.216753 10287 raft_consensus.cc:1275] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:21.205047 10288 raft_consensus.cc:2468] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 3.
I20260709 13:04:21.225587 11166 consensus_queue.cc:1048] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:21.258179  9962 catalog_manager.cc:5697] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 2 to 3, leader changed from 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:21.353272 10288 raft_consensus.cc:1275] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.354943 11166 consensus_queue.cc:1048] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:21.371220 10139 raft_consensus.cc:1275] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.373173 11149 consensus_queue.cc:1048] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:21.455705 10139 raft_consensus.cc:1275] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.457160 11166 consensus_queue.cc:1048] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:21.474119 10288 raft_consensus.cc:1275] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.481763 11149 consensus_queue.cc:1048] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:21.482889 10138 raft_consensus.cc:1275] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer bcf45a2176234db5957dc38e32554577: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:21.485014 11116 consensus_queue.cc:1048] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:21.526618 10288 raft_consensus.cc:1275] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Refusing update from remote peer bcf45a2176234db5957dc38e32554577: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:21.528304 11135 consensus_queue.cc:1048] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:21.569952 10139 raft_consensus.cc:1275] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.571988 11166 consensus_queue.cc:1048] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:21.588402 10211 raft_consensus.cc:1275] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.590111 11149 consensus_queue.cc:1048] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:21.609320 10139 raft_consensus.cc:1275] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer 8c3d3c9022c94705ac4ee834e1dc49d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:21.611698 11148 consensus_queue.cc:1048] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:21.624390 10211 raft_consensus.cc:1275] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Refusing update from remote peer 8c3d3c9022c94705ac4ee834e1dc49d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:21.629354   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:21.640938 11141 consensus_queue.cc:1048] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:21.720088   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:21.754876 10211 raft_consensus.cc:1275] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.756712 11167 consensus_queue.cc:1048] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:21.779244 10285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "027075b60dc34541a40249671ad2fcc9"
dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:44960
I20260709 13:04:21.779975 10285 raft_consensus.cc:606] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:21.784550 10288 raft_consensus.cc:1275] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:21.785810 10211 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8dc5cf0e36da48d0946769d2bb27c981"
dest_uuid: "bcf45a2176234db5957dc38e32554577"
mode: GRACEFUL
new_leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
 from {username='slave'} at 127.0.0.1:59300
I20260709 13:04:21.786619 10211 raft_consensus.cc:606] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Received request to transfer leadership to TS 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:21.790380 10138 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "96acb559acc745428add2c682cf83cfc"
dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:45054
I20260709 13:04:21.791059 10138 raft_consensus.cc:606] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:21.793727 11150 consensus_queue.cc:1048] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:21.796806 10211 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4"
dest_uuid: "bcf45a2176234db5957dc38e32554577"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:59300
I20260709 13:04:21.797509 10211 raft_consensus.cc:606] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:21.802999 10211 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ccc0007e852f46b2b3ce899d1101e17c"
dest_uuid: "bcf45a2176234db5957dc38e32554577"
mode: GRACEFUL
new_leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
 from {username='slave'} at 127.0.0.1:59300
I20260709 13:04:21.803751 10211 raft_consensus.cc:606] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Received request to transfer leadership to TS 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:21.809026 10285 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dde8fc103c3d453abd4d298b1beffd74"
dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:44960
I20260709 13:04:21.809698 10285 raft_consensus.cc:606] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:21.811511   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:04:21.813388   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:21.932463 11121 raft_consensus.cc:993] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:21.933012 11116 raft_consensus.cc:1081] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:21.934697 10995 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56744
I20260709 13:04:21.935292 10995 raft_consensus.cc:493] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:21.935663 10995 raft_consensus.cc:3060] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.943478 10995 raft_consensus.cc:515] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:21.946492 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:21.947137 10210 raft_consensus.cc:3055] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:21.947356 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9a78dad3cd8b43a0bf32fc168eb162d4" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:21.948005 10995 leader_election.cc:290] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:21.948133 10288 raft_consensus.cc:3060] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.947482 10210 raft_consensus.cc:740] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:04:21.950366 10210 consensus_queue.cc:260] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:21.951498 10210 raft_consensus.cc:3060] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:21.978196 10288 raft_consensus.cc:2468] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:21.979882 10877 leader_election.cc:304] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 
I20260709 13:04:21.980795 11167 raft_consensus.cc:2804] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:21.981272 11167 raft_consensus.cc:697] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:21.982242 11167 consensus_queue.cc:237] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:21.990388 10210 raft_consensus.cc:2468] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:22.019035  9959 catalog_manager.cc:5697] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 1 to 2, leader changed from bcf45a2176234db5957dc38e32554577 (127.0.107.195) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:22.105237 11184 raft_consensus.cc:993] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:22.106036 11142 raft_consensus.cc:1081] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:22.108244 10989 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "96acb559acc745428add2c682cf83cfc"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56738
I20260709 13:04:22.108930 10989 raft_consensus.cc:493] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.109321 10989 raft_consensus.cc:3060] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.118278 11121 raft_consensus.cc:993] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577: : Instructing follower 8c3d3c9022c94705ac4ee834e1dc49d0 to start an election
I20260709 13:04:22.117938 10989 raft_consensus.cc:515] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:22.118904 11135 raft_consensus.cc:1081] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Signalling peer 8c3d3c9022c94705ac4ee834e1dc49d0 to start an election
I20260709 13:04:22.120420 10288 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ccc0007e852f46b2b3ce899d1101e17c"
dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
 from {username='slave'} at 127.0.0.1:39532
I20260709 13:04:22.120883 10288 raft_consensus.cc:493] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.120940 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96acb559acc745428add2c682cf83cfc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:22.121311 10288 raft_consensus.cc:3060] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.121665 10285 raft_consensus.cc:3060] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.122478 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96acb559acc745428add2c682cf83cfc" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:22.123145 10139 raft_consensus.cc:3055] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:22.123618 10139 raft_consensus.cc:740] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:22.124666 10139 consensus_queue.cc:260] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:22.125939 10139 raft_consensus.cc:3060] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.129796 10989 leader_election.cc:290] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:22.129724 10285 raft_consensus.cc:2468] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 3.
I20260709 13:04:22.131978 10877 leader_election.cc:304] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 
I20260709 13:04:22.132356 10288 raft_consensus.cc:515] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:22.134387 11150 raft_consensus.cc:2804] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:22.137001 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0007e852f46b2b3ce899d1101e17c" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:22.137764 10138 raft_consensus.cc:3060] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.137840 10288 leader_election.cc:290] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:22.138015 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc0007e852f46b2b3ce899d1101e17c" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:22.138608 10210 raft_consensus.cc:3055] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:22.138909 10210 raft_consensus.cc:740] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:04:22.139678 10210 consensus_queue.cc:260] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:22.140695 11150 raft_consensus.cc:697] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:22.141865 11150 consensus_queue.cc:237] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:22.140775 10210 raft_consensus.cc:3060] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.167955 10138 raft_consensus.cc:2468] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 2.
I20260709 13:04:22.173542 10253 leader_election.cc:304] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:22.174336 11189 raft_consensus.cc:2804] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:22.174911 11189 raft_consensus.cc:697] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:22.175911 11189 consensus_queue.cc:237] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:22.176743 11140 raft_consensus.cc:993] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:22.176306  9962 catalog_manager.cc:5697] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 2 to 3, leader changed from bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.177241 11140 raft_consensus.cc:1081] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:22.179908 10210 raft_consensus.cc:2468] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 2.
I20260709 13:04:22.180940 10989 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "027075b60dc34541a40249671ad2fcc9"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56726
I20260709 13:04:22.181660 10989 raft_consensus.cc:493] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.182066 10989 raft_consensus.cc:3060] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.207922 10139 raft_consensus.cc:2468] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 3.
I20260709 13:04:22.212016  9962 catalog_manager.cc:5697] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 1 to 2, leader changed from bcf45a2176234db5957dc38e32554577 (127.0.107.195) to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:22.216190 10989 raft_consensus.cc:515] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:22.219228 10989 leader_election.cc:290] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:22.219756 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027075b60dc34541a40249671ad2fcc9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:22.220392 10288 raft_consensus.cc:3055] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:22.220687 10288 raft_consensus.cc:740] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:22.221487 10288 consensus_queue.cc:260] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:22.222752 10288 raft_consensus.cc:3060] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.222643 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "027075b60dc34541a40249671ad2fcc9" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:22.223382 10139 raft_consensus.cc:3060] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:22.231233 10139 raft_consensus.cc:2468] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 3.
I20260709 13:04:22.233148 10288 raft_consensus.cc:2468] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 3.
I20260709 13:04:22.235886 10877 leader_election.cc:304] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 
I20260709 13:04:22.236891 11150 raft_consensus.cc:2804] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:22.237394 11150 raft_consensus.cc:697] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:22.238255 11150 consensus_queue.cc:237] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:22.248631  9962 catalog_manager.cc:5697] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 2 to 3, leader changed from 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 3 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.290498 11189 raft_consensus.cc:993] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:22.291025 11141 raft_consensus.cc:1081] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:22.292809 10989 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dde8fc103c3d453abd4d298b1beffd74"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56726
I20260709 13:04:22.293510 10989 raft_consensus.cc:493] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.294160 10989 raft_consensus.cc:3060] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.302000 10989 raft_consensus.cc:515] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:22.304534 10989 leader_election.cc:290] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:22.304682 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde8fc103c3d453abd4d298b1beffd74" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:22.305456 10288 raft_consensus.cc:3055] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:22.305807 10288 raft_consensus.cc:740] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:22.306005 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dde8fc103c3d453abd4d298b1beffd74" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:22.306581 10139 raft_consensus.cc:3060] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.306705 10288 consensus_queue.cc:260] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:22.308207 10288 raft_consensus.cc:3060] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.311844 10139 raft_consensus.cc:2468] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:22.315230 10877 leader_election.cc:304] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:22.317602 11150 raft_consensus.cc:2804] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:22.318156 11150 raft_consensus.cc:697] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:22.319108 11150 consensus_queue.cc:237] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:22.324833 10288 raft_consensus.cc:2468] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:22.334187  9962 catalog_manager.cc:5697] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 1 to 2, leader changed from 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:22.391140 11121 raft_consensus.cc:993] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577: : Instructing follower 8c3d3c9022c94705ac4ee834e1dc49d0 to start an election
I20260709 13:04:22.391641 11121 raft_consensus.cc:1081] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Signalling peer 8c3d3c9022c94705ac4ee834e1dc49d0 to start an election
I20260709 13:04:22.393362 10288 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8dc5cf0e36da48d0946769d2bb27c981"
dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
 from {username='slave'} at 127.0.0.1:39532
I20260709 13:04:22.394248 10288 raft_consensus.cc:493] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:22.394610 10288 raft_consensus.cc:3060] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.402822 10288 raft_consensus.cc:515] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:22.405079 10288 leader_election.cc:290] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:22.406282 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5cf0e36da48d0946769d2bb27c981" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:22.407151 10139 raft_consensus.cc:3060] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.406282 10210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8dc5cf0e36da48d0946769d2bb27c981" candidate_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:22.407817 10210 raft_consensus.cc:3055] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:22.408202 10210 raft_consensus.cc:740] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:04:22.409368 10210 consensus_queue.cc:260] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:22.413938 10210 raft_consensus.cc:3060] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:22.417815 10139 raft_consensus.cc:2468] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 2.
I20260709 13:04:22.421787 10210 raft_consensus.cc:2468] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c3d3c9022c94705ac4ee834e1dc49d0 in term 2.
I20260709 13:04:22.422947 10253 leader_election.cc:304] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:22.426472 11141 raft_consensus.cc:2804] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:22.427117 11141 raft_consensus.cc:697] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Becoming Leader. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:22.428287 11141 consensus_queue.cc:237] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:22.432358 10210 raft_consensus.cc:1275] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:22.437680 11150 consensus_queue.cc:1048] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:22.441402  9962 catalog_manager.cc:5697] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 reported cstate change: term changed from 1 to 2, leader changed from bcf45a2176234db5957dc38e32554577 (127.0.107.195) to 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196). New cstate: current_term: 2 leader_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:22.498785 10288 raft_consensus.cc:1275] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:22.500422 11149 consensus_queue.cc:1048] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:22.552500 10139 raft_consensus.cc:1275] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:22.554268 11149 consensus_queue.cc:1048] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:22.580564 10285 raft_consensus.cc:1275] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:22.582041 11149 consensus_queue.cc:1048] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:22.632007 10210 raft_consensus.cc:1275] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Refusing update from remote peer 8c3d3c9022c94705ac4ee834e1dc49d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:22.633213 11189 consensus_queue.cc:1048] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:22.644440 10139 raft_consensus.cc:1275] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer 8c3d3c9022c94705ac4ee834e1dc49d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:22.647948 11141 consensus_queue.cc:1048] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:22.718451 10139 raft_consensus.cc:1275] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:22.719743 11149 consensus_queue.cc:1048] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:22.731822 10288 raft_consensus.cc:1275] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:22.735700 11150 consensus_queue.cc:1048] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:22.813125 10288 raft_consensus.cc:1275] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:22.814406   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:22.815972 11150 consensus_queue.cc:1048] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:22.828390 10139 raft_consensus.cc:1275] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:22.830154 11163 consensus_queue.cc:1048] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:22.879144 10211 raft_consensus.cc:1275] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Refusing update from remote peer 8c3d3c9022c94705ac4ee834e1dc49d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:22.880822 11141 consensus_queue.cc:1048] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:22.903858 10139 raft_consensus.cc:1275] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer 8c3d3c9022c94705ac4ee834e1dc49d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:22.909096 11140 consensus_queue.cc:1048] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:04:22.965093   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:23.028862 10139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01f3be65cb0c478194f88fb404dd329f"
dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:45054
I20260709 13:04:23.029619 10139 raft_consensus.cc:606] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:23.032752 10139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0504e36ef8da48b189ebb41995e15efd"
dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:45054
I20260709 13:04:23.033372 10139 raft_consensus.cc:606] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:23.036463 10139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "940beae71dca43e58d3879c233f87dbb"
dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:45054
I20260709 13:04:23.037060 10139 raft_consensus.cc:606] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:23.040968 10287 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ea8a48dadff4881a7cdc0b27cf97db3"
dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
mode: GRACEFUL
new_leader_uuid: "bcf45a2176234db5957dc38e32554577"
 from {username='slave'} at 127.0.0.1:44960
I20260709 13:04:23.041690 10287 raft_consensus.cc:606] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Received request to transfer leadership to TS bcf45a2176234db5957dc38e32554577
I20260709 13:04:23.047636 10287 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f1aaf32c658c483eb6a4a3858745515f"
dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:44960
I20260709 13:04:23.048306 10287 raft_consensus.cc:606] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:23.048775 11142 raft_consensus.cc:993] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:23.051210   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:04:23.051558 11119 raft_consensus.cc:1081] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:23.053216   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:23.055855 10989 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0504e36ef8da48b189ebb41995e15efd"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56738
I20260709 13:04:23.056370 10989 raft_consensus.cc:493] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.056658 10989 raft_consensus.cc:3060] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.064790 10989 raft_consensus.cc:515] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:23.067965 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0504e36ef8da48b189ebb41995e15efd" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:23.068692 10287 raft_consensus.cc:3060] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.069664 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0504e36ef8da48b189ebb41995e15efd" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:23.070345 10139 raft_consensus.cc:3055] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.070641 10139 raft_consensus.cc:740] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:23.071368 10139 consensus_queue.cc:260] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:23.072434 10139 raft_consensus.cc:3060] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.080976 10989 leader_election.cc:290] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:23.085719 10139 raft_consensus.cc:2468] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:23.087741 10877 leader_election.cc:304] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:23.088773 11149 raft_consensus.cc:2804] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.089509 11149 raft_consensus.cc:697] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:23.090620 11149 consensus_queue.cc:237] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:23.098719 10287 raft_consensus.cc:2468] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:23.119946  9959 catalog_manager.cc:5697] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 1 to 2, leader changed from bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.408375 11212 raft_consensus.cc:993] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:23.408911 11211 raft_consensus.cc:1081] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:23.411015 10987 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "01f3be65cb0c478194f88fb404dd329f"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56738
I20260709 13:04:23.411631 10987 raft_consensus.cc:493] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.412014 10987 raft_consensus.cc:3060] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.422636 10987 raft_consensus.cc:515] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:23.426456 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01f3be65cb0c478194f88fb404dd329f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:23.426926 10211 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01f3be65cb0c478194f88fb404dd329f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:23.427490 10138 raft_consensus.cc:3055] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.427631 10211 raft_consensus.cc:3060] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.435752 11189 raft_consensus.cc:993] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0: : Instructing follower bcf45a2176234db5957dc38e32554577 to start an election
I20260709 13:04:23.427925 10138 raft_consensus.cc:740] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:23.436506 11209 raft_consensus.cc:1081] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Signalling peer bcf45a2176234db5957dc38e32554577 to start an election
I20260709 13:04:23.438550 10210 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ea8a48dadff4881a7cdc0b27cf97db3"
dest_uuid: "bcf45a2176234db5957dc38e32554577"
 from {username='slave'} at 127.0.0.1:51176
I20260709 13:04:23.439111 10210 raft_consensus.cc:493] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.439445 10210 raft_consensus.cc:3060] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.441742 10138 consensus_queue.cc:260] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:23.442993 10138 raft_consensus.cc:3060] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.452344 11209 raft_consensus.cc:993] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:23.452925 11209 raft_consensus.cc:1081] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:23.454579 10989 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f1aaf32c658c483eb6a4a3858745515f"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56726
I20260709 13:04:23.455156 10989 raft_consensus.cc:493] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.455511 10989 raft_consensus.cc:3060] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.456988 10210 raft_consensus.cc:515] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:23.459225 10210 leader_election.cc:290] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:23.428149 10987 leader_election.cc:290] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:23.461035 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea8a48dadff4881a7cdc0b27cf97db3" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:23.461652 10139 raft_consensus.cc:3060] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.449982 10211 raft_consensus.cc:2468] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:23.462568 10989 raft_consensus.cc:515] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:23.463500 10875 leader_election.cc:304] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577; no voters: 
I20260709 13:04:23.464517 10989 leader_election.cc:290] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Requested vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219)
I20260709 13:04:23.464748 11150 raft_consensus.cc:2804] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.465341 11150 raft_consensus.cc:697] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:23.466277 11150 consensus_queue.cc:237] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:23.466650 10214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1aaf32c658c483eb6a4a3858745515f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:23.467332 10214 raft_consensus.cc:3060] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.461071 10287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ea8a48dadff4881a7cdc0b27cf97db3" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:23.468304 10287 raft_consensus.cc:3055] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.468627 10287 raft_consensus.cc:740] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:23.470011 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f1aaf32c658c483eb6a4a3858745515f" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:23.470623 10285 raft_consensus.cc:3055] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.470322 10287 consensus_queue.cc:260] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:23.470921 10285 raft_consensus.cc:740] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8c3d3c9022c94705ac4ee834e1dc49d0, State: Running, Role: LEADER
I20260709 13:04:23.471630 10287 raft_consensus.cc:3060] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.471644 10285 consensus_queue.cc:260] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:23.502235 10285 raft_consensus.cc:3060] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.464725 10138 raft_consensus.cc:2468] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:23.518375 10285 raft_consensus.cc:2468] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:23.524935 10877 leader_election.cc:304] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 
I20260709 13:04:23.519117 10214 raft_consensus.cc:2468] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:23.519652 10139 raft_consensus.cc:2468] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 2.
I20260709 13:04:23.530582 10179 leader_election.cc:304] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcf45a2176234db5957dc38e32554577, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:23.520093 10287 raft_consensus.cc:2468] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bcf45a2176234db5957dc38e32554577 in term 2.
I20260709 13:04:23.529170 11149 raft_consensus.cc:2804] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.533130 11149 raft_consensus.cc:697] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:23.534087 11149 consensus_queue.cc:237] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:23.535094  9959 catalog_manager.cc:5697] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 1 to 2, leader changed from bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.544207 11135 raft_consensus.cc:2804] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.553375 11135 raft_consensus.cc:697] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 2 LEADER]: Becoming Leader. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:04:23.554684 11135 consensus_queue.cc:237] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } }
I20260709 13:04:23.578552  9959 catalog_manager.cc:5697] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 1 to 2, leader changed from 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:23.580080 11212 raft_consensus.cc:993] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:23.580612 11211 raft_consensus.cc:1081] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:23.599436 10987 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "940beae71dca43e58d3879c233f87dbb"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56738
I20260709 13:04:23.600086 10987 raft_consensus.cc:493] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:23.600471 10987 raft_consensus.cc:3060] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.607322  9959 catalog_manager.cc:5697] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 reported cstate change: term changed from 1 to 2, leader changed from 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196) to bcf45a2176234db5957dc38e32554577 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "bcf45a2176234db5957dc38e32554577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.609627 10987 raft_consensus.cc:515] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:23.611905 10987 leader_election.cc:290] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Requested vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:23.612957 10285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940beae71dca43e58d3879c233f87dbb" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0"
I20260709 13:04:23.613544 10139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "940beae71dca43e58d3879c233f87dbb" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:23.613725 10285 raft_consensus.cc:3060] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.614173 10139 raft_consensus.cc:3055] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:23.614539 10139 raft_consensus.cc:740] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bfa051e18d0549eabe79dfcc7610d99b, State: Running, Role: LEADER
I20260709 13:04:23.615335 10139 consensus_queue.cc:260] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:23.616603 10139 raft_consensus.cc:3060] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:23.621685 10285 raft_consensus.cc:2468] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:23.623328 10877 leader_election.cc:304] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 
I20260709 13:04:23.624194 11149 raft_consensus.cc:2804] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:23.624739 11149 raft_consensus.cc:697] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:23.625821 10139 raft_consensus.cc:2468] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:23.625733 11149 consensus_queue.cc:237] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:23.636960 10139 raft_consensus.cc:1275] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.638507 11149 consensus_queue.cc:1048] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.638140  9959 catalog_manager.cc:5697] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 1 to 2, leader changed from bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:23.659107 10288 raft_consensus.cc:1275] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.669478 11163 consensus_queue.cc:1048] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:23.885672 10139 raft_consensus.cc:1275] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.887228 11163 consensus_queue.cc:1048] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.897477 10214 raft_consensus.cc:1275] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.901760 11149 consensus_queue.cc:1048] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.933871 10288 raft_consensus.cc:1275] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.935097 11149 consensus_queue.cc:1048] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:23.948518 10214 raft_consensus.cc:1275] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:23.950057 11163 consensus_queue.cc:1048] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:24.026772 10287 raft_consensus.cc:1275] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Refusing update from remote peer bcf45a2176234db5957dc38e32554577: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.028169 11219 consensus_queue.cc:1048] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.040459 10138 raft_consensus.cc:1275] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer bcf45a2176234db5957dc38e32554577: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.042901 11135 consensus_queue.cc:1048] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.055231   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:24.098162 10138 raft_consensus.cc:1275] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.100426 11149 consensus_queue.cc:1048] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.111930 10287 raft_consensus.cc:1275] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:24.115509 11163 consensus_queue.cc:1048] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:24.131364   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:24.190809 10214 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "080320e522f941c9961ebf9de652365a"
dest_uuid: "bcf45a2176234db5957dc38e32554577"
mode: GRACEFUL
new_leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:59300
I20260709 13:04:24.191778 10214 raft_consensus.cc:606] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Received request to transfer leadership to TS 5e0edfcb63bb4d16ac123a82ecb7f331
I20260709 13:04:24.193744   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:04:24.195101   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:24.713593 11219 raft_consensus.cc:993] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577: : Instructing follower 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:24.714146 11135 raft_consensus.cc:1081] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Signalling peer 5e0edfcb63bb4d16ac123a82ecb7f331 to start an election
I20260709 13:04:24.715757 10995 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "080320e522f941c9961ebf9de652365a"
dest_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331"
 from {username='slave'} at 127.0.0.1:56744
I20260709 13:04:24.716239 10995 raft_consensus.cc:493] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:24.716526 10995 raft_consensus.cc:3060] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:24.721707 10995 raft_consensus.cc:515] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:24.724860 10995 leader_election.cc:290] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Requested vote from peers bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711), bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673)
I20260709 13:04:24.725313 10212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bcf45a2176234db5957dc38e32554577"
I20260709 13:04:24.726106 10212 raft_consensus.cc:3055] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:24.726485 10212 raft_consensus.cc:740] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bcf45a2176234db5957dc38e32554577, State: Running, Role: LEADER
I20260709 13:04:24.727415 10212 consensus_queue.cc:260] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:24.728814 10212 raft_consensus.cc:3060] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:24.728997 10138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "080320e522f941c9961ebf9de652365a" candidate_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa051e18d0549eabe79dfcc7610d99b"
I20260709 13:04:24.729708 10138 raft_consensus.cc:3060] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:24.737084 10138 raft_consensus.cc:2468] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:24.738301 10877 leader_election.cc:304] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b; no voters: 
I20260709 13:04:24.739552 11238 raft_consensus.cc:2804] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:24.740370 11238 raft_consensus.cc:697] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Becoming Leader. State: Replica: 5e0edfcb63bb4d16ac123a82ecb7f331, State: Running, Role: LEADER
I20260709 13:04:24.740819 10212 raft_consensus.cc:2468] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e0edfcb63bb4d16ac123a82ecb7f331 in term 2.
I20260709 13:04:24.741706 11238 consensus_queue.cc:237] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } }
I20260709 13:04:24.757334  9959 catalog_manager.cc:5697] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 reported cstate change: term changed from 1 to 2, leader changed from bcf45a2176234db5957dc38e32554577 (127.0.107.195) to 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:25.146730 10214 raft_consensus.cc:1275] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:25.148236 11238 consensus_queue.cc:1048] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:25.163944 10138 raft_consensus.cc:1275] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Refusing update from remote peer 5e0edfcb63bb4d16ac123a82ecb7f331: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:25.168816 11149 consensus_queue.cc:1048] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:25.196038   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:25.386183   431 tablet_server.cc:179] TabletServer@127.0.107.193:43611 shutting down...
W20260709 13:04:25.392498 10252 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 310 similar messages]
W20260709 13:04:25.403692 10252 consensus_peers.cc:597] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.412519 10104 consensus_peers.cc:597] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.437367 10104 consensus_peers.cc:597] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.453274 10178 consensus_peers.cc:597] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.454537   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:25.459482   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.460726   431 raft_consensus.cc:2243] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.461303   431 raft_consensus.cc:2272] T bdf60e84958749f9826c9e8de7d266d9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.464766   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.465780   431 raft_consensus.cc:2243] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.466374   431 raft_consensus.cc:2272] T b84c07107a48406d9fd585e9982d3069 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:25.466871 10104 consensus_peers.cc:597] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.469843   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:04:25.470886 10178 consensus_peers.cc:597] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.470927   431 raft_consensus.cc:2243] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.471650   431 raft_consensus.cc:2272] T e2ab3c2067a1463ca3f4319c644bd2f2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.475062   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.476035   431 raft_consensus.cc:2243] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.476624   431 raft_consensus.cc:2272] T 2c9785f60ee640d686f599267cf005fb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.480031   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.480957   431 raft_consensus.cc:2243] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.481525   431 raft_consensus.cc:2272] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.485189   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:04:25.485477 10178 consensus_peers.cc:597] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.486475   431 raft_consensus.cc:2243] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.487188   431 raft_consensus.cc:2272] T 711d81634fdc47aba353f79bcbfede48 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.490545   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.491580   431 raft_consensus.cc:2243] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.492193   431 raft_consensus.cc:2272] T d4c2e323c7ec414eb53a9c1622b81dcc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.495618   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.496531   431 raft_consensus.cc:2243] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.497251   431 raft_consensus.cc:2272] T ee9b66ba4b81428f896a77bd88bc7e4d P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.500609   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.501622   431 raft_consensus.cc:2243] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.502811   431 raft_consensus.cc:2272] T 1feaf9d0d9ca42c083b85e63a09d0081 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:25.504745 10104 consensus_peers.cc:597] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.506023   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.507205   431 raft_consensus.cc:2243] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.507898   431 raft_consensus.cc:2272] T ed4225af888b4e36ad1b03bdddc8f95f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:25.489955 10178 consensus_peers.cc:597] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.511319   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.512826   431 raft_consensus.cc:2243] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.514066   431 raft_consensus.cc:2272] T c1e9969e98ab4be4942e5d55666fd86f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.517539   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.518597   431 raft_consensus.cc:2243] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.519249   431 raft_consensus.cc:2272] T 60fa8e2d405c485dbf19dff403d2328f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.523854   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:04:25.524461 10252 consensus_peers.cc:597] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.524983   431 raft_consensus.cc:2243] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.525715   431 raft_consensus.cc:2272] T 63e94ff7589e401ba16e6019e0a4d6eb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:25.526114 10178 consensus_peers.cc:597] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.529139   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.530156   431 raft_consensus.cc:2243] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.531220   431 raft_consensus.cc:2272] T 0504e36ef8da48b189ebb41995e15efd P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.534480   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.535425   431 raft_consensus.cc:2243] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.536021   431 raft_consensus.cc:2272] T c29bf223e8224f72926bd39f6dcb3f08 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.539211   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.540082   431 raft_consensus.cc:2243] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.540647   431 raft_consensus.cc:2272] T 6ea0b03477ff4d1ab861dc093edbe68a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.543895   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.544780   431 raft_consensus.cc:2243] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Raft consensus shutting down.
W20260709 13:04:25.544808 10178 consensus_peers.cc:597] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.545972   431 raft_consensus.cc:2272] T 7c0cced7e11946f09a13c6adcf95833b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.549360   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.550313   431 raft_consensus.cc:2243] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.550817   431 raft_consensus.cc:2272] T 13fc2f8cffc54c41b5bdde0305fe868c P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.553982   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.554855   431 raft_consensus.cc:2243] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.555433   431 raft_consensus.cc:2272] T cd01deadf87945439c630e949d9c2e7b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.558923   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.559906   431 raft_consensus.cc:2243] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.560662   431 raft_consensus.cc:2272] T 5893ec967cb241af8709bba918f3429a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.563966   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.564922   431 raft_consensus.cc:2243] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.565572   431 raft_consensus.cc:2272] T 737f9858e8dc4fd488d562cac2d54aa4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.568971   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.569885   431 raft_consensus.cc:2243] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.570964   431 raft_consensus.cc:2272] T f1aaf32c658c483eb6a4a3858745515f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.573974   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.574867   431 raft_consensus.cc:2243] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.575416   431 raft_consensus.cc:2272] T a8bc0e52b9424f6dab141d4c1e28023b P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.578840   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.579542   431 raft_consensus.cc:2243] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.580053   431 raft_consensus.cc:2272] T dd372c208b444c4e9e4e02ee51c19601 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.582324   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.582901   431 raft_consensus.cc:2243] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.583319   431 raft_consensus.cc:2272] T 0065434c541d4f10a459f9003a463d86 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.585600   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.586222   431 raft_consensus.cc:2243] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.587088   431 raft_consensus.cc:2272] T 940beae71dca43e58d3879c233f87dbb P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.589990   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.590883   431 raft_consensus.cc:2243] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.591431   431 raft_consensus.cc:2272] T 3384978cd09b40f29b899f22ded46604 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.594717   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.595620   431 raft_consensus.cc:2243] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.596216   431 raft_consensus.cc:2272] T 8f0709ff33d24996ae638187a8e7c57f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:25.596683 10252 consensus_peers.cc:597] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.599524   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.600477   431 raft_consensus.cc:2243] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.601069   431 raft_consensus.cc:2272] T 1ffb355c77dc43e78812c2b2a2df92c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:25.602563 10104 consensus_peers.cc:597] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.604146   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.605124   431 raft_consensus.cc:2243] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:04:25.605191 10178 consensus_peers.cc:597] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.606303   431 raft_consensus.cc:2272] T 17c5f39847af4013aa501ce097f58b87 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:25.608536 10178 consensus_peers.cc:597] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.609328   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.610293   431 raft_consensus.cc:2243] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.611194   431 raft_consensus.cc:2272] T 080320e522f941c9961ebf9de652365a P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.614493   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.615087   431 raft_consensus.cc:2243] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.615552   431 raft_consensus.cc:2272] T 4d87b05b458741adb45ae4a1753447d0 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.618153   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.618772   431 raft_consensus.cc:2243] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.619191   431 raft_consensus.cc:2272] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.621698   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.622450   431 raft_consensus.cc:2243] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.622969   431 raft_consensus.cc:2272] T 16541bea7ded48538adada244223f2c9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.625999   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.626760   431 raft_consensus.cc:2243] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.627769   431 raft_consensus.cc:2272] T a68082d0468d49ada47812f6c8e3f106 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.630476   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.631179   431 raft_consensus.cc:2243] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.631682   431 raft_consensus.cc:2272] T aa60f11d5c674f3180f2616f97adf40f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.634639   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.635372   431 raft_consensus.cc:2243] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.635947   431 raft_consensus.cc:2272] T bdc9831f2c60444db097f11766636c56 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.639000   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.639753   431 raft_consensus.cc:2243] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.640651   431 raft_consensus.cc:2272] T 96acb559acc745428add2c682cf83cfc P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.643298   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.643942   431 raft_consensus.cc:2243] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.644781   431 raft_consensus.cc:2272] T dde8fc103c3d453abd4d298b1beffd74 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.647833   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.648949   431 raft_consensus.cc:2243] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:04:25.649184 10178 consensus_peers.cc:597] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.649765   431 raft_consensus.cc:2272] T 220a3db850dd4a5280bf7c8e9519d9d2 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.653115   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.653832   431 raft_consensus.cc:2243] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.654428   431 raft_consensus.cc:2272] T 7939918ac27c421f99a6747c1ea4a994 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.656842   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.657531   431 raft_consensus.cc:2243] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.658524   431 raft_consensus.cc:2272] T 01f3be65cb0c478194f88fb404dd329f P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.661578   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.662308   431 raft_consensus.cc:2243] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:25.662842   431 raft_consensus.cc:2272] T be75bbbdef504a5793ba0b07feaff6f8 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:25.664726 10252 consensus_peers.cc:597] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.665900   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.666673   431 raft_consensus.cc:2243] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.667687   431 raft_consensus.cc:2272] T 027075b60dc34541a40249671ad2fcc9 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:25.670714   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:25.671355   431 raft_consensus.cc:2243] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:25.672344   431 raft_consensus.cc:2272] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 5e0edfcb63bb4d16ac123a82ecb7f331 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:25.673535 10178 consensus_peers.cc:597] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.685341 10104 consensus_peers.cc:597] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.701236 10104 consensus_peers.cc:597] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.729553 10252 consensus_peers.cc:597] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.733551 10252 consensus_peers.cc:597] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.741569 10252 consensus_peers.cc:597] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.761363 10178 consensus_peers.cc:597] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.765514   431 tablet_server.cc:196] TabletServer@127.0.107.193:43611 shutdown complete.
W20260709 13:04:25.794862 10178 consensus_peers.cc:597] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.816293 10252 consensus_peers.cc:597] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.818660 10252 consensus_peers.cc:597] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:25.915887   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:04:25.963927 10103 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:37673, user_credentials={real_user=slave}} blocked reactor thread for 73904.9us
W20260709 13:04:25.972431 10253 consensus_peers.cc:597] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.972137 10252 consensus_peers.cc:597] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.979177 10104 consensus_peers.cc:597] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:25.986162 10253 consensus_peers.cc:597] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:26.007573 10104 consensus_peers.cc:597] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b -> Peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Couldn't send request to peer 5e0edfcb63bb4d16ac123a82ecb7f331. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:26.025413 10179 consensus_peers.cc:597] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:26.032289 10179 consensus_peers.cc:597] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:26.044378 10253 consensus_peers.cc:597] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.080260   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:26.081437   431 tablet_replica.cc:333] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.082527   431 raft_consensus.cc:2243] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.083489   431 raft_consensus.cc:2272] T b88ba89b2d1849848da78b9b3747e66e P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.086021   431 tablet_replica.cc:333] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.086793   431 raft_consensus.cc:2243] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.087884   431 raft_consensus.cc:2272] T 60fa8e2d405c485dbf19dff403d2328f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.090898   431 tablet_replica.cc:333] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.091806   431 raft_consensus.cc:2243] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.093007   431 raft_consensus.cc:2272] T a8bc0e52b9424f6dab141d4c1e28023b P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.096030   431 tablet_replica.cc:333] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.096877   431 raft_consensus.cc:2243] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.098623   431 raft_consensus.cc:2272] T 1feaf9d0d9ca42c083b85e63a09d0081 P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.101948   431 tablet_replica.cc:333] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.102988   431 raft_consensus.cc:2243] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.103631   431 raft_consensus.cc:2272] T 17c5f39847af4013aa501ce097f58b87 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.106957   431 tablet_replica.cc:333] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.108189   431 raft_consensus.cc:2243] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.108858   431 raft_consensus.cc:2272] T 027075b60dc34541a40249671ad2fcc9 P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.111089 10253 consensus_peers.cc:597] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.113924   431 tablet_replica.cc:333] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.115296   431 raft_consensus.cc:2243] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.116529   431 raft_consensus.cc:2272] T 63e94ff7589e401ba16e6019e0a4d6eb P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.121590   431 tablet_replica.cc:333] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.122583   431 raft_consensus.cc:2243] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.123790   431 raft_consensus.cc:2272] T c6314d5916384527a155e2c956145d98 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.127013   431 tablet_replica.cc:333] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.127985   431 raft_consensus.cc:2243] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.128641   431 raft_consensus.cc:2272] T 1ffb355c77dc43e78812c2b2a2df92c9 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.132032   431 tablet_replica.cc:333] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.133000   431 raft_consensus.cc:2243] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.133682   431 raft_consensus.cc:2272] T 13fc2f8cffc54c41b5bdde0305fe868c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.136847   431 tablet_replica.cc:333] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.137835   431 raft_consensus.cc:2243] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.138451   431 raft_consensus.cc:2272] T 737f9858e8dc4fd488d562cac2d54aa4 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.141497   431 tablet_replica.cc:333] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.142388   431 raft_consensus.cc:2243] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.143520   431 raft_consensus.cc:2272] T af4aeb2cefe7417e9c4364dd33b00847 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.146696   431 tablet_replica.cc:333] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
W20260709 13:04:26.147078 10253 consensus_peers.cc:597] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.147639   431 raft_consensus.cc:2243] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.148304   431 raft_consensus.cc:2272] T 01f3be65cb0c478194f88fb404dd329f P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.151331   431 tablet_replica.cc:333] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.152194   431 raft_consensus.cc:2243] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.152765   431 raft_consensus.cc:2272] T be75bbbdef504a5793ba0b07feaff6f8 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.155745   431 tablet_replica.cc:333] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.156703   431 raft_consensus.cc:2243] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.157346   431 raft_consensus.cc:2272] T a68082d0468d49ada47812f6c8e3f106 P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.160379   431 tablet_replica.cc:333] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.161134   431 raft_consensus.cc:2243] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.161716   431 raft_consensus.cc:2272] T 7c0cced7e11946f09a13c6adcf95833b P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.164467   431 tablet_replica.cc:333] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.165212   431 raft_consensus.cc:2243] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.166321   431 raft_consensus.cc:2272] T 6ea0b03477ff4d1ab861dc093edbe68a P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.169277   431 tablet_replica.cc:333] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.170127   431 raft_consensus.cc:2243] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.170764   431 raft_consensus.cc:2272] T 8f0709ff33d24996ae638187a8e7c57f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.172238 10179 consensus_peers.cc:597] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.173914   431 tablet_replica.cc:333] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.174832   431 raft_consensus.cc:2243] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.175439   431 raft_consensus.cc:2272] T aae810c860eb4b05a93e815248f988fc P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.178591   431 tablet_replica.cc:333] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.179530   431 raft_consensus.cc:2243] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.180644   431 raft_consensus.cc:2272] T 0d8d5b10e4fe4ce994c81bf45aa93591 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.183866   431 tablet_replica.cc:333] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
W20260709 13:04:26.183882 10179 consensus_peers.cc:597] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.184852   431 raft_consensus.cc:2243] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.185539   431 raft_consensus.cc:2272] T 0504e36ef8da48b189ebb41995e15efd P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.188680   431 tablet_replica.cc:333] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.189703   431 raft_consensus.cc:2243] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.190317   431 raft_consensus.cc:2272] T 22b9c093052a4414be72162a905b9a1d P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.193349   431 tablet_replica.cc:333] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.194365   431 raft_consensus.cc:2243] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.194998   431 raft_consensus.cc:2272] T 96acb559acc745428add2c682cf83cfc P bfa051e18d0549eabe79dfcc7610d99b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.198097   431 tablet_replica.cc:333] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.198949   431 raft_consensus.cc:2243] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.199549   431 raft_consensus.cc:2272] T dde8fc103c3d453abd4d298b1beffd74 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.202428   431 tablet_replica.cc:333] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.203336   431 raft_consensus.cc:2243] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.203930   431 raft_consensus.cc:2272] T 080320e522f941c9961ebf9de652365a P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.207324   431 tablet_replica.cc:333] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.208184   431 raft_consensus.cc:2243] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.209283   431 raft_consensus.cc:2272] T 2679daef0c4447c5823833ed4577441c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.209682 10179 consensus_peers.cc:597] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.212380   431 tablet_replica.cc:333] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.213405   431 raft_consensus.cc:2243] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.214562   431 raft_consensus.cc:2272] T 220a3db850dd4a5280bf7c8e9519d9d2 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.217931   431 tablet_replica.cc:333] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.218843   431 raft_consensus.cc:2243] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.219471   431 raft_consensus.cc:2272] T 7939918ac27c421f99a6747c1ea4a994 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.222400   431 tablet_replica.cc:333] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.223021   431 raft_consensus.cc:2243] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.223465   431 raft_consensus.cc:2272] T a9d3e10044ce408294e1b86ce960b84a P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.225616   431 tablet_replica.cc:333] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.226171   431 raft_consensus.cc:2243] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.227015   431 raft_consensus.cc:2272] T 20d07303bc2640fe9d6ea20f824ce00e P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.229187   431 tablet_replica.cc:333] T 16541bea7ded48538adada244223f2c9 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.229813   431 raft_consensus.cc:2243] T 16541bea7ded48538adada244223f2c9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.230298   431 raft_consensus.cc:2272] T 16541bea7ded48538adada244223f2c9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.232513   431 tablet_replica.cc:333] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.233095   431 raft_consensus.cc:2243] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.233603   431 raft_consensus.cc:2272] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.235728   431 tablet_replica.cc:333] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.236274   431 raft_consensus.cc:2243] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.237116   431 raft_consensus.cc:2272] T 711d81634fdc47aba353f79bcbfede48 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.239279   431 tablet_replica.cc:333] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.239835   431 raft_consensus.cc:2243] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.240675   431 raft_consensus.cc:2272] T 102b42f582ce49458990caefcc8d451c P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.243228   431 tablet_replica.cc:333] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.244045   431 raft_consensus.cc:2243] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.245154   431 raft_consensus.cc:2272] T ed4225af888b4e36ad1b03bdddc8f95f P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.248106   431 tablet_replica.cc:333] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.248968   431 raft_consensus.cc:2243] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.249553   431 raft_consensus.cc:2272] T 1e4d469562c14664860f49baa277f1d9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.250450 10179 consensus_peers.cc:597] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.252381   431 tablet_replica.cc:333] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.253286   431 raft_consensus.cc:2243] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.253854   431 raft_consensus.cc:2272] T ccc0007e852f46b2b3ce899d1101e17c P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.256593   431 tablet_replica.cc:333] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.257386   431 raft_consensus.cc:2243] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.258011   431 raft_consensus.cc:2272] T 8dc5cf0e36da48d0946769d2bb27c981 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.258131 10179 consensus_peers.cc:597] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.260867   431 tablet_replica.cc:333] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.261840   431 raft_consensus.cc:2243] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.263069   431 raft_consensus.cc:2272] T b6c597501a7d4e89a7a60fe4a56fcef8 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.266127   431 tablet_replica.cc:333] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.266933   431 raft_consensus.cc:2243] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.268091   431 raft_consensus.cc:2272] T c29bf223e8224f72926bd39f6dcb3f08 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.271345   431 tablet_replica.cc:333] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.272148   431 raft_consensus.cc:2243] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.272778   431 raft_consensus.cc:2272] T 3384978cd09b40f29b899f22ded46604 P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.274995 10253 consensus_peers.cc:597] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.276140   431 tablet_replica.cc:333] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.276988   431 raft_consensus.cc:2243] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.277586   431 raft_consensus.cc:2272] T 940beae71dca43e58d3879c233f87dbb P bfa051e18d0549eabe79dfcc7610d99b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.280599   431 tablet_replica.cc:333] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.281401   431 raft_consensus.cc:2243] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.282122   431 raft_consensus.cc:2272] T 0065434c541d4f10a459f9003a463d86 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.285312   431 tablet_replica.cc:333] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b: stopping tablet replica
I20260709 13:04:26.286217   431 raft_consensus.cc:2243] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.286902   431 raft_consensus.cc:2272] T bdf60e84958749f9826c9e8de7d266d9 P bfa051e18d0549eabe79dfcc7610d99b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.306339 10179 consensus_peers.cc:597] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:26.329581 10253 consensus_peers.cc:597] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:26.333487 10179 consensus_peers.cc:597] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.383237   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:04:26.439637 10179 consensus_peers.cc:597] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33711: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.491569   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:04:26.503907 10253 consensus_peers.cc:597] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Couldn't send request to peer bfa051e18d0549eabe79dfcc7610d99b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:33711: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:26.507392 10251 consensus_peers.cc:597] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.549813   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:26.550755   431 tablet_replica.cc:333] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.551854   431 raft_consensus.cc:2243] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.552492   431 raft_consensus.cc:2272] T 60fa8e2d405c485dbf19dff403d2328f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.554801   431 tablet_replica.cc:333] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.555382   431 raft_consensus.cc:2243] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.556195   431 raft_consensus.cc:2272] T 1ffb355c77dc43e78812c2b2a2df92c9 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.558211   431 tablet_replica.cc:333] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.558770   431 raft_consensus.cc:2243] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.559139   431 raft_consensus.cc:2272] T aa60f11d5c674f3180f2616f97adf40f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.561076   431 tablet_replica.cc:333] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.561721   431 raft_consensus.cc:2243] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.562129   431 raft_consensus.cc:2272] T bdc9831f2c60444db097f11766636c56 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.564175   431 tablet_replica.cc:333] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.564303 11265 raft_consensus.cc:493] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:26.564822   431 raft_consensus.cc:2243] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.565357   431 raft_consensus.cc:2272] T c6314d5916384527a155e2c956145d98 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.564949 11265 raft_consensus.cc:515] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:26.567953   431 tablet_replica.cc:333] T a8bc0e52b9424f6dab141d4c1e28023b P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.568018 10288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1e9969e98ab4be4942e5d55666fd86f" candidate_uuid: "bcf45a2176234db5957dc38e32554577" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" is_pre_election: true
I20260709 13:04:26.568295 11265 leader_election.cc:290] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8c3d3c9022c94705ac4ee834e1dc49d0 (127.0.107.196:42219), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:26.568933   431 raft_consensus.cc:2243] T a8bc0e52b9424f6dab141d4c1e28023b P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.569640   431 raft_consensus.cc:2272] T a8bc0e52b9424f6dab141d4c1e28023b P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.571609 10178 leader_election.cc:336] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:26.572154 10178 leader_election.cc:304] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bcf45a2176234db5957dc38e32554577; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, 8c3d3c9022c94705ac4ee834e1dc49d0
I20260709 13:04:26.572501   431 tablet_replica.cc:333] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.572891 11265 raft_consensus.cc:2749] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:26.573488   431 raft_consensus.cc:2243] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.574126   431 raft_consensus.cc:2272] T f1aaf32c658c483eb6a4a3858745515f P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.575024 10251 consensus_peers.cc:597] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.576717   431 tablet_replica.cc:333] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.577541   431 raft_consensus.cc:2243] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.578164   431 raft_consensus.cc:2272] T af4aeb2cefe7417e9c4364dd33b00847 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.580883   431 tablet_replica.cc:333] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.581802   431 raft_consensus.cc:2243] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.582405   431 raft_consensus.cc:2272] T aae810c860eb4b05a93e815248f988fc P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.584587   431 tablet_replica.cc:333] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.585172   431 raft_consensus.cc:2243] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.586021   431 raft_consensus.cc:2272] T b88ba89b2d1849848da78b9b3747e66e P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.588258   431 tablet_replica.cc:333] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.588858   431 raft_consensus.cc:2243] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.589898   431 raft_consensus.cc:2272] T 4d87b05b458741adb45ae4a1753447d0 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.592372   431 tablet_replica.cc:333] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.593113   431 raft_consensus.cc:2243] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.593921   431 raft_consensus.cc:2272] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.595921   431 tablet_replica.cc:333] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.596478   431 raft_consensus.cc:2243] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.597259   431 raft_consensus.cc:2272] T 5893ec967cb241af8709bba918f3429a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.599351   431 tablet_replica.cc:333] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.599912   431 raft_consensus.cc:2243] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.600723   431 raft_consensus.cc:2272] T 13fc2f8cffc54c41b5bdde0305fe868c P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.602910   431 tablet_replica.cc:333] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.603672   431 raft_consensus.cc:2243] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.604218   431 raft_consensus.cc:2272] T cd01deadf87945439c630e949d9c2e7b P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.606645   431 tablet_replica.cc:333] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.607291   431 raft_consensus.cc:2243] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.607720   431 raft_consensus.cc:2272] T 01f3be65cb0c478194f88fb404dd329f P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.609709   431 tablet_replica.cc:333] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.610255   431 raft_consensus.cc:2243] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.611056   431 raft_consensus.cc:2272] T be75bbbdef504a5793ba0b07feaff6f8 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.612920   431 tablet_replica.cc:333] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.613541   431 raft_consensus.cc:2243] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.614077   431 raft_consensus.cc:2272] T d4c2e323c7ec414eb53a9c1622b81dcc P bcf45a2176234db5957dc38e32554577 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.615983   431 tablet_replica.cc:333] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.616549   431 raft_consensus.cc:2243] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.617004   431 raft_consensus.cc:2272] T ee9b66ba4b81428f896a77bd88bc7e4d P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.619072   431 tablet_replica.cc:333] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.619761   431 raft_consensus.cc:2243] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.620189   431 raft_consensus.cc:2272] T 080320e522f941c9961ebf9de652365a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.622013   431 tablet_replica.cc:333] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.622635   431 raft_consensus.cc:2243] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.623085   431 raft_consensus.cc:2272] T 2679daef0c4447c5823833ed4577441c P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.625391   431 tablet_replica.cc:333] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.626068   431 raft_consensus.cc:2243] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.626860   431 raft_consensus.cc:2272] T b84c07107a48406d9fd585e9982d3069 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.628875   431 tablet_replica.cc:333] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.629474   431 raft_consensus.cc:2243] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.630265   431 raft_consensus.cc:2272] T 22b9c093052a4414be72162a905b9a1d P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.632241   431 tablet_replica.cc:333] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.632802   431 raft_consensus.cc:2243] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.633280   431 raft_consensus.cc:2272] T 7c0cced7e11946f09a13c6adcf95833b P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.635179   431 tablet_replica.cc:333] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.635744   431 raft_consensus.cc:2243] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.636180   431 raft_consensus.cc:2272] T 6ea0b03477ff4d1ab861dc093edbe68a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.638200   431 tablet_replica.cc:333] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.638725   431 raft_consensus.cc:2243] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.639218   431 raft_consensus.cc:2272] T 1e4d469562c14664860f49baa277f1d9 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.641971   431 tablet_replica.cc:333] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.642741   431 raft_consensus.cc:2243] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.643288   431 raft_consensus.cc:2272] T ccc0007e852f46b2b3ce899d1101e17c P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.645852   431 tablet_replica.cc:333] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.646634   431 raft_consensus.cc:2243] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.647207   431 raft_consensus.cc:2272] T e2ab3c2067a1463ca3f4319c644bd2f2 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.650138   431 tablet_replica.cc:333] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.650909   431 raft_consensus.cc:2243] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.651464   431 raft_consensus.cc:2272] T 8dc5cf0e36da48d0946769d2bb27c981 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.652813 10251 consensus_peers.cc:597] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.654055   431 tablet_replica.cc:333] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.654820   431 raft_consensus.cc:2243] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.655277   431 raft_consensus.cc:2272] T 102b42f582ce49458990caefcc8d451c P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.657366   431 tablet_replica.cc:333] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.657982   431 raft_consensus.cc:2243] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.658406   431 raft_consensus.cc:2272] T a9d3e10044ce408294e1b86ce960b84a P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.660429   431 tablet_replica.cc:333] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.660996   431 raft_consensus.cc:2243] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.661479   431 raft_consensus.cc:2272] T c1e9969e98ab4be4942e5d55666fd86f P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.663390   431 tablet_replica.cc:333] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.664007   431 raft_consensus.cc:2243] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.664505   431 raft_consensus.cc:2272] T 20d07303bc2640fe9d6ea20f824ce00e P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.667246   431 tablet_replica.cc:333] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.667920   431 raft_consensus.cc:2243] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.668936   431 raft_consensus.cc:2272] T 2c9785f60ee640d686f599267cf005fb P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.671273   431 tablet_replica.cc:333] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.671979   431 raft_consensus.cc:2243] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.672801   431 raft_consensus.cc:2272] T 9ea8a48dadff4881a7cdc0b27cf97db3 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.674831   431 tablet_replica.cc:333] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.675400   431 raft_consensus.cc:2243] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.676149   431 raft_consensus.cc:2272] T 8f0709ff33d24996ae638187a8e7c57f P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.678171   431 tablet_replica.cc:333] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.678649   431 raft_consensus.cc:2243] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.679343   431 raft_consensus.cc:2272] T bdf60e84958749f9826c9e8de7d266d9 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.681332   431 tablet_replica.cc:333] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.681932   431 raft_consensus.cc:2243] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.682379   431 raft_consensus.cc:2272] T 220a3db850dd4a5280bf7c8e9519d9d2 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.684461   431 tablet_replica.cc:333] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.685016   431 raft_consensus.cc:2243] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.685508   431 raft_consensus.cc:2272] T b6c597501a7d4e89a7a60fe4a56fcef8 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.687830   431 tablet_replica.cc:333] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.688539   431 raft_consensus.cc:2243] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.689069   431 raft_consensus.cc:2272] T c29bf223e8224f72926bd39f6dcb3f08 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.691924   431 tablet_replica.cc:333] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.692600   431 raft_consensus.cc:2243] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.693153   431 raft_consensus.cc:2272] T dd372c208b444c4e9e4e02ee51c19601 P bcf45a2176234db5957dc38e32554577 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.691670 11189 raft_consensus.cc:493] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:26.694792 11189 raft_consensus.cc:515] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:26.696262   431 tablet_replica.cc:333] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
W20260709 13:04:26.691609 10251 consensus_peers.cc:597] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.697111   431 raft_consensus.cc:2243] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.699940 11189 leader_election.cc:290] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:26.700598   431 raft_consensus.cc:2272] T 3384978cd09b40f29b899f22ded46604 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.703926   431 tablet_replica.cc:333] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
W20260709 13:04:26.704208 10252 leader_election.cc:336] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:26.704752   431 raft_consensus.cc:2243] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.705791   431 raft_consensus.cc:2272] T 0065434c541d4f10a459f9003a463d86 P bcf45a2176234db5957dc38e32554577 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.706209 10253 leader_election.cc:336] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Network error: Client connection negotiation failed: client connection to 127.0.107.194:33711: connect: Connection refused (error 111)
I20260709 13:04:26.706673 10253 leader_election.cc:304] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:26.707409 11189 raft_consensus.cc:2749] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:26.708498   431 tablet_replica.cc:333] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577: stopping tablet replica
I20260709 13:04:26.709091   431 raft_consensus.cc:2243] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.709584   431 raft_consensus.cc:2272] T 9a78dad3cd8b43a0bf32fc168eb162d4 P bcf45a2176234db5957dc38e32554577 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.727223 11189 raft_consensus.cc:493] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:26.727737 11189 raft_consensus.cc:515] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:26.729844 11189 leader_election.cc:290] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
W20260709 13:04:26.734153 10253 leader_election.cc:336] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Network error: Client connection negotiation failed: client connection to 127.0.107.194:33711: connect: Connection refused (error 111)
W20260709 13:04:26.734699 10252 leader_election.cc:336] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:26.735061 10252 leader_election.cc:304] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:26.735733 11189 raft_consensus.cc:2749] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:26.747753 11189 raft_consensus.cc:493] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:26.748297 11189 raft_consensus.cc:515] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:26.750399 11189 leader_election.cc:290] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
W20260709 13:04:26.754614 10253 leader_election.cc:336] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Network error: Client connection negotiation failed: client connection to 127.0.107.194:33711: connect: Connection refused (error 111)
W20260709 13:04:26.755134 10252 leader_election.cc:336] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:26.755510 10252 leader_election.cc:304] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:26.756132 11189 raft_consensus.cc:2749] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:26.760541 10251 consensus_peers.cc:597] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:26.770439 10251 consensus_peers.cc:597] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.770462 11263 raft_consensus.cc:493] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
W20260709 13:04:26.771332 10251 consensus_peers.cc:597] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.771219 11263 raft_consensus.cc:515] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:26.774341 11263 leader_election.cc:290] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
W20260709 13:04:26.779146 10253 leader_election.cc:336] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Network error: Client connection negotiation failed: client connection to 127.0.107.194:33711: connect: Connection refused (error 111)
W20260709 13:04:26.780514 10252 leader_election.cc:336] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:26.781005 10252 leader_election.cc:304] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:26.781761 11263 raft_consensus.cc:2749] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:26.787019 10251 consensus_peers.cc:597] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:37673: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.818816   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
W20260709 13:04:26.876253 10251 consensus_peers.cc:597] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:37673: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.890579   431 tablet_server.cc:179] TabletServer@127.0.107.196:0 shutting down...
I20260709 13:04:26.898509 11263 raft_consensus.cc:493] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:26.899180 11263 raft_consensus.cc:515] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:26.901574 11263 leader_election.cc:290] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
W20260709 13:04:26.907999 10252 leader_election.cc:336] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:26.908641 10253 leader_election.cc:336] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Network error: Client connection negotiation failed: client connection to 127.0.107.194:33711: connect: Connection refused (error 111)
I20260709 13:04:26.909204 10253 leader_election.cc:304] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:26.910483 11263 raft_consensus.cc:2749] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:04:26.914198 10251 consensus_peers.cc:597] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:37673: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.934063   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:26.934933   431 tablet_replica.cc:333] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.935606   431 raft_consensus.cc:2243] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.936271   431 raft_consensus.cc:2272] T 63e94ff7589e401ba16e6019e0a4d6eb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.939285   431 tablet_replica.cc:333] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.940279   431 raft_consensus.cc:2243] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.940618 11263 raft_consensus.cc:493] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:26.940963   431 raft_consensus.cc:2272] T c6314d5916384527a155e2c956145d98 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.941331 11263 raft_consensus.cc:515] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } } peers { permanent_uuid: "bfa051e18d0549eabe79dfcc7610d99b" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 33711 } }
I20260709 13:04:26.943682 11263 leader_election.cc:290] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611), bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711)
I20260709 13:04:26.944077   431 tablet_replica.cc:333] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.945024   431 raft_consensus.cc:2243] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.945652   431 raft_consensus.cc:2272] T 17c5f39847af4013aa501ce097f58b87 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.946296 10252 leader_election.cc:336] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
W20260709 13:04:26.947878 10253 leader_election.cc:336] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bfa051e18d0549eabe79dfcc7610d99b (127.0.107.194:33711): Network error: Client connection negotiation failed: client connection to 127.0.107.194:33711: connect: Connection refused (error 111)
I20260709 13:04:26.948210   431 tablet_replica.cc:333] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.948252 10253 leader_election.cc:304] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bfa051e18d0549eabe79dfcc7610d99b
I20260709 13:04:26.948973   431 raft_consensus.cc:2243] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.949060 11263 raft_consensus.cc:2749] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:26.949497   431 raft_consensus.cc:2272] T 027075b60dc34541a40249671ad2fcc9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.951578   431 tablet_replica.cc:333] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.952138   431 raft_consensus.cc:2243] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.952893   431 raft_consensus.cc:2272] T dd372c208b444c4e9e4e02ee51c19601 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.954849   431 tablet_replica.cc:333] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.955355   431 raft_consensus.cc:2243] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.955714   431 raft_consensus.cc:2272] T 940beae71dca43e58d3879c233f87dbb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.957656   431 tablet_replica.cc:333] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.958158   431 raft_consensus.cc:2243] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.958534 11263 raft_consensus.cc:493] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5e0edfcb63bb4d16ac123a82ecb7f331)
I20260709 13:04:26.958673   431 raft_consensus.cc:2272] T f1aaf32c658c483eb6a4a3858745515f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.959057 11263 raft_consensus.cc:515] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf45a2176234db5957dc38e32554577" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37673 } } peers { permanent_uuid: "8c3d3c9022c94705ac4ee834e1dc49d0" member_type: VOTER last_known_addr { host: "127.0.107.196" port: 42219 } } peers { permanent_uuid: "5e0edfcb63bb4d16ac123a82ecb7f331" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 43611 } }
I20260709 13:04:26.960922 11263 leader_election.cc:290] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673), 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611)
I20260709 13:04:26.961185   431 tablet_replica.cc:333] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.962036   431 raft_consensus.cc:2243] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.962510   431 raft_consensus.cc:2272] T af4aeb2cefe7417e9c4364dd33b00847 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.965406 10251 leader_election.cc:336] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Network error: Client connection negotiation failed: client connection to 127.0.107.195:37673: connect: Connection refused (error 111)
I20260709 13:04:26.965641   431 tablet_replica.cc:333] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
W20260709 13:04:26.966169 10252 leader_election.cc:336] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5e0edfcb63bb4d16ac123a82ecb7f331 (127.0.107.193:43611): Network error: Client connection negotiation failed: client connection to 127.0.107.193:43611: connect: Connection refused (error 111)
I20260709 13:04:26.966508 10252 leader_election.cc:304] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c3d3c9022c94705ac4ee834e1dc49d0; no voters: 5e0edfcb63bb4d16ac123a82ecb7f331, bcf45a2176234db5957dc38e32554577
I20260709 13:04:26.966512   431 raft_consensus.cc:2243] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.967182   431 raft_consensus.cc:2272] T b6c597501a7d4e89a7a60fe4a56fcef8 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.967223 11263 raft_consensus.cc:2749] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:26.969326   431 tablet_replica.cc:333] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.969997   431 raft_consensus.cc:2243] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.970831   431 raft_consensus.cc:2272] T 737f9858e8dc4fd488d562cac2d54aa4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.972802   431 tablet_replica.cc:333] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.973471   431 raft_consensus.cc:2243] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.973917   431 raft_consensus.cc:2272] T b88ba89b2d1849848da78b9b3747e66e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.975860   431 tablet_replica.cc:333] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.976464   431 raft_consensus.cc:2243] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.976883   431 raft_consensus.cc:2272] T 4d87b05b458741adb45ae4a1753447d0 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.978847   431 tablet_replica.cc:333] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.979410   431 raft_consensus.cc:2243] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.979856   431 raft_consensus.cc:2272] T 0c4f4fa6d20f4f1ca0b1befcc3a605b5 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.982048   431 tablet_replica.cc:333] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.982618   431 raft_consensus.cc:2243] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.983094   431 raft_consensus.cc:2272] T 102b42f582ce49458990caefcc8d451c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.985011   431 tablet_replica.cc:333] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.985644   431 raft_consensus.cc:2243] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.986235   431 raft_consensus.cc:2272] T 5893ec967cb241af8709bba918f3429a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.988332   431 tablet_replica.cc:333] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.989301   431 raft_consensus.cc:2243] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.990208   431 raft_consensus.cc:2272] T cd01deadf87945439c630e949d9c2e7b P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:26.992375   431 tablet_replica.cc:333] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.993119   431 raft_consensus.cc:2243] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:26.994794   431 raft_consensus.cc:2272] T aae810c860eb4b05a93e815248f988fc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:26.996619 10251 consensus_peers.cc:597] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:37673: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:26.997795   431 tablet_replica.cc:333] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:26.998782   431 raft_consensus.cc:2243] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:26.999452   431 raft_consensus.cc:2272] T 0d8d5b10e4fe4ce994c81bf45aa93591 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.001585   431 tablet_replica.cc:333] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.002190   431 raft_consensus.cc:2243] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.003068   431 raft_consensus.cc:2272] T aa60f11d5c674f3180f2616f97adf40f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.005210   431 tablet_replica.cc:333] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.005841   431 raft_consensus.cc:2243] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.006596   431 raft_consensus.cc:2272] T bdc9831f2c60444db097f11766636c56 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.008553   431 tablet_replica.cc:333] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.009160   431 raft_consensus.cc:2243] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.009614   431 raft_consensus.cc:2272] T a68082d0468d49ada47812f6c8e3f106 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.011492   431 tablet_replica.cc:333] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.012056   431 raft_consensus.cc:2243] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.012821   431 raft_consensus.cc:2272] T a9d3e10044ce408294e1b86ce960b84a P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.014925   431 tablet_replica.cc:333] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.015534   431 raft_consensus.cc:2243] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.015974   431 raft_consensus.cc:2272] T c1e9969e98ab4be4942e5d55666fd86f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.018254   431 tablet_replica.cc:333] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.018841   431 raft_consensus.cc:2243] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.019402   431 raft_consensus.cc:2272] T 20d07303bc2640fe9d6ea20f824ce00e P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.021807   431 tablet_replica.cc:333] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
W20260709 13:04:27.022517 10251 consensus_peers.cc:597] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 -> Peer bcf45a2176234db5957dc38e32554577 (127.0.107.195:37673): Couldn't send request to peer bcf45a2176234db5957dc38e32554577. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.195:37673: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:27.022605   431 raft_consensus.cc:2243] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.023298   431 raft_consensus.cc:2272] T b84c07107a48406d9fd585e9982d3069 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.025348   431 tablet_replica.cc:333] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.026087   431 raft_consensus.cc:2243] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.026535   431 raft_consensus.cc:2272] T 96acb559acc745428add2c682cf83cfc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.028515   431 tablet_replica.cc:333] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.029105   431 raft_consensus.cc:2243] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.029569   431 raft_consensus.cc:2272] T dde8fc103c3d453abd4d298b1beffd74 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.031495   431 tablet_replica.cc:333] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.032060   431 raft_consensus.cc:2243] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.032481   431 raft_consensus.cc:2272] T 2679daef0c4447c5823833ed4577441c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.034544   431 tablet_replica.cc:333] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.035089   431 raft_consensus.cc:2243] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.035835   431 raft_consensus.cc:2272] T 7939918ac27c421f99a6747c1ea4a994 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.037889   431 tablet_replica.cc:333] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.038417   431 raft_consensus.cc:2243] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.038784   431 raft_consensus.cc:2272] T 0504e36ef8da48b189ebb41995e15efd P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.040760   431 tablet_replica.cc:333] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.041272   431 raft_consensus.cc:2243] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.041706   431 raft_consensus.cc:2272] T 22b9c093052a4414be72162a905b9a1d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.043545   431 tablet_replica.cc:333] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.044054   431 raft_consensus.cc:2243] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.044404   431 raft_consensus.cc:2272] T 1feaf9d0d9ca42c083b85e63a09d0081 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.046314   431 tablet_replica.cc:333] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.046837   431 raft_consensus.cc:2243] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.047593   431 raft_consensus.cc:2272] T ee9b66ba4b81428f896a77bd88bc7e4d P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.049597   431 tablet_replica.cc:333] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.050078   431 raft_consensus.cc:2243] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.050770   431 raft_consensus.cc:2272] T d4c2e323c7ec414eb53a9c1622b81dcc P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.052630   431 tablet_replica.cc:333] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.053117   431 raft_consensus.cc:2243] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.053591   431 raft_consensus.cc:2272] T ed4225af888b4e36ad1b03bdddc8f95f P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.055586   431 tablet_replica.cc:333] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.056121   431 raft_consensus.cc:2243] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.056857   431 raft_consensus.cc:2272] T 1e4d469562c14664860f49baa277f1d9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.059422   431 tablet_replica.cc:333] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.060086   431 raft_consensus.cc:2243] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.060866   431 raft_consensus.cc:2272] T ccc0007e852f46b2b3ce899d1101e17c P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.063062   431 tablet_replica.cc:333] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.063546   431 raft_consensus.cc:2243] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.064244   431 raft_consensus.cc:2272] T e2ab3c2067a1463ca3f4319c644bd2f2 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.066361   431 tablet_replica.cc:333] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.066861   431 raft_consensus.cc:2243] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.067607   431 raft_consensus.cc:2272] T 8dc5cf0e36da48d0946769d2bb27c981 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.069600   431 tablet_replica.cc:333] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.070089   431 raft_consensus.cc:2243] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.070451   431 raft_consensus.cc:2272] T 2c9785f60ee640d686f599267cf005fb P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.072325   431 tablet_replica.cc:333] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.072786   431 raft_consensus.cc:2243] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.073137   431 raft_consensus.cc:2272] T 9ea8a48dadff4881a7cdc0b27cf97db3 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.075073   431 tablet_replica.cc:333] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.075549   431 raft_consensus.cc:2243] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.076227   431 raft_consensus.cc:2272] T 16541bea7ded48538adada244223f2c9 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.078191   431 tablet_replica.cc:333] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.078640   431 raft_consensus.cc:2243] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.079080   431 raft_consensus.cc:2272] T 711d81634fdc47aba353f79bcbfede48 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.081017   431 tablet_replica.cc:333] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0: stopping tablet replica
I20260709 13:04:27.081593   431 raft_consensus.cc:2243] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:27.082050   431 raft_consensus.cc:2272] T 9a78dad3cd8b43a0bf32fc168eb162d4 P 8c3d3c9022c94705ac4ee834e1dc49d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.187476   431 tablet_server.cc:196] TabletServer@127.0.107.196:0 shutdown complete.
I20260709 13:04:27.252151   431 master.cc:562] Master@127.0.107.254:44803 shutting down...
I20260709 13:04:27.288236   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:27.288827   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:27.289146   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 51ac909067f74d7e8fa4b33a1a8523c7: stopping tablet replica
I20260709 13:04:27.311498   431 master.cc:584] Master@127.0.107.254:44803 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (34988 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:04:27.403026   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:38461
I20260709 13:04:27.404138   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:27.410214 11267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:27.410233 11268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:27.411625 11270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:27.414330   431 server_base.cc:1061] running on GCE node
I20260709 13:04:27.415565   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:27.415766   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:27.415884   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602267415872 us; error 0 us; skew 500 ppm
I20260709 13:04:27.416389   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:27.419301   431 webserver.cc:533] Webserver started at http://127.0.107.254:45445/ using document root <none> and password file <none>
I20260709 13:04:27.419800   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:27.419970   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:27.420192   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:27.421399   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/master-0-root/instance:
uuid: "94f0028b98c7416eb5e86fe165dc3e20"
format_stamp: "Formatted at 2026-07-09 13:04:27 on dist-test-slave-jk05"
I20260709 13:04:27.426231   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 13:04:27.429916 11275 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:27.430707   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:04:27.430986   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/master-0-root
uuid: "94f0028b98c7416eb5e86fe165dc3e20"
format_stamp: "Formatted at 2026-07-09 13:04:27 on dist-test-slave-jk05"
I20260709 13:04:27.431249   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:27.455367   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:27.456583   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:27.495877   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:38461
I20260709 13:04:27.495973 11326 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:38461 every 8 connection(s)
I20260709 13:04:27.499989 11327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:27.511102 11327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20: Bootstrap starting.
I20260709 13:04:27.516093 11327 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:27.520582 11327 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20: No bootstrap required, opened a new log
I20260709 13:04:27.523010 11327 raft_consensus.cc:359] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94f0028b98c7416eb5e86fe165dc3e20" member_type: VOTER }
I20260709 13:04:27.523394 11327 raft_consensus.cc:385] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:27.523574 11327 raft_consensus.cc:740] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94f0028b98c7416eb5e86fe165dc3e20, State: Initialized, Role: FOLLOWER
I20260709 13:04:27.524097 11327 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94f0028b98c7416eb5e86fe165dc3e20" member_type: VOTER }
I20260709 13:04:27.524506 11327 raft_consensus.cc:399] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:27.524694 11327 raft_consensus.cc:493] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:27.524907 11327 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:27.529877 11327 raft_consensus.cc:515] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94f0028b98c7416eb5e86fe165dc3e20" member_type: VOTER }
I20260709 13:04:27.530396 11327 leader_election.cc:304] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94f0028b98c7416eb5e86fe165dc3e20; no voters: 
I20260709 13:04:27.531525 11327 leader_election.cc:290] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:27.531924 11330 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:27.533538 11330 raft_consensus.cc:697] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [term 1 LEADER]: Becoming Leader. State: Replica: 94f0028b98c7416eb5e86fe165dc3e20, State: Running, Role: LEADER
I20260709 13:04:27.534382 11330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94f0028b98c7416eb5e86fe165dc3e20" member_type: VOTER }
I20260709 13:04:27.535177 11327 sys_catalog.cc:565] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:27.538957 11331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94f0028b98c7416eb5e86fe165dc3e20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94f0028b98c7416eb5e86fe165dc3e20" member_type: VOTER } }
I20260709 13:04:27.539736 11331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:27.539906 11332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94f0028b98c7416eb5e86fe165dc3e20. Latest consensus state: current_term: 1 leader_uuid: "94f0028b98c7416eb5e86fe165dc3e20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94f0028b98c7416eb5e86fe165dc3e20" member_type: VOTER } }
I20260709 13:04:27.540521 11332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:27.542920 11337 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:27.551266 11337 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:27.552251   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:27.560583 11337 catalog_manager.cc:1383] Generated new cluster ID: 1067ae938ebd45e68d3bb23be25f5ecd
I20260709 13:04:27.561079 11337 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:27.587077 11337 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:27.588414 11337 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:27.605111 11337 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20: Generated new TSK 0
I20260709 13:04:27.605789 11337 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:27.619700   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:27.626025 11348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:27.627094 11349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:27.629392 11352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:27.629530   431 server_base.cc:1061] running on GCE node
I20260709 13:04:27.630915   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:27.631119   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:27.631332   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602267631297 us; error 0 us; skew 500 ppm
I20260709 13:04:27.631901   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:27.634583   431 webserver.cc:533] Webserver started at http://127.0.107.193:45027/ using document root <none> and password file <none>
I20260709 13:04:27.635097   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:27.635273   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:27.635604   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:27.636922   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-0-root/instance:
uuid: "dc3230c8b0704c5ab54fd82d58545db2"
format_stamp: "Formatted at 2026-07-09 13:04:27 on dist-test-slave-jk05"
I20260709 13:04:27.641772   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:04:27.645355 11356 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:27.646338   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:04:27.646736   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-0-root
uuid: "dc3230c8b0704c5ab54fd82d58545db2"
format_stamp: "Formatted at 2026-07-09 13:04:27 on dist-test-slave-jk05"
I20260709 13:04:27.647034   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:27.662931   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:27.664278   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:27.665850   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:27.668267   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:27.668510   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:27.668793   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:27.668979   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:27.710973   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:38057
I20260709 13:04:27.711073 11418 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:38057 every 8 connection(s)
I20260709 13:04:27.716728   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:27.724135 11423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:27.726150 11424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:27.729305 11426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:27.729820   431 server_base.cc:1061] running on GCE node
I20260709 13:04:27.731194   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:27.731439   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:27.731631   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602267731611 us; error 0 us; skew 500 ppm
I20260709 13:04:27.732393   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:27.733850 11419 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38461
I20260709 13:04:27.734315 11419 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:27.735100 11419 heartbeater.cc:507] Master 127.0.107.254:38461 requested a full tablet report, sending...
I20260709 13:04:27.735616   431 webserver.cc:533] Webserver started at http://127.0.107.194:33981/ using document root <none> and password file <none>
I20260709 13:04:27.736295   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:27.736575   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:27.736931   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:27.737614 11292 ts_manager.cc:194] Registered new tserver with Master: dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057)
I20260709 13:04:27.738366   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-1-root/instance:
uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
format_stamp: "Formatted at 2026-07-09 13:04:27 on dist-test-slave-jk05"
I20260709 13:04:27.739617 11292 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44144
I20260709 13:04:27.743865   431 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:04:27.747843 11431 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:27.748971   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:04:27.749328   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-1-root
uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
format_stamp: "Formatted at 2026-07-09 13:04:27 on dist-test-slave-jk05"
I20260709 13:04:27.749650   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:27.776839   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:27.778455   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:27.780099   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:27.782699   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:27.782907   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:27.783221   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:27.783370   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:27.825328   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:45491
I20260709 13:04:27.825495 11493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:45491 every 8 connection(s)
I20260709 13:04:27.830327   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:27.837837 11497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:27.841866 11498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:27.844408 11494 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38461
I20260709 13:04:27.844954 11494 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:27.845289   431 server_base.cc:1061] running on GCE node
W20260709 13:04:27.845361 11500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:27.846010 11494 heartbeater.cc:507] Master 127.0.107.254:38461 requested a full tablet report, sending...
I20260709 13:04:27.846688   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:27.846995   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:27.847186   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602267847171 us; error 0 us; skew 500 ppm
I20260709 13:04:27.847898   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:27.848457 11292 ts_manager.cc:194] Registered new tserver with Master: a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:27.850607 11292 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44150
I20260709 13:04:27.850996   431 webserver.cc:533] Webserver started at http://127.0.107.195:34189/ using document root <none> and password file <none>
I20260709 13:04:27.851716   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:27.851943   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:27.852226   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:27.853384   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-2-root/instance:
uuid: "3ece719b875a475eaa0c79ffde68cb20"
format_stamp: "Formatted at 2026-07-09 13:04:27 on dist-test-slave-jk05"
I20260709 13:04:27.858125   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:04:27.861732 11505 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:27.862545   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:04:27.862874   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-2-root
uuid: "3ece719b875a475eaa0c79ffde68cb20"
format_stamp: "Formatted at 2026-07-09 13:04:27 on dist-test-slave-jk05"
I20260709 13:04:27.863171   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:27.892148   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:27.893391   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:27.894980   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:27.897334   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:27.897606   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:27.897873   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:27.898049   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:27.940968   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:46191
I20260709 13:04:27.941082 11567 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:46191 every 8 connection(s)
I20260709 13:04:27.955754 11568 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38461
I20260709 13:04:27.956144 11568 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:27.956913 11568 heartbeater.cc:507] Master 127.0.107.254:38461 requested a full tablet report, sending...
I20260709 13:04:27.958989 11292 ts_manager.cc:194] Registered new tserver with Master: 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:27.959157   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015157119s
I20260709 13:04:27.960496 11292 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44154
I20260709 13:04:27.990773 11292 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44164:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:27.993934 11292 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:28.071884 11384 tablet_service.cc:1511] Processing CreateTablet for tablet 92cd1ee5127a4f578b1f2f9125999904 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:04:28.072685 11457 tablet_service.cc:1511] Processing CreateTablet for tablet f4891fec6e634dc7ade7db078a5f3bab (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:04:28.073616 11384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92cd1ee5127a4f578b1f2f9125999904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.074182 11457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4891fec6e634dc7ade7db078a5f3bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.078608 11456 tablet_service.cc:1511] Processing CreateTablet for tablet fd340b80e0bf42e080b36269ec8b5c4a (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:04:28.079378 11383 tablet_service.cc:1511] Processing CreateTablet for tablet f7a03a51a3ed458baa6e9dbc849793cc (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:04:28.080253 11456 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd340b80e0bf42e080b36269ec8b5c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.080780 11383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7a03a51a3ed458baa6e9dbc849793cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.082785 11458 tablet_service.cc:1511] Processing CreateTablet for tablet f7a03a51a3ed458baa6e9dbc849793cc (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:04:28.084306 11458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7a03a51a3ed458baa6e9dbc849793cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.086686 11459 tablet_service.cc:1511] Processing CreateTablet for tablet 92cd1ee5127a4f578b1f2f9125999904 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:04:28.088198 11459 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92cd1ee5127a4f578b1f2f9125999904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.095938 11381 tablet_service.cc:1511] Processing CreateTablet for tablet fd340b80e0bf42e080b36269ec8b5c4a (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:04:28.097615 11381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd340b80e0bf42e080b36269ec8b5c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.098306 11533 tablet_service.cc:1511] Processing CreateTablet for tablet 92cd1ee5127a4f578b1f2f9125999904 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:04:28.099821 11533 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92cd1ee5127a4f578b1f2f9125999904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.100715 11380 tablet_service.cc:1511] Processing CreateTablet for tablet bc846bd430d248fd812392acd98e9ac1 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:04:28.102260 11380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc846bd430d248fd812392acd98e9ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.104369 11455 tablet_service.cc:1511] Processing CreateTablet for tablet bc846bd430d248fd812392acd98e9ac1 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:04:28.106678 11454 tablet_service.cc:1511] Processing CreateTablet for tablet d4cc664f62b248ec92a2cef483c2c665 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:04:28.110018 11453 tablet_service.cc:1511] Processing CreateTablet for tablet 1f216ed20c2043bf82cedeeeab582b59 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:04:28.110930 11379 tablet_service.cc:1511] Processing CreateTablet for tablet d4cc664f62b248ec92a2cef483c2c665 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:04:28.108856 11382 tablet_service.cc:1511] Processing CreateTablet for tablet f4891fec6e634dc7ade7db078a5f3bab (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:04:28.111474 11452 tablet_service.cc:1511] Processing CreateTablet for tablet fb34dc48d7194d7bb15ecb7094dabf43 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:04:28.112461 11379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4cc664f62b248ec92a2cef483c2c665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.113099 11455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc846bd430d248fd812392acd98e9ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.110950 11377 tablet_service.cc:1511] Processing CreateTablet for tablet fb34dc48d7194d7bb15ecb7094dabf43 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:04:28.116997 11382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4891fec6e634dc7ade7db078a5f3bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.117030 11378 tablet_service.cc:1511] Processing CreateTablet for tablet 1f216ed20c2043bf82cedeeeab582b59 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:04:28.118546 11378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f216ed20c2043bf82cedeeeab582b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.118364 11532 tablet_service.cc:1511] Processing CreateTablet for tablet f7a03a51a3ed458baa6e9dbc849793cc (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:04:28.119884 11532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7a03a51a3ed458baa6e9dbc849793cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.124387 11531 tablet_service.cc:1511] Processing CreateTablet for tablet f4891fec6e634dc7ade7db078a5f3bab (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:04:28.125980 11531 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4891fec6e634dc7ade7db078a5f3bab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.128561 11453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f216ed20c2043bf82cedeeeab582b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.129022 11377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb34dc48d7194d7bb15ecb7094dabf43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.150126 11530 tablet_service.cc:1511] Processing CreateTablet for tablet fd340b80e0bf42e080b36269ec8b5c4a (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:04:28.152228 11530 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd340b80e0bf42e080b36269ec8b5c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.155318 11528 tablet_service.cc:1511] Processing CreateTablet for tablet d4cc664f62b248ec92a2cef483c2c665 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:04:28.159371 11529 tablet_service.cc:1511] Processing CreateTablet for tablet bc846bd430d248fd812392acd98e9ac1 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:04:28.159371 11527 tablet_service.cc:1511] Processing CreateTablet for tablet 1f216ed20c2043bf82cedeeeab582b59 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:04:28.162828 11528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4cc664f62b248ec92a2cef483c2c665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.164364 11452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb34dc48d7194d7bb15ecb7094dabf43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.165741 11583 tablet_bootstrap.cc:492] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:28.166510 11527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f216ed20c2043bf82cedeeeab582b59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.165833 11454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4cc664f62b248ec92a2cef483c2c665. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.172145 11529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc846bd430d248fd812392acd98e9ac1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.198278 11526 tablet_service.cc:1511] Processing CreateTablet for tablet fb34dc48d7194d7bb15ecb7094dabf43 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:04:28.199949 11526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb34dc48d7194d7bb15ecb7094dabf43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.204384 11384 tablet_service.cc:1511] Processing CreateTablet for tablet 92b5d1a8d996415982df5ed08c242522 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:04:28.205865 11384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92b5d1a8d996415982df5ed08c242522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.210537 11457 tablet_service.cc:1511] Processing CreateTablet for tablet 92b5d1a8d996415982df5ed08c242522 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:04:28.212023 11457 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92b5d1a8d996415982df5ed08c242522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.214597 11525 tablet_service.cc:1511] Processing CreateTablet for tablet 92b5d1a8d996415982df5ed08c242522 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:04:28.216133 11525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92b5d1a8d996415982df5ed08c242522. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.220072 11377 tablet_service.cc:1511] Processing CreateTablet for tablet a2130ecf60b94295aff5ea3f1aeafd98 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:04:28.222045 11377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2130ecf60b94295aff5ea3f1aeafd98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.239279 11583 tablet_bootstrap.cc:654] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.243600 11458 tablet_service.cc:1511] Processing CreateTablet for tablet a2130ecf60b94295aff5ea3f1aeafd98 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:04:28.245137 11458 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2130ecf60b94295aff5ea3f1aeafd98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.262940 11585 tablet_bootstrap.cc:492] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20: Bootstrap starting.
I20260709 13:04:28.270087 11584 tablet_bootstrap.cc:492] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27: Bootstrap starting.
I20260709 13:04:28.290674 11532 tablet_service.cc:1511] Processing CreateTablet for tablet a2130ecf60b94295aff5ea3f1aeafd98 (DEFAULT_TABLE table=test-workload [id=cd2a9dd6a0c242c9b26fe7d9f4515d1e]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:04:28.292327 11532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2130ecf60b94295aff5ea3f1aeafd98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:28.294097 11585 tablet_bootstrap.cc:654] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.309955 11583 tablet_bootstrap.cc:492] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2: No bootstrap required, opened a new log
I20260709 13:04:28.310600 11583 ts_tablet_manager.cc:1403] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.145s	user 0.028s	sys 0.009s
I20260709 13:04:28.317108 11584 tablet_bootstrap.cc:654] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.337133 11583 raft_consensus.cc:359] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.337880 11584 tablet_bootstrap.cc:492] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27: No bootstrap required, opened a new log
I20260709 13:04:28.338044 11583 raft_consensus.cc:385] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.338610 11583 raft_consensus.cc:740] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.338832 11584 ts_tablet_manager.cc:1403] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27: Time spent bootstrapping tablet: real 0.069s	user 0.010s	sys 0.005s
I20260709 13:04:28.339514 11583 consensus_queue.cc:260] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.342829 11583 ts_tablet_manager.cc:1434] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.003s
I20260709 13:04:28.342995 11585 tablet_bootstrap.cc:492] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20: No bootstrap required, opened a new log
I20260709 13:04:28.342825 11584 raft_consensus.cc:359] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.343895 11585 ts_tablet_manager.cc:1403] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20: Time spent bootstrapping tablet: real 0.081s	user 0.022s	sys 0.009s
I20260709 13:04:28.344103 11584 raft_consensus.cc:385] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.344401 11583 tablet_bootstrap.cc:492] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:28.344610 11584 raft_consensus.cc:740] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.345674 11584 consensus_queue.cc:260] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
W20260709 13:04:28.379457 11278 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.0.107.195:46191, user_credentials={real_user=slave}} blocked reactor thread for 56898.9us
I20260709 13:04:28.382531 11419 heartbeater.cc:499] Master 127.0.107.254:38461 was elected leader, sending a full tablet report...
I20260709 13:04:28.389448 11585 raft_consensus.cc:359] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.391932 11585 raft_consensus.cc:385] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.392675 11585 raft_consensus.cc:740] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.394410 11585 consensus_queue.cc:260] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.417070 11584 ts_tablet_manager.cc:1434] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27: Time spent starting tablet: real 0.078s	user 0.012s	sys 0.003s
I20260709 13:04:28.419269 11494 heartbeater.cc:499] Master 127.0.107.254:38461 was elected leader, sending a full tablet report...
I20260709 13:04:28.421350 11585 ts_tablet_manager.cc:1434] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20: Time spent starting tablet: real 0.077s	user 0.011s	sys 0.025s
I20260709 13:04:28.424260 11584 tablet_bootstrap.cc:492] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27: Bootstrap starting.
I20260709 13:04:28.429771 11568 heartbeater.cc:499] Master 127.0.107.254:38461 was elected leader, sending a full tablet report...
I20260709 13:04:28.439515 11583 tablet_bootstrap.cc:654] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.442847 11585 tablet_bootstrap.cc:492] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20: Bootstrap starting.
W20260709 13:04:28.445465 11569 tablet.cc:2406] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:04:28.458336 11584 tablet_bootstrap.cc:654] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.464236 11585 tablet_bootstrap.cc:654] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.466634 11583 tablet_bootstrap.cc:492] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2: No bootstrap required, opened a new log
I20260709 13:04:28.466475 11592 raft_consensus.cc:493] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.467675 11583 ts_tablet_manager.cc:1403] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.124s	user 0.012s	sys 0.018s
I20260709 13:04:28.467507 11592 raft_consensus.cc:515] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.472782 11583 raft_consensus.cc:359] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.473750 11583 raft_consensus.cc:385] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.474126 11583 raft_consensus.cc:740] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.476655 11592 leader_election.cc:290] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057)
I20260709 13:04:28.481491 11585 tablet_bootstrap.cc:492] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20: No bootstrap required, opened a new log
I20260709 13:04:28.482355 11585 ts_tablet_manager.cc:1403] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.007s
I20260709 13:04:28.484196 11591 raft_consensus.cc:493] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.485172 11591 raft_consensus.cc:515] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.479991 11583 consensus_queue.cc:260] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.499624 11591 leader_election.cc:290] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:28.503640 11583 ts_tablet_manager.cc:1434] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.035s	user 0.012s	sys 0.001s
I20260709 13:04:28.505263 11583 tablet_bootstrap.cc:492] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:28.506384 11585 raft_consensus.cc:359] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.507387 11585 raft_consensus.cc:385] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.507638 11585 raft_consensus.cc:740] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.515506 11583 tablet_bootstrap.cc:654] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.516218 11585 consensus_queue.cc:260] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.519402 11585 ts_tablet_manager.cc:1434] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 13:04:28.520345 11584 tablet_bootstrap.cc:492] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27: No bootstrap required, opened a new log
I20260709 13:04:28.523409 11585 tablet_bootstrap.cc:492] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20: Bootstrap starting.
I20260709 13:04:28.529337 11584 ts_tablet_manager.cc:1403] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent bootstrapping tablet: real 0.106s	user 0.030s	sys 0.004s
I20260709 13:04:28.536530 11585 tablet_bootstrap.cc:654] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.539215 11584 raft_consensus.cc:359] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.540329 11584 raft_consensus.cc:385] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.540690 11584 raft_consensus.cc:740] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.541661 11584 consensus_queue.cc:260] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.544134 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a03a51a3ed458baa6e9dbc849793cc" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2" is_pre_election: true
I20260709 13:04:28.545145 11394 raft_consensus.cc:2468] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 0.
I20260709 13:04:28.545706 11584 ts_tablet_manager.cc:1434] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 13:04:28.547015 11584 tablet_bootstrap.cc:492] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27: Bootstrap starting.
W20260709 13:04:28.549837 11340 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:28.556668 11434 leader_election.cc:304] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:28.557976 11591 raft_consensus.cc:2804] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:28.558385 11591 raft_consensus.cc:493] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.558733 11591 raft_consensus.cc:3060] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.559371 11585 tablet_bootstrap.cc:492] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20: No bootstrap required, opened a new log
I20260709 13:04:28.560041 11585 ts_tablet_manager.cc:1403] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.002s
I20260709 13:04:28.560262 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92cd1ee5127a4f578b1f2f9125999904" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
I20260709 13:04:28.561223 11469 raft_consensus.cc:2468] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 0.
I20260709 13:04:28.561687 11584 tablet_bootstrap.cc:654] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.563117 11508 leader_election.cc:304] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: 
I20260709 13:04:28.563671 11585 raft_consensus.cc:359] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.564190 11592 raft_consensus.cc:2804] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:28.564510 11585 raft_consensus.cc:385] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.564637 11592 raft_consensus.cc:493] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.564953 11585 raft_consensus.cc:740] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.565371 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92cd1ee5127a4f578b1f2f9125999904" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2" is_pre_election: true
I20260709 13:04:28.566220 11394 raft_consensus.cc:2468] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 0.
I20260709 13:04:28.565313 11592 raft_consensus.cc:3060] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.567991 11585 consensus_queue.cc:260] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.569123 11583 tablet_bootstrap.cc:492] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2: No bootstrap required, opened a new log
I20260709 13:04:28.569649 11583 ts_tablet_manager.cc:1403] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.007s
I20260709 13:04:28.571405 11585 ts_tablet_manager.cc:1434] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.008s
I20260709 13:04:28.572728 11585 tablet_bootstrap.cc:492] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20: Bootstrap starting.
I20260709 13:04:28.573132 11583 raft_consensus.cc:359] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.573976 11583 raft_consensus.cc:385] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.574278 11583 raft_consensus.cc:740] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.575366 11583 consensus_queue.cc:260] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.578166 11583 ts_tablet_manager.cc:1434] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:04:28.579535 11583 tablet_bootstrap.cc:492] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:28.581007 11585 tablet_bootstrap.cc:654] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.594002 11585 tablet_bootstrap.cc:492] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20: No bootstrap required, opened a new log
I20260709 13:04:28.594574 11585 ts_tablet_manager.cc:1403] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.007s
I20260709 13:04:28.595801 11583 tablet_bootstrap.cc:654] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.597915 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a03a51a3ed458baa6e9dbc849793cc" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:28.598037 11585 raft_consensus.cc:359] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.598865 11543 raft_consensus.cc:2468] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 0.
I20260709 13:04:28.598883 11585 raft_consensus.cc:385] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.600596 11585 raft_consensus.cc:740] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.601596 11585 consensus_queue.cc:260] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.604136 11585 ts_tablet_manager.cc:1434] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:04:28.605480 11585 tablet_bootstrap.cc:492] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20: Bootstrap starting.
I20260709 13:04:28.608878 11591 raft_consensus.cc:515] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.609751 11583 tablet_bootstrap.cc:492] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: No bootstrap required, opened a new log
I20260709 13:04:28.614135 11583 ts_tablet_manager.cc:1403] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.035s	user 0.008s	sys 0.011s
I20260709 13:04:28.612483 11592 raft_consensus.cc:515] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.617118 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92cd1ee5127a4f578b1f2f9125999904" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:28.617887 11469 raft_consensus.cc:3060] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.621163 11585 tablet_bootstrap.cc:654] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.622138 11583 raft_consensus.cc:359] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.623298 11583 raft_consensus.cc:385] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.623716 11583 raft_consensus.cc:740] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.627161 11583 consensus_queue.cc:260] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.629963 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a03a51a3ed458baa6e9dbc849793cc" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:28.630607 11394 raft_consensus.cc:3060] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.632513 11583 ts_tablet_manager.cc:1434] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.001s
I20260709 13:04:28.638583 11583 tablet_bootstrap.cc:492] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:28.636941 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a03a51a3ed458baa6e9dbc849793cc" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:28.637131 11592 leader_election.cc:290] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 election: Requested vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057)
I20260709 13:04:28.638309 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92cd1ee5127a4f578b1f2f9125999904" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:28.640640 11393 raft_consensus.cc:3060] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.642359 11394 raft_consensus.cc:2468] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 1.
I20260709 13:04:28.644033 11434 leader_election.cc:304] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:28.647527 11583 tablet_bootstrap.cc:654] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.635494 11469 raft_consensus.cc:2468] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 1.
I20260709 13:04:28.628087 11591 leader_election.cc:290] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:28.640056 11543 raft_consensus.cc:3060] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.650898 11584 tablet_bootstrap.cc:492] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27: No bootstrap required, opened a new log
I20260709 13:04:28.650956 11508 leader_election.cc:304] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: 
I20260709 13:04:28.651587 11584 ts_tablet_manager.cc:1403] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27: Time spent bootstrapping tablet: real 0.105s	user 0.016s	sys 0.049s
I20260709 13:04:28.652238 11592 raft_consensus.cc:2804] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:28.653679 11592 raft_consensus.cc:697] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Becoming Leader. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:28.654695 11592 consensus_queue.cc:237] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.654930 11584 raft_consensus.cc:359] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.655689 11584 raft_consensus.cc:385] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.656024 11584 raft_consensus.cc:740] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.656869 11584 consensus_queue.cc:260] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.660575 11585 tablet_bootstrap.cc:492] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20: No bootstrap required, opened a new log
I20260709 13:04:28.662823 11585 ts_tablet_manager.cc:1403] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20: Time spent bootstrapping tablet: real 0.058s	user 0.022s	sys 0.002s
I20260709 13:04:28.663930 11591 raft_consensus.cc:2804] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:28.664440 11591 raft_consensus.cc:697] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 LEADER]: Becoming Leader. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:28.666296 11585 raft_consensus.cc:359] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.667104 11585 raft_consensus.cc:385] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.667388 11585 raft_consensus.cc:740] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.667924 11393 raft_consensus.cc:2468] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 1.
I20260709 13:04:28.668227 11585 consensus_queue.cc:260] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.670533 11591 consensus_queue.cc:237] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.676290 11543 raft_consensus.cc:2468] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 1.
I20260709 13:04:28.676683 11585 ts_tablet_manager.cc:1434] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:04:28.678748 11585 tablet_bootstrap.cc:492] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20: Bootstrap starting.
I20260709 13:04:28.690330 11585 tablet_bootstrap.cc:654] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.691603 11583 tablet_bootstrap.cc:492] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2: No bootstrap required, opened a new log
I20260709 13:04:28.692337 11290 catalog_manager.cc:5697] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3ece719b875a475eaa0c79ffde68cb20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.694929 11584 ts_tablet_manager.cc:1434] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27: Time spent starting tablet: real 0.043s	user 0.011s	sys 0.010s
I20260709 13:04:28.696501 11584 tablet_bootstrap.cc:492] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27: Bootstrap starting.
I20260709 13:04:28.696569 11590 raft_consensus.cc:493] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.697206 11590 raft_consensus.cc:515] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.692313 11583 ts_tablet_manager.cc:1403] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.010s
I20260709 13:04:28.702593 11590 leader_election.cc:290] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:28.702231 11291 catalog_manager.cc:5697] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:28.703394 11583 raft_consensus.cc:359] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.704592 11583 raft_consensus.cc:385] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.705044 11583 raft_consensus.cc:740] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.714697 11583 consensus_queue.cc:260] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.722476 11584 tablet_bootstrap.cc:654] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.722623 11585 tablet_bootstrap.cc:492] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20: No bootstrap required, opened a new log
I20260709 13:04:28.729954 11585 ts_tablet_manager.cc:1403] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20: Time spent bootstrapping tablet: real 0.051s	user 0.014s	sys 0.026s
I20260709 13:04:28.722842 11583 ts_tablet_manager.cc:1434] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.001s
I20260709 13:04:28.734856 11583 tablet_bootstrap.cc:492] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:28.740290 11585 raft_consensus.cc:359] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.741153 11585 raft_consensus.cc:385] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.741523 11585 raft_consensus.cc:740] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.742414 11585 consensus_queue.cc:260] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.746124 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f216ed20c2043bf82cedeeeab582b59" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
W20260709 13:04:28.748943 11359 leader_election.cc:343] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:28.749630 11583 tablet_bootstrap.cc:654] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.753396 11585 ts_tablet_manager.cc:1434] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.004s
I20260709 13:04:28.755398 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f216ed20c2043bf82cedeeeab582b59" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:28.756028 11585 tablet_bootstrap.cc:492] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20: Bootstrap starting.
I20260709 13:04:28.756136 11543 raft_consensus.cc:2468] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc3230c8b0704c5ab54fd82d58545db2 in term 0.
I20260709 13:04:28.758610 11359 leader_election.cc:304] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2; no voters: a0690c7fd263405a9adb36b9bd4f2f27
I20260709 13:04:28.760332 11590 raft_consensus.cc:2804] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:28.760749 11590 raft_consensus.cc:493] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.761108 11590 raft_consensus.cc:3060] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.762183 11584 tablet_bootstrap.cc:492] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27: No bootstrap required, opened a new log
I20260709 13:04:28.762828 11584 ts_tablet_manager.cc:1403] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.014s
I20260709 13:04:28.766342 11584 raft_consensus.cc:359] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.767122 11584 raft_consensus.cc:385] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.767433 11584 raft_consensus.cc:740] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.768240 11584 consensus_queue.cc:260] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.771703 11590 raft_consensus.cc:515] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.776290 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f216ed20c2043bf82cedeeeab582b59" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:28.777303 11583 tablet_bootstrap.cc:492] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2: No bootstrap required, opened a new log
I20260709 13:04:28.778021 11583 ts_tablet_manager.cc:1403] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.043s	user 0.011s	sys 0.011s
I20260709 13:04:28.778115 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f216ed20c2043bf82cedeeeab582b59" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:28.778961 11543 raft_consensus.cc:3060] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.778429 11584 ts_tablet_manager.cc:1434] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:04:28.781188 11584 tablet_bootstrap.cc:492] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27: Bootstrap starting.
I20260709 13:04:28.781850 11585 tablet_bootstrap.cc:654] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.781613 11583 raft_consensus.cc:359] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.782485 11583 raft_consensus.cc:385] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.782824 11583 raft_consensus.cc:740] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.782887 11592 raft_consensus.cc:493] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.783562 11592 raft_consensus.cc:515] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.783834 11583 consensus_queue.cc:260] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.786533 11592 leader_election.cc:290] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:28.788755 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd340b80e0bf42e080b36269ec8b5c4a" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
I20260709 13:04:28.789897 11469 raft_consensus.cc:2468] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 0.
I20260709 13:04:28.790693 11583 ts_tablet_manager.cc:1434] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.005s
I20260709 13:04:28.791890 11508 leader_election.cc:304] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: 
I20260709 13:04:28.794891 11592 raft_consensus.cc:2804] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:28.795951 11592 raft_consensus.cc:493] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.801620 11592 raft_consensus.cc:3060] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.802160 11584 tablet_bootstrap.cc:654] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.802479 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd340b80e0bf42e080b36269ec8b5c4a" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2" is_pre_election: true
I20260709 13:04:28.803059 11585 tablet_bootstrap.cc:492] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20: No bootstrap required, opened a new log
I20260709 13:04:28.797925 11583 tablet_bootstrap.cc:492] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
W20260709 13:04:28.803048 11359 leader_election.cc:343] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:28.808116 11508 leader_election.cc:343] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:28.804198 11590 leader_election.cc:290] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 election: Requested vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:28.804267 11585 ts_tablet_manager.cc:1403] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.012s
I20260709 13:04:28.811466 11604 raft_consensus.cc:493] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.812282 11604 raft_consensus.cc:515] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.815074 11604 leader_election.cc:290] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:28.815073 11585 raft_consensus.cc:359] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.816079 11584 tablet_bootstrap.cc:492] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27: No bootstrap required, opened a new log
I20260709 13:04:28.816565 11585 raft_consensus.cc:385] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.816819 11584 ts_tablet_manager.cc:1403] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent bootstrapping tablet: real 0.036s	user 0.020s	sys 0.000s
I20260709 13:04:28.817101 11585 raft_consensus.cc:740] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.817668 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
I20260709 13:04:28.818440 11469 raft_consensus.cc:2468] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 0.
I20260709 13:04:28.818658 11585 consensus_queue.cc:260] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.820443 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2" is_pre_election: true
I20260709 13:04:28.820484 11508 leader_election.cc:304] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: 
I20260709 13:04:28.821671 11604 raft_consensus.cc:2804] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:28.822113 11604 raft_consensus.cc:493] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:04:28.822245 11508 leader_election.cc:343] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:28.822597 11604 raft_consensus.cc:3060] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.823140 11585 ts_tablet_manager.cc:1434] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.001s
I20260709 13:04:28.823720 11584 raft_consensus.cc:359] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.824594 11584 raft_consensus.cc:385] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.824939 11584 raft_consensus.cc:740] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.825273 11585 tablet_bootstrap.cc:492] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20: Bootstrap starting.
I20260709 13:04:28.825822 11584 consensus_queue.cc:260] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.828547 11584 ts_tablet_manager.cc:1434] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:04:28.829892 11584 tablet_bootstrap.cc:492] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27: Bootstrap starting.
I20260709 13:04:28.830410 11543 raft_consensus.cc:2468] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc3230c8b0704c5ab54fd82d58545db2 in term 1.
I20260709 13:04:28.838131 11584 tablet_bootstrap.cc:654] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.840363 11585 tablet_bootstrap.cc:654] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.843377 11359 leader_election.cc:304] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2; no voters: a0690c7fd263405a9adb36b9bd4f2f27
I20260709 13:04:28.844767 11590 raft_consensus.cc:2804] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:28.845329 11590 raft_consensus.cc:697] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 LEADER]: Becoming Leader. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Running, Role: LEADER
I20260709 13:04:28.846261 11590 consensus_queue.cc:237] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.848877 11592 raft_consensus.cc:515] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.855180 11585 tablet_bootstrap.cc:492] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20: No bootstrap required, opened a new log
I20260709 13:04:28.863171 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd340b80e0bf42e080b36269ec8b5c4a" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:28.865571 11585 ts_tablet_manager.cc:1403] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20: Time spent bootstrapping tablet: real 0.041s	user 0.017s	sys 0.009s
I20260709 13:04:28.865926 11592 leader_election.cc:290] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
W20260709 13:04:28.868309 11508 leader_election.cc:343] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:28.868943 11585 raft_consensus.cc:359] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.869786 11585 raft_consensus.cc:385] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.870100 11585 raft_consensus.cc:740] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.870802 11585 consensus_queue.cc:260] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.872587 11585 ts_tablet_manager.cc:1434] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:04:28.856855 11583 tablet_bootstrap.cc:654] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.873473 11585 tablet_bootstrap.cc:492] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20: Bootstrap starting.
I20260709 13:04:28.873876 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd340b80e0bf42e080b36269ec8b5c4a" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:28.874629 11469 raft_consensus.cc:3060] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.881778 11585 tablet_bootstrap.cc:654] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.858721 11604 raft_consensus.cc:515] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.892416 11585 tablet_bootstrap.cc:492] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20: No bootstrap required, opened a new log
I20260709 13:04:28.893023 11585 ts_tablet_manager.cc:1403] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.007s
I20260709 13:04:28.893978 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:28.894279 11289 catalog_manager.cc:5697] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "dc3230c8b0704c5ab54fd82d58545db2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:28.896420 11508 leader_election.cc:343] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:28.896427 11585 raft_consensus.cc:359] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.897413 11585 raft_consensus.cc:385] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.897732 11585 raft_consensus.cc:740] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.898012 11604 leader_election.cc:290] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:28.898607 11585 consensus_queue.cc:260] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.901021 11584 tablet_bootstrap.cc:492] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27: No bootstrap required, opened a new log
I20260709 13:04:28.901535 11585 ts_tablet_manager.cc:1434] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:04:28.901607 11584 ts_tablet_manager.cc:1403] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent bootstrapping tablet: real 0.072s	user 0.020s	sys 0.005s
I20260709 13:04:28.902640 11585 tablet_bootstrap.cc:492] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20: Bootstrap starting.
I20260709 13:04:28.905411 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:28.905352 11584 raft_consensus.cc:359] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.906198 11584 raft_consensus.cc:385] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.906239 11468 raft_consensus.cc:3060] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.906613 11584 raft_consensus.cc:740] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.907616 11584 consensus_queue.cc:260] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.908277 11585 tablet_bootstrap.cc:654] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.913679 11610 raft_consensus.cc:493] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.914235 11610 raft_consensus.cc:515] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.916594 11610 leader_election.cc:290] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:28.917799 11468 raft_consensus.cc:2468] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 1.
I20260709 13:04:28.918665 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc846bd430d248fd812392acd98e9ac1" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2" is_pre_election: true
I20260709 13:04:28.919479 11393 raft_consensus.cc:2468] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 0.
I20260709 13:04:28.919902 11584 ts_tablet_manager.cc:1434] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:04:28.921200 11584 tablet_bootstrap.cc:492] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27: Bootstrap starting.
I20260709 13:04:28.921029 11434 leader_election.cc:304] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:28.922712 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc846bd430d248fd812392acd98e9ac1" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:28.923483 11543 raft_consensus.cc:2468] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 0.
I20260709 13:04:28.922142 11610 raft_consensus.cc:2804] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:28.926069 11610 raft_consensus.cc:493] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:28.926549 11610 raft_consensus.cc:3060] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.930501 11583 tablet_bootstrap.cc:492] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2: No bootstrap required, opened a new log
I20260709 13:04:28.931155 11583 ts_tablet_manager.cc:1403] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.134s	user 0.014s	sys 0.022s
I20260709 13:04:28.933909 11469 raft_consensus.cc:2468] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 1.
I20260709 13:04:28.934537 11583 raft_consensus.cc:359] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.936163 11583 raft_consensus.cc:385] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.935876 11508 leader_election.cc:304] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:28.936548 11583 raft_consensus.cc:740] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.937027 11604 raft_consensus.cc:2804] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:28.937584 11604 raft_consensus.cc:697] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Becoming Leader. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:28.937912 11585 tablet_bootstrap.cc:492] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20: No bootstrap required, opened a new log
I20260709 13:04:28.938794 11584 tablet_bootstrap.cc:654] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:28.938345 11583 consensus_queue.cc:260] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.938446 11604 consensus_queue.cc:237] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.939154 11610 raft_consensus.cc:515] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.943033 11508 leader_election.cc:304] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:28.938467 11585 ts_tablet_manager.cc:1403] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.017s
I20260709 13:04:28.945274 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc846bd430d248fd812392acd98e9ac1" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:28.945673 11592 raft_consensus.cc:2804] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:28.945981 11393 raft_consensus.cc:3060] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.951545 11585 raft_consensus.cc:359] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.952596 11585 raft_consensus.cc:385] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:28.952884 11585 raft_consensus.cc:740] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Initialized, Role: FOLLOWER
I20260709 13:04:28.955502 11585 consensus_queue.cc:260] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.956923 11610 leader_election.cc:290] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:28.958362 11393 raft_consensus.cc:2468] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 1.
I20260709 13:04:28.960759 11434 leader_election.cc:304] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:28.962131 11610 raft_consensus.cc:2804] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:28.964151 11610 raft_consensus.cc:697] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 LEADER]: Becoming Leader. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:28.966280 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc846bd430d248fd812392acd98e9ac1" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:28.967249 11543 raft_consensus.cc:3060] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:28.967769 11610 consensus_queue.cc:237] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.973577 11592 raft_consensus.cc:697] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Becoming Leader. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:28.975358 11592 consensus_queue.cc:237] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.985627 11291 catalog_manager.cc:5697] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3ece719b875a475eaa0c79ffde68cb20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:28.991432 11583 ts_tablet_manager.cc:1434] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.060s	user 0.007s	sys 0.002s
I20260709 13:04:28.995903 11590 raft_consensus.cc:493] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:28.996775 11590 raft_consensus.cc:515] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:28.999840 11590 leader_election.cc:290] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:29.003127 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cc664f62b248ec92a2cef483c2c665" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
I20260709 13:04:29.003647 11583 tablet_bootstrap.cc:492] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:29.003995 11469 raft_consensus.cc:2468] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc3230c8b0704c5ab54fd82d58545db2 in term 0.
I20260709 13:04:29.006078 11359 leader_election.cc:304] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:29.004175 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cc664f62b248ec92a2cef483c2c665" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:29.008244 11542 raft_consensus.cc:2468] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc3230c8b0704c5ab54fd82d58545db2 in term 0.
I20260709 13:04:29.011050 11543 raft_consensus.cc:2468] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 1.
I20260709 13:04:29.010174 11590 raft_consensus.cc:2804] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:29.012442 11590 raft_consensus.cc:493] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.012529 11621 raft_consensus.cc:493] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.012909 11590 raft_consensus.cc:3060] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.013094 11621 raft_consensus.cc:515] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.015944 11585 ts_tablet_manager.cc:1434] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20: Time spent starting tablet: real 0.071s	user 0.020s	sys 0.027s
I20260709 13:04:29.017059 11621 leader_election.cc:290] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:29.021117 11583 tablet_bootstrap.cc:654] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:29.021840 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb34dc48d7194d7bb15ecb7094dabf43" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
W20260709 13:04:29.024061 11359 leader_election.cc:343] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:29.025686 11592 raft_consensus.cc:493] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.026819 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb34dc48d7194d7bb15ecb7094dabf43" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:29.030193 11543 raft_consensus.cc:2468] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc3230c8b0704c5ab54fd82d58545db2 in term 0.
I20260709 13:04:29.027690 11592 raft_consensus.cc:515] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.031106 11584 tablet_bootstrap.cc:492] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27: No bootstrap required, opened a new log
I20260709 13:04:29.031688 11584 ts_tablet_manager.cc:1403] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent bootstrapping tablet: real 0.111s	user 0.032s	sys 0.046s
I20260709 13:04:29.031723 11359 leader_election.cc:304] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2; no voters: a0690c7fd263405a9adb36b9bd4f2f27
I20260709 13:04:29.031899 11590 raft_consensus.cc:515] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.032989 11621 raft_consensus.cc:2804] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:29.033380 11621 raft_consensus.cc:493] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.034116 11621 raft_consensus.cc:3060] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.035015 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b5d1a8d996415982df5ed08c242522" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
I20260709 13:04:29.035215 11584 raft_consensus.cc:359] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.035917 11584 raft_consensus.cc:385] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:29.036239 11584 raft_consensus.cc:740] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Initialized, Role: FOLLOWER
I20260709 13:04:29.036422 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b5d1a8d996415982df5ed08c242522" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2" is_pre_election: true
I20260709 13:04:29.036962 11584 consensus_queue.cc:260] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
W20260709 13:04:29.038305 11508 leader_election.cc:343] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:29.030232 11292 catalog_manager.cc:5697] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:29.039726 11584 ts_tablet_manager.cc:1434] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
W20260709 13:04:29.039587 11508 leader_election.cc:343] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:29.033453 11592 leader_election.cc:290] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:29.040290 11508 leader_election.cc:304] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20; no voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:29.041292 11592 raft_consensus.cc:2749] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:29.042739 11590 leader_election.cc:290] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 election: Requested vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:29.043211 11621 raft_consensus.cc:515] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.043979 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cc664f62b248ec92a2cef483c2c665" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:29.043720 11291 catalog_manager.cc:5697] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3ece719b875a475eaa0c79ffde68cb20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:29.045962 11621 leader_election.cc:290] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 election: Requested vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:29.046304 11584 tablet_bootstrap.cc:492] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27: Bootstrap starting.
I20260709 13:04:29.049378 11583 tablet_bootstrap.cc:492] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2: No bootstrap required, opened a new log
I20260709 13:04:29.044585 11469 raft_consensus.cc:3060] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.051311 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb34dc48d7194d7bb15ecb7094dabf43" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:29.052368 11468 raft_consensus.cc:3060] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.050050 11583 ts_tablet_manager.cc:1403] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.007s
I20260709 13:04:29.051234 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cc664f62b248ec92a2cef483c2c665" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:29.054978 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb34dc48d7194d7bb15ecb7094dabf43" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:29.055660 11542 raft_consensus.cc:3060] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.056619 11584 tablet_bootstrap.cc:654] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:29.057565 11583 raft_consensus.cc:359] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.058344 11583 raft_consensus.cc:385] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:29.058612 11583 raft_consensus.cc:740] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:29.059508 11583 consensus_queue.cc:260] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.061676 11583 ts_tablet_manager.cc:1434] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:04:29.054664 11543 raft_consensus.cc:3060] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.070860 11583 tablet_bootstrap.cc:492] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:29.072669 11469 raft_consensus.cc:2468] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc3230c8b0704c5ab54fd82d58545db2 in term 1.
I20260709 13:04:29.076699 11359 leader_election.cc:304] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:29.077077 11610 consensus_queue.cc:1048] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:29.077634 11621 raft_consensus.cc:2804] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:29.078115 11621 raft_consensus.cc:697] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 LEADER]: Becoming Leader. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Running, Role: LEADER
I20260709 13:04:29.078984 11621 consensus_queue.cc:237] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.084802 11584 tablet_bootstrap.cc:492] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27: No bootstrap required, opened a new log
I20260709 13:04:29.086839 11584 ts_tablet_manager.cc:1403] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent bootstrapping tablet: real 0.041s	user 0.007s	sys 0.013s
I20260709 13:04:29.086031 11542 raft_consensus.cc:2468] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc3230c8b0704c5ab54fd82d58545db2 in term 1.
I20260709 13:04:29.089305 11359 leader_election.cc:304] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:29.090695 11590 raft_consensus.cc:2804] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:29.091723 11468 raft_consensus.cc:2468] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc3230c8b0704c5ab54fd82d58545db2 in term 1.
I20260709 13:04:29.093263 11590 raft_consensus.cc:697] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 LEADER]: Becoming Leader. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Running, Role: LEADER
I20260709 13:04:29.094717 11590 consensus_queue.cc:237] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.094981 11543 raft_consensus.cc:2468] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc3230c8b0704c5ab54fd82d58545db2 in term 1.
I20260709 13:04:29.098507 11583 tablet_bootstrap.cc:654] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:29.105863 11590 raft_consensus.cc:493] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.106384 11590 raft_consensus.cc:515] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.108305 11590 leader_election.cc:290] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:29.094688 11584 raft_consensus.cc:359] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.109467 11584 raft_consensus.cc:385] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:29.109732 11584 raft_consensus.cc:740] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Initialized, Role: FOLLOWER
I20260709 13:04:29.109966 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd340b80e0bf42e080b36269ec8b5c4a" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:29.110626 11584 consensus_queue.cc:260] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.111704 11583 tablet_bootstrap.cc:492] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2: No bootstrap required, opened a new log
I20260709 13:04:29.112273 11583 ts_tablet_manager.cc:1403] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.005s
I20260709 13:04:29.112411 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd340b80e0bf42e080b36269ec8b5c4a" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
I20260709 13:04:29.113336 11468 raft_consensus.cc:2393] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dc3230c8b0704c5ab54fd82d58545db2 in current term 1: Already voted for candidate 3ece719b875a475eaa0c79ffde68cb20 in this term.
I20260709 13:04:29.114657 11359 leader_election.cc:304] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc3230c8b0704c5ab54fd82d58545db2; no voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27
I20260709 13:04:29.115306 11583 raft_consensus.cc:359] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.115913 11621 raft_consensus.cc:3060] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.116065 11583 raft_consensus.cc:385] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:29.116904 11583 raft_consensus.cc:740] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:29.117831 11583 consensus_queue.cc:260] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.119899 11583 ts_tablet_manager.cc:1434] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:04:29.120878 11583 tablet_bootstrap.cc:492] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:29.123090 11584 ts_tablet_manager.cc:1434] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 13:04:29.124339 11584 tablet_bootstrap.cc:492] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27: Bootstrap starting.
I20260709 13:04:29.123342 11289 catalog_manager.cc:5697] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "dc3230c8b0704c5ab54fd82d58545db2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:29.127884 11583 tablet_bootstrap.cc:654] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:29.138026 11583 tablet_bootstrap.cc:492] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2: No bootstrap required, opened a new log
I20260709 13:04:29.138549 11583 ts_tablet_manager.cc:1403] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.018s	user 0.007s	sys 0.007s
I20260709 13:04:29.139329 11621 raft_consensus.cc:2749] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:29.140317 11584 tablet_bootstrap.cc:654] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:29.141592 11583 raft_consensus.cc:359] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.142321 11583 raft_consensus.cc:385] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:29.142582 11583 raft_consensus.cc:740] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:29.143318 11583 consensus_queue.cc:260] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.145506 11583 ts_tablet_manager.cc:1434] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:04:29.152819 11584 tablet_bootstrap.cc:492] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27: No bootstrap required, opened a new log
I20260709 13:04:29.153635 11584 ts_tablet_manager.cc:1403] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.004s
I20260709 13:04:29.154879 11289 catalog_manager.cc:5697] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "dc3230c8b0704c5ab54fd82d58545db2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:29.157529 11584 raft_consensus.cc:359] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.158582 11584 raft_consensus.cc:385] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:29.158953 11584 raft_consensus.cc:740] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Initialized, Role: FOLLOWER
I20260709 13:04:29.160100 11584 consensus_queue.cc:260] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.163326 11584 ts_tablet_manager.cc:1434] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:04:29.164799 11584 tablet_bootstrap.cc:492] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27: Bootstrap starting.
I20260709 13:04:29.170959 11584 tablet_bootstrap.cc:654] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:29.173969 11591 consensus_queue.cc:1048] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:29.175858 11584 tablet_bootstrap.cc:492] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27: No bootstrap required, opened a new log
I20260709 13:04:29.176343 11584 ts_tablet_manager.cc:1403] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent bootstrapping tablet: real 0.012s	user 0.006s	sys 0.003s
I20260709 13:04:29.178471 11584 raft_consensus.cc:359] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.179193 11584 raft_consensus.cc:385] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:29.179544 11584 raft_consensus.cc:740] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Initialized, Role: FOLLOWER
I20260709 13:04:29.180338 11584 consensus_queue.cc:260] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.182574 11584 ts_tablet_manager.cc:1434] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:04:29.228274 11592 consensus_queue.cc:1048] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:29.240130 11634 consensus_queue.cc:1048] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:29.254222 11591 raft_consensus.cc:493] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.255009 11591 raft_consensus.cc:515] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.257539 11591 leader_election.cc:290] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:29.258426 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f216ed20c2043bf82cedeeeab582b59" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2" is_pre_election: true
I20260709 13:04:29.259008 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f216ed20c2043bf82cedeeeab582b59" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:29.259888 11543 raft_consensus.cc:2393] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0690c7fd263405a9adb36b9bd4f2f27 in current term 1: Already voted for candidate dc3230c8b0704c5ab54fd82d58545db2 in this term.
I20260709 13:04:29.260965 11434 leader_election.cc:304] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27; no voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:29.261830 11591 raft_consensus.cc:3060] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.268059 11591 raft_consensus.cc:2749] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:29.316156 11591 raft_consensus.cc:493] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.316130 11621 consensus_queue.cc:1048] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:29.316877 11591 raft_consensus.cc:515] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.319111 11591 leader_election.cc:290] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:29.319852 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b5d1a8d996415982df5ed08c242522" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2" is_pre_election: true
I20260709 13:04:29.320770 11393 raft_consensus.cc:2468] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 0.
I20260709 13:04:29.320919 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b5d1a8d996415982df5ed08c242522" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:29.321795 11543 raft_consensus.cc:2468] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 0.
I20260709 13:04:29.322189 11434 leader_election.cc:304] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:29.323200 11591 raft_consensus.cc:2804] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:29.324067 11591 raft_consensus.cc:493] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.324414 11591 raft_consensus.cc:3060] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.328878 11641 consensus_queue.cc:1048] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:29.335872 11591 raft_consensus.cc:515] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.337584 11634 raft_consensus.cc:493] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.338375 11591 leader_election.cc:290] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:29.338198 11634 raft_consensus.cc:515] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.339095 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b5d1a8d996415982df5ed08c242522" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:29.339560 11541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b5d1a8d996415982df5ed08c242522" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:29.339744 11393 raft_consensus.cc:3060] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.340229 11541 raft_consensus.cc:3060] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.340473 11634 leader_election.cc:290] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:29.343292 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2130ecf60b94295aff5ea3f1aeafd98" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
I20260709 13:04:29.344060 11468 raft_consensus.cc:2468] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 0.
I20260709 13:04:29.345487 11508 leader_election.cc:304] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: 
I20260709 13:04:29.348616 11394 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2130ecf60b94295aff5ea3f1aeafd98" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2" is_pre_election: true
I20260709 13:04:29.349346 11394 raft_consensus.cc:2468] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 0.
I20260709 13:04:29.348624 11634 raft_consensus.cc:2804] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:29.350551 11634 raft_consensus.cc:493] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:29.350906 11634 raft_consensus.cc:3060] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.351531 11393 raft_consensus.cc:2468] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 1.
I20260709 13:04:29.352092 11541 raft_consensus.cc:2468] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 1.
I20260709 13:04:29.352968 11434 leader_election.cc:304] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:29.354029 11591 raft_consensus.cc:2804] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:29.354516 11591 raft_consensus.cc:697] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 LEADER]: Becoming Leader. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:29.355361 11591 consensus_queue.cc:237] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.359586 11634 raft_consensus.cc:515] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.360244 11604 consensus_queue.cc:1048] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:29.361928 11634 leader_election.cc:290] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:29.362736 11393 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2130ecf60b94295aff5ea3f1aeafd98" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:29.363433 11393 raft_consensus.cc:3060] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.365038 11290 catalog_manager.cc:5697] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:29.366556 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2130ecf60b94295aff5ea3f1aeafd98" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:29.367233 11469 raft_consensus.cc:3060] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.373828 11393 raft_consensus.cc:2468] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 1.
I20260709 13:04:29.375527 11508 leader_election.cc:304] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:29.376479 11634 raft_consensus.cc:2804] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:29.376991 11634 raft_consensus.cc:697] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Becoming Leader. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:29.377890 11634 consensus_queue.cc:237] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.384258 11469 raft_consensus.cc:2468] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 1.
I20260709 13:04:29.385502 11592 consensus_queue.cc:1048] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:29.391515 11290 catalog_manager.cc:5697] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "3ece719b875a475eaa0c79ffde68cb20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:29.448976 11591 consensus_queue.cc:1048] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:29.462208 11610 consensus_queue.cc:1048] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:29.531371 11641 raft_consensus.cc:493] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:29.531880 11641 raft_consensus.cc:515] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:29.531874 11634 consensus_queue.cc:1048] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:29.534166 11641 leader_election.cc:290] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:29.534754 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
I20260709 13:04:29.535318 11541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:29.536996 11359 leader_election.cc:304] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc3230c8b0704c5ab54fd82d58545db2; no voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27
I20260709 13:04:29.538004 11641 raft_consensus.cc:2749] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:29.543299 11393 raft_consensus.cc:3060] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:29.552336 11592 consensus_queue.cc:1048] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:29.563186   431 ts_manager.cc:295] Set tserver state for dc3230c8b0704c5ab54fd82d58545db2 to MAINTENANCE_MODE
I20260709 13:04:29.564298   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:04:29.593648   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:29.594756   431 tablet_replica.cc:333] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2: stopping tablet replica
I20260709 13:04:29.595150 11641 consensus_queue.cc:1048] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:29.595744   431 raft_consensus.cc:2243] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:29.596279   431 raft_consensus.cc:2272] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:29.599169   431 tablet_replica.cc:333] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2: stopping tablet replica
I20260709 13:04:29.600162   431 raft_consensus.cc:2243] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:29.600670   431 raft_consensus.cc:2272] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:29.603567   431 tablet_replica.cc:333] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2: stopping tablet replica
I20260709 13:04:29.604382   431 raft_consensus.cc:2243] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:29.604962   431 raft_consensus.cc:2272] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:29.606861 11621 consensus_queue.cc:1048] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:29.609563   431 tablet_replica.cc:333] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2: stopping tablet replica
I20260709 13:04:29.610508   431 raft_consensus.cc:2243] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:29.611071   431 raft_consensus.cc:2272] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:29.614137   431 tablet_replica.cc:333] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2: stopping tablet replica
I20260709 13:04:29.614845   431 raft_consensus.cc:2243] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:29.615406   431 raft_consensus.cc:2272] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:29.618350   431 tablet_replica.cc:333] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2: stopping tablet replica
I20260709 13:04:29.618984   431 raft_consensus.cc:2243] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:29.619812   431 raft_consensus.cc:2272] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:29.622320   431 tablet_replica.cc:333] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2: stopping tablet replica
I20260709 13:04:29.622951   431 raft_consensus.cc:2243] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:29.623380   431 raft_consensus.cc:2272] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:29.626721   431 tablet_replica.cc:333] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: stopping tablet replica
I20260709 13:04:29.627483   431 raft_consensus.cc:2243] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:29.628273   431 pending_rounds.cc:70] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: Trying to abort 1 pending ops.
I20260709 13:04:29.628530   431 pending_rounds.cc:77] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305634894223843328 op_type: NO_OP noop_request { }
I20260709 13:04:29.628883   431 raft_consensus.cc:2889] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:29.629164   431 raft_consensus.cc:2272] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:29.631402   431 tablet_replica.cc:333] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2: stopping tablet replica
I20260709 13:04:29.631959   431 raft_consensus.cc:2243] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:29.632376   431 raft_consensus.cc:2272] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:29.634866   431 tablet_replica.cc:333] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2: stopping tablet replica
I20260709 13:04:29.635414   431 raft_consensus.cc:2243] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:29.636242   431 raft_consensus.cc:2272] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:29.669159   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:04:29.787801 11592 consensus_queue.cc:1048] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:04:29.792819 11434 consensus_peers.cc:597] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:29.805730 11434 consensus_peers.cc:597] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:29.809314 11508 consensus_peers.cc:597] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:29.813675 11508 consensus_peers.cc:597] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:29.923521 11508 consensus_peers.cc:597] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:29.964246 11434 consensus_peers.cc:597] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:29.968108 11591 consensus_queue.cc:1048] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:04:30.005142 11508 consensus_peers.cc:597] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:30.446493 11508 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111) [suppressed 148 similar messages]
W20260709 13:04:30.587155 11340 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:04:30.826932 11610 raft_consensus.cc:493] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dc3230c8b0704c5ab54fd82d58545db2)
I20260709 13:04:30.827536 11610 raft_consensus.cc:515] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:30.829454 11610 leader_election.cc:290] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:30.831071 11541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f216ed20c2043bf82cedeeeab582b59" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
W20260709 13:04:30.833138 11434 leader_election.cc:336] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111)
I20260709 13:04:30.833644 11434 leader_election.cc:304] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27; no voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:30.834396 11610 raft_consensus.cc:2749] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:30.834611 11633 raft_consensus.cc:493] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dc3230c8b0704c5ab54fd82d58545db2)
I20260709 13:04:30.835160 11633 raft_consensus.cc:515] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:30.837139 11633 leader_election.cc:290] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:30.838717 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f216ed20c2043bf82cedeeeab582b59" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
I20260709 13:04:30.839493 11469 raft_consensus.cc:2468] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 1.
I20260709 13:04:30.841596 11508 leader_election.cc:304] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: 
I20260709 13:04:30.842473 11633 raft_consensus.cc:2804] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:04:30.842512 11508 leader_election.cc:336] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111)
I20260709 13:04:30.842851 11633 raft_consensus.cc:493] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dc3230c8b0704c5ab54fd82d58545db2)
I20260709 13:04:30.843252 11633 raft_consensus.cc:3060] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:30.848691 11633 raft_consensus.cc:515] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:30.851004 11633 leader_election.cc:290] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:30.852677 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f216ed20c2043bf82cedeeeab582b59" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:30.853281 11469 raft_consensus.cc:3060] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:30.854585 11508 leader_election.cc:336] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111)
I20260709 13:04:30.857760 11608 raft_consensus.cc:493] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:30.858372 11608 raft_consensus.cc:515] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:30.859404 11469 raft_consensus.cc:2468] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 2.
I20260709 13:04:30.860698 11608 leader_election.cc:290] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:30.860770 11508 leader_election.cc:304] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:30.861852 11633 raft_consensus.cc:2804] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:30.862432 11633 raft_consensus.cc:697] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 LEADER]: Becoming Leader. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:30.863202 11633 consensus_queue.cc:237] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:30.864635 11541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb34dc48d7194d7bb15ecb7094dabf43" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:30.865361 11541 raft_consensus.cc:2468] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 1.
W20260709 13:04:30.865708 11434 leader_election.cc:336] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111)
I20260709 13:04:30.866730 11434 leader_election.cc:304] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:30.868310 11608 raft_consensus.cc:2804] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:30.868711 11608 raft_consensus.cc:493] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:30.869118 11608 raft_consensus.cc:3060] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:30.873569 11291 catalog_manager.cc:5697] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 reported cstate change: term changed from 1 to 2, leader changed from dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193) to 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "3ece719b875a475eaa0c79ffde68cb20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:30.877126 11608 raft_consensus.cc:515] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:30.879349 11608 leader_election.cc:290] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:30.881085 11541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb34dc48d7194d7bb15ecb7094dabf43" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:30.881860 11541 raft_consensus.cc:3060] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:30.883646 11434 leader_election.cc:336] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111)
I20260709 13:04:30.889741 11541 raft_consensus.cc:2468] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 2.
I20260709 13:04:30.891136 11434 leader_election.cc:304] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:30.892159 11608 raft_consensus.cc:2804] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:30.892807 11608 raft_consensus.cc:697] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Becoming Leader. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:30.894052 11608 consensus_queue.cc:237] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:30.904176 11291 catalog_manager.cc:5697] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 reported cstate change: term changed from 1 to 2, leader changed from dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193) to a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:31.105746 11608 raft_consensus.cc:493] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dc3230c8b0704c5ab54fd82d58545db2)
I20260709 13:04:31.106330 11608 raft_consensus.cc:515] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:31.108032 11608 leader_election.cc:290] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:31.109932 11541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cc664f62b248ec92a2cef483c2c665" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:31.110723 11541 raft_consensus.cc:2468] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 1.
I20260709 13:04:31.111225 11633 raft_consensus.cc:493] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dc3230c8b0704c5ab54fd82d58545db2)
I20260709 13:04:31.111625 11633 raft_consensus.cc:515] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
W20260709 13:04:31.112414 11434 leader_election.cc:336] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111)
I20260709 13:04:31.113453 11633 leader_election.cc:290] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:31.113483 11434 leader_election.cc:304] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:31.115185 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cc664f62b248ec92a2cef483c2c665" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
I20260709 13:04:31.115350 11608 raft_consensus.cc:2804] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:31.115959 11469 raft_consensus.cc:2468] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 1.
I20260709 13:04:31.116675 11608 raft_consensus.cc:493] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dc3230c8b0704c5ab54fd82d58545db2)
I20260709 13:04:31.116981 11508 leader_election.cc:304] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: 
I20260709 13:04:31.117120 11608 raft_consensus.cc:3060] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:31.117955 11508 leader_election.cc:336] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111)
I20260709 13:04:31.118028 11633 raft_consensus.cc:2804] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:31.118582 11633 raft_consensus.cc:493] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dc3230c8b0704c5ab54fd82d58545db2)
I20260709 13:04:31.119026 11633 raft_consensus.cc:3060] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:31.124208 11608 raft_consensus.cc:515] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:31.124303 11633 raft_consensus.cc:515] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:31.126674 11633 leader_election.cc:290] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:31.126734 11608 leader_election.cc:290] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:31.129403 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cc664f62b248ec92a2cef483c2c665" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:31.130174 11541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cc664f62b248ec92a2cef483c2c665" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:31.130491 11469 raft_consensus.cc:2393] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 3ece719b875a475eaa0c79ffde68cb20 in current term 2: Already voted for candidate a0690c7fd263405a9adb36b9bd4f2f27 in this term.
W20260709 13:04:31.131975 11434 leader_election.cc:336] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111)
I20260709 13:04:31.131120 11541 raft_consensus.cc:2393] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a0690c7fd263405a9adb36b9bd4f2f27 in current term 2: Already voted for candidate 3ece719b875a475eaa0c79ffde68cb20 in this term.
W20260709 13:04:31.132391 11508 leader_election.cc:336] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111)
I20260709 13:04:31.133004 11508 leader_election.cc:304] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20; no voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:31.133723 11633 raft_consensus.cc:2749] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:31.134506 11434 leader_election.cc:304] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27; no voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:31.135394 11608 raft_consensus.cc:2749] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:04:31.332315 11469 raft_consensus.cc:1275] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Refusing update from remote peer 3ece719b875a475eaa0c79ffde68cb20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:31.333925 11633 consensus_queue.cc:1048] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:31.348513 11508 consensus_peers.cc:597] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:31.361934 11434 consensus_peers.cc:597] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:31.429744 11541 raft_consensus.cc:1275] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Refusing update from remote peer a0690c7fd263405a9adb36b9bd4f2f27: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:04:31.431816 11608 consensus_queue.cc:1048] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:04:31.588253 11340 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:04:32.216910 11508 consensus_peers.cc:597] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:32.316074 11434 consensus_peers.cc:597] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:32.381610 11508 consensus_peers.cc:597] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:32.388672 11508 consensus_peers.cc:597] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:32.431690 11434 consensus_peers.cc:597] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:32.512620 11508 consensus_peers.cc:597] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:32.589231 11340 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:04:32.611690 11434 consensus_peers.cc:597] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:04:32.850893 11608 raft_consensus.cc:493] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:32.851423 11608 raft_consensus.cc:515] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:32.853228 11608 leader_election.cc:290] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:32.854840 11541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cc664f62b248ec92a2cef483c2c665" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:32.855593 11541 raft_consensus.cc:2468] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 2.
I20260709 13:04:32.856776 11434 leader_election.cc:304] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: 
W20260709 13:04:32.857507 11434 leader_election.cc:336] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111)
I20260709 13:04:32.857514 11608 raft_consensus.cc:2804] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:32.858131 11608 raft_consensus.cc:493] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:32.858458 11608 raft_consensus.cc:3060] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:32.864128 11608 raft_consensus.cc:515] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:32.865832 11608 leader_election.cc:290] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 3 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:32.867798 11541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cc664f62b248ec92a2cef483c2c665" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:32.868486 11541 raft_consensus.cc:3060] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:32.869685 11434 leader_election.cc:336] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111)
I20260709 13:04:32.873766 11541 raft_consensus.cc:2468] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 3.
I20260709 13:04:32.875025 11434 leader_election.cc:304] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:32.875947 11608 raft_consensus.cc:2804] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:32.876780 11608 raft_consensus.cc:697] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 LEADER]: Becoming Leader. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:32.877632 11608 consensus_queue.cc:237] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:32.889240 11291 catalog_manager.cc:5697] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 reported cstate change: term changed from 1 to 3, leader changed from dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193) to a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:33.321399 11434 consensus_peers.cc:597] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:33.333480 11541 raft_consensus.cc:1275] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Refusing update from remote peer a0690c7fd263405a9adb36b9bd4f2f27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:33.334645 11608 consensus_queue.cc:1048] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:33.590344 11340 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:04:33.761257 11434 consensus_peers.cc:597] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:33.967613 11508 consensus_peers.cc:597] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:04:34.426481 11508 consensus_peers.cc:597] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:04:34.591347 11340 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:04:34.690585   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:34.697837 11694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:34.698078 11695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:34.702502   431 server_base.cc:1061] running on GCE node
W20260709 13:04:34.702471 11697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:34.704941   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:34.705212   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:34.705468   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602274705424 us; error 0 us; skew 500 ppm
I20260709 13:04:34.706075   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:34.709175   431 webserver.cc:533] Webserver started at http://127.0.107.193:45027/ using document root <none> and password file <none>
I20260709 13:04:34.709795   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:34.710047   431 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:04:34.714521   431 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:04:34.718644 11702 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:34.719491   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:04:34.719861   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-0-root
uuid: "dc3230c8b0704c5ab54fd82d58545db2"
format_stamp: "Formatted at 2026-07-09 13:04:27 on dist-test-slave-jk05"
I20260709 13:04:34.720268   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783601935286233-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:34.736761   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:34.738229   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:34.739810   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:34.743737 11709 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
I20260709 13:04:34.799628   431 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:04:34.799923   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.057s	user 0.003s	sys 0.000s
I20260709 13:04:34.800289   431 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:04:34.805234 11709 tablet_bootstrap.cc:492] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:34.822481 11709 tablet_bootstrap.cc:492] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:34.823530 11709 tablet_bootstrap.cc:492] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap complete.
I20260709 13:04:34.824172 11709 ts_tablet_manager.cc:1403] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 13:04:34.826996 11709 raft_consensus.cc:359] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:34.827708 11709 raft_consensus.cc:740] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.828454 11709 consensus_queue.cc:260] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:34.831175 11709 ts_tablet_manager.cc:1434] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:04:34.832049 11709 tablet_bootstrap.cc:492] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:34.848336 11709 tablet_bootstrap.cc:492] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:04:34.849392 11709 tablet_bootstrap.cc:492] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap complete.
I20260709 13:04:34.850113 11709 ts_tablet_manager.cc:1403] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.003s
I20260709 13:04:34.852806 11709 raft_consensus.cc:359] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:34.853690 11709 raft_consensus.cc:740] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.854399 11709 consensus_queue.cc:260] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:34.856451 11709 ts_tablet_manager.cc:1434] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:34.857344 11709 tablet_bootstrap.cc:492] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:34.866518   431 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:04:34.867038   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.067s	user 0.067s	sys 0.000s
W20260709 13:04:34.867573 11508 consensus_peers.cc:597] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:34.873979 11709 tablet_bootstrap.cc:492] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:34.874895 11709 tablet_bootstrap.cc:492] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap complete.
I20260709 13:04:34.875676 11709 ts_tablet_manager.cc:1403] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.003s
I20260709 13:04:34.878208 11709 raft_consensus.cc:359] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:34.878794 11709 raft_consensus.cc:740] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.879510 11709 consensus_queue.cc:260] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:34.881126 11709 ts_tablet_manager.cc:1434] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:04:34.881873 11709 tablet_bootstrap.cc:492] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:34.895169 11709 tablet_bootstrap.cc:492] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:34.896045 11709 tablet_bootstrap.cc:492] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap complete.
I20260709 13:04:34.896742 11709 ts_tablet_manager.cc:1403] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.015s	user 0.014s	sys 0.000s
I20260709 13:04:34.899514 11709 raft_consensus.cc:359] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:34.900229 11709 raft_consensus.cc:740] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.901011 11709 consensus_queue.cc:260] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:34.902853 11709 ts_tablet_manager.cc:1434] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:04:34.903554 11709 tablet_bootstrap.cc:492] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:34.916003 11709 tablet_bootstrap.cc:492] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:34.916831 11709 tablet_bootstrap.cc:492] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap complete.
I20260709 13:04:34.917922 11709 ts_tablet_manager.cc:1403] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 13:04:34.920521 11709 raft_consensus.cc:359] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:34.921267 11709 raft_consensus.cc:740] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.922155 11709 consensus_queue.cc:260] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:34.924075 11709 ts_tablet_manager.cc:1434] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:04:34.924978 11709 tablet_bootstrap.cc:492] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:34.936606 11709 tablet_bootstrap.cc:492] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:34.937711 11709 tablet_bootstrap.cc:492] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap complete.
I20260709 13:04:34.938462 11709 ts_tablet_manager.cc:1403] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.003s
W20260709 13:04:34.941174 11508 consensus_peers.cc:597] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:34.941243 11709 raft_consensus.cc:359] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:34.942350 11709 raft_consensus.cc:740] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:34.943152 11709 consensus_queue.cc:260] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:34.945384 11709 ts_tablet_manager.cc:1434] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.004s
I20260709 13:04:34.945647   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:38057
I20260709 13:04:34.945772 11772 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:38057 every 8 connection(s)
I20260709 13:04:34.946420 11709 tablet_bootstrap.cc:492] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:34.950935   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:34.997483 11709 tablet_bootstrap.cc:492] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:34.999303 11709 tablet_bootstrap.cc:492] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap complete.
I20260709 13:04:35.001696 11709 ts_tablet_manager.cc:1403] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.056s	user 0.023s	sys 0.027s
W20260709 13:04:35.002866 11434 consensus_peers.cc:597] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 13:04:35.004376 11709 raft_consensus.cc:359] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.005228 11709 raft_consensus.cc:740] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.005833 11541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f216ed20c2043bf82cedeeeab582b59"
dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
mode: GRACEFUL
new_leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
 from {username='slave'} at 127.0.0.1:53360
I20260709 13:04:35.006052 11709 consensus_queue.cc:260] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.006672 11541 raft_consensus.cc:606] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 LEADER]: Received request to transfer leadership to TS a0690c7fd263405a9adb36b9bd4f2f27
I20260709 13:04:35.006960 11773 heartbeater.cc:344] Connected to a master server at 127.0.107.254:38461
I20260709 13:04:35.007427 11773 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:35.008334 11709 ts_tablet_manager.cc:1434] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:35.008432 11773 heartbeater.cc:507] Master 127.0.107.254:38461 requested a full tablet report, sending...
I20260709 13:04:35.009308 11709 tablet_bootstrap.cc:492] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:35.016048 11291 ts_manager.cc:194] Re-registered known tserver with Master: dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057)
I20260709 13:04:35.020177 11468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92b5d1a8d996415982df5ed08c242522"
dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
mode: GRACEFUL
new_leader_uuid: "3ece719b875a475eaa0c79ffde68cb20"
 from {username='slave'} at 127.0.0.1:37842
I20260709 13:04:35.020947 11468 raft_consensus.cc:606] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 LEADER]: Received request to transfer leadership to TS 3ece719b875a475eaa0c79ffde68cb20
I20260709 13:04:35.022595   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:35.023195   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:35.023840   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:35.024946 11709 tablet_bootstrap.cc:492] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:35.026044 11709 tablet_bootstrap.cc:492] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap complete.
I20260709 13:04:35.026726 11709 ts_tablet_manager.cc:1403] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 13:04:35.028793 11291 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57840
I20260709 13:04:35.029366 11709 raft_consensus.cc:359] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.030112 11709 raft_consensus.cc:740] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.030923 11709 consensus_queue.cc:260] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.032778 11773 heartbeater.cc:499] Master 127.0.107.254:38461 was elected leader, sending a full tablet report...
I20260709 13:04:35.032971 11709 ts_tablet_manager.cc:1434] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:04:35.033833 11709 tablet_bootstrap.cc:492] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:35.049299 11709 tablet_bootstrap.cc:492] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:35.050325 11709 tablet_bootstrap.cc:492] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap complete.
I20260709 13:04:35.051100 11709 ts_tablet_manager.cc:1403] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.017s	user 0.016s	sys 0.000s
I20260709 13:04:35.052009 11746 raft_consensus.cc:3060] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:35.053236 11709 raft_consensus.cc:359] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.053903 11709 raft_consensus.cc:740] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.054565 11709 consensus_queue.cc:260] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.056128 11709 ts_tablet_manager.cc:1434] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:04:35.056810 11709 tablet_bootstrap.cc:492] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap starting.
I20260709 13:04:35.059659 11746 pending_rounds.cc:85] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: Aborting all ops after (but not including) 0
I20260709 13:04:35.060021 11746 pending_rounds.cc:107] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:04:35.060288 11746 raft_consensus.cc:2889] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:04:35.071520 11709 tablet_bootstrap.cc:492] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:04:35.072330 11709 tablet_bootstrap.cc:492] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2: Bootstrap complete.
I20260709 13:04:35.073098 11709 ts_tablet_manager.cc:1403] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.000s
I20260709 13:04:35.075682 11709 raft_consensus.cc:359] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.076294 11709 raft_consensus.cc:740] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc3230c8b0704c5ab54fd82d58545db2, State: Initialized, Role: FOLLOWER
I20260709 13:04:35.076936 11709 consensus_queue.cc:260] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.078444 11709 ts_tablet_manager.cc:1434] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:04:35.156864 11633 raft_consensus.cc:993] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20: : Instructing follower a0690c7fd263405a9adb36b9bd4f2f27 to start an election
I20260709 13:04:35.157356 11789 raft_consensus.cc:1081] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 LEADER]: Signalling peer a0690c7fd263405a9adb36b9bd4f2f27 to start an election
I20260709 13:04:35.159070 11468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1f216ed20c2043bf82cedeeeab582b59"
dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
 from {username='slave'} at 127.0.0.1:56446
I20260709 13:04:35.159567 11468 raft_consensus.cc:493] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:35.159821 11468 raft_consensus.cc:3060] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.165187 11468 raft_consensus.cc:515] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.166941 11468 leader_election.cc:290] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 3 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:35.167702 11746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f216ed20c2043bf82cedeeeab582b59" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:35.167902 11541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f216ed20c2043bf82cedeeeab582b59" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:35.168454 11746 raft_consensus.cc:3060] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.168536 11541 raft_consensus.cc:3055] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:35.168987 11541 raft_consensus.cc:740] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:35.169709 11541 consensus_queue.cc:260] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.170738 11541 raft_consensus.cc:3060] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.174659 11746 raft_consensus.cc:2468] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 3.
I20260709 13:04:35.175839 11541 raft_consensus.cc:2468] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 3.
I20260709 13:04:35.176092 11434 leader_election.cc:304] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:35.176975 11608 raft_consensus.cc:2804] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:35.178064 11608 raft_consensus.cc:697] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 LEADER]: Becoming Leader. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:35.178884 11608 consensus_queue.cc:237] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.185488 11291 catalog_manager.cc:5697] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 reported cstate change: term changed from 2 to 3, leader changed from 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195) to a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:35.303732 11746 raft_consensus.cc:3060] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:04:35.405275 11790 raft_consensus.cc:993] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27: : Instructing follower 3ece719b875a475eaa0c79ffde68cb20 to start an election
I20260709 13:04:35.405879 11608 raft_consensus.cc:1081] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 LEADER]: Signalling peer 3ece719b875a475eaa0c79ffde68cb20 to start an election
I20260709 13:04:35.407626 11541 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92b5d1a8d996415982df5ed08c242522"
dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
 from {username='slave'} at 127.0.0.1:51290
I20260709 13:04:35.408124 11541 raft_consensus.cc:493] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:35.408458 11541 raft_consensus.cc:3060] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:35.413820 11541 raft_consensus.cc:515] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.415434 11541 leader_election.cc:290] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:35.416407 11746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b5d1a8d996415982df5ed08c242522" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:35.416527 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92b5d1a8d996415982df5ed08c242522" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:35.417109 11746 raft_consensus.cc:3060] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:35.417238 11468 raft_consensus.cc:3055] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:35.417764 11468 raft_consensus.cc:740] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:35.418597 11468 consensus_queue.cc:260] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.419971 11468 raft_consensus.cc:3060] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:35.424461 11746 raft_consensus.cc:2468] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 2.
I20260709 13:04:35.425740 11508 leader_election.cc:304] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:35.426512 11789 raft_consensus.cc:2804] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:35.427019 11789 raft_consensus.cc:697] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 LEADER]: Becoming Leader. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:35.427839 11789 consensus_queue.cc:237] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:35.431171 11468 raft_consensus.cc:2468] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 2.
I20260709 13:04:35.435283 11290 catalog_manager.cc:5697] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 reported cstate change: term changed from 1 to 2, leader changed from a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194) to 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "3ece719b875a475eaa0c79ffde68cb20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: HEALTHY } } }
W20260709 13:04:36.154134 11434 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:46191, user_credentials={real_user=slave}} blocked reactor thread for 72884.2us
I20260709 13:04:36.170347   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:04:36.757253 11340 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:04:36.763162 11805 raft_consensus.cc:493] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ece719b875a475eaa0c79ffde68cb20)
I20260709 13:04:36.763835 11805 raft_consensus.cc:515] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:36.766189 11805 leader_election.cc:290] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:36.794569 11746 raft_consensus.cc:1275] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Refusing update from remote peer a0690c7fd263405a9adb36b9bd4f2f27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:36.796124 11805 consensus_queue.cc:1048] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:36.798214 11711 raft_consensus.cc:493] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a0690c7fd263405a9adb36b9bd4f2f27)
I20260709 13:04:36.798866 11711 raft_consensus.cc:515] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:36.801560 11711 leader_election.cc:290] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:36.808835 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2130ecf60b94295aff5ea3f1aeafd98" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:36.810954 11711 raft_consensus.cc:493] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a0690c7fd263405a9adb36b9bd4f2f27)
I20260709 13:04:36.815589 11711 raft_consensus.cc:515] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:36.817248 11743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2130ecf60b94295aff5ea3f1aeafd98" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2" is_pre_election: true
I20260709 13:04:36.818415 11711 leader_election.cc:290] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:36.816016 11744 raft_consensus.cc:1275] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Refusing update from remote peer 3ece719b875a475eaa0c79ffde68cb20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:36.820004 11434 leader_election.cc:304] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27; no voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:36.820502 11806 raft_consensus.cc:493] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ece719b875a475eaa0c79ffde68cb20)
I20260709 13:04:36.821184 11806 raft_consensus.cc:515] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:36.821626 11799 raft_consensus.cc:2749] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:36.823491 11806 leader_election.cc:290] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:36.830739 11803 consensus_queue.cc:1048] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:36.839862 11608 raft_consensus.cc:493] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 3ece719b875a475eaa0c79ffde68cb20)
I20260709 13:04:36.840548 11608 raft_consensus.cc:515] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:36.845018 11608 leader_election.cc:290] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:36.864004 11542 raft_consensus.cc:1275] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Refusing update from remote peer a0690c7fd263405a9adb36b9bd4f2f27: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:36.864959 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:36.848024 11541 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab"
dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
mode: GRACEFUL
new_leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
 from {username='slave'} at 127.0.0.1:53360
I20260709 13:04:36.867379 11541 raft_consensus.cc:606] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Received request to transfer leadership to TS a0690c7fd263405a9adb36b9bd4f2f27
I20260709 13:04:36.867645 11743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2" is_pre_election: true
I20260709 13:04:36.876295 11469 raft_consensus.cc:1275] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Refusing update from remote peer 3ece719b875a475eaa0c79ffde68cb20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:36.877331 11465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f7a03a51a3ed458baa6e9dbc849793cc"
dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
mode: GRACEFUL
new_leader_uuid: "3ece719b875a475eaa0c79ffde68cb20"
 from {username='slave'} at 127.0.0.1:37842
I20260709 13:04:36.878082 11465 raft_consensus.cc:606] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 LEADER]: Received request to transfer leadership to TS 3ece719b875a475eaa0c79ffde68cb20
I20260709 13:04:36.880409 11434 leader_election.cc:304] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27; no voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2
I20260709 13:04:36.879952 11608 consensus_queue.cc:1048] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:36.882412 11608 raft_consensus.cc:2749] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:36.882640 11543 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc846bd430d248fd812392acd98e9ac1" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:36.883093 11541 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd340b80e0bf42e080b36269ec8b5c4a" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
I20260709 13:04:36.880541   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:36.884380   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:36.885149   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:36.885818 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cc664f62b248ec92a2cef483c2c665" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "3ece719b875a475eaa0c79ffde68cb20" is_pre_election: true
W20260709 13:04:36.908522 11323 debug-util.cc:398] Leaking SignalData structure 0x7b08000feae0 after lost signal to thread 434
W20260709 13:04:36.909922 11323 debug-util.cc:398] Leaking SignalData structure 0x7b08002fb2c0 after lost signal to thread 11326
I20260709 13:04:36.910657 11800 consensus_queue.cc:1048] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:36.914376 11789 raft_consensus.cc:993] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20: : Instructing follower a0690c7fd263405a9adb36b9bd4f2f27 to start an election
W20260709 13:04:36.914546 11323 debug-util.cc:398] Leaking SignalData structure 0x7b0800322be0 after lost signal to thread 11772
I20260709 13:04:36.914886 11633 raft_consensus.cc:1081] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Signalling peer a0690c7fd263405a9adb36b9bd4f2f27 to start an election
I20260709 13:04:36.919270 11465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab"
dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
 from {username='slave'} at 127.0.0.1:56446
I20260709 13:04:36.920130 11465 raft_consensus.cc:493] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:36.920672 11465 raft_consensus.cc:3060] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:36.932166 11465 raft_consensus.cc:515] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:36.939523 11744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:36.940235 11744 raft_consensus.cc:3060] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:36.943568 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:36.944324 11542 raft_consensus.cc:3055] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:36.944664 11542 raft_consensus.cc:740] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:36.945580 11542 consensus_queue.cc:260] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:36.946830 11542 raft_consensus.cc:3060] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:36.948101 11744 raft_consensus.cc:2468] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 2.
I20260709 13:04:36.950210 11434 leader_election.cc:304] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:36.951100 11790 raft_consensus.cc:2804] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:36.952215 11790 raft_consensus.cc:697] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Becoming Leader. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:36.953123 11790 consensus_queue.cc:237] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:36.955250 11542 raft_consensus.cc:2468] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 2.
I20260709 13:04:36.956403 11465 leader_election.cc:290] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:36.957645 11466 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4cc664f62b248ec92a2cef483c2c665" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
I20260709 13:04:36.958847 11468 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc846bd430d248fd812392acd98e9ac1" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
I20260709 13:04:36.959681 11705 leader_election.cc:304] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc3230c8b0704c5ab54fd82d58545db2; no voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27
I20260709 13:04:36.960593 11806 raft_consensus.cc:2749] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:04:36.961128 11705 leader_election.cc:304] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc3230c8b0704c5ab54fd82d58545db2; no voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27
I20260709 13:04:36.961874 11806 raft_consensus.cc:2749] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:36.962604 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd340b80e0bf42e080b36269ec8b5c4a" candidate_uuid: "dc3230c8b0704c5ab54fd82d58545db2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" is_pre_election: true
I20260709 13:04:36.964063 11705 leader_election.cc:304] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc3230c8b0704c5ab54fd82d58545db2; no voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27
I20260709 13:04:36.965600 11806 raft_consensus.cc:2749] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:36.969882 11289 catalog_manager.cc:5697] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 reported cstate change: term changed from 1 to 2, leader changed from 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195) to a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:37.221024 11608 raft_consensus.cc:993] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27: : Instructing follower 3ece719b875a475eaa0c79ffde68cb20 to start an election
I20260709 13:04:37.221556 11790 raft_consensus.cc:1081] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 LEADER]: Signalling peer 3ece719b875a475eaa0c79ffde68cb20 to start an election
I20260709 13:04:37.223374 11542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f7a03a51a3ed458baa6e9dbc849793cc"
dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
 from {username='slave'} at 127.0.0.1:51290
I20260709 13:04:37.223922 11542 raft_consensus.cc:493] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:37.224232 11542 raft_consensus.cc:3060] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:37.230077 11542 raft_consensus.cc:515] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:37.232259 11542 leader_election.cc:290] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:37.232929 11744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a03a51a3ed458baa6e9dbc849793cc" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:37.233289 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7a03a51a3ed458baa6e9dbc849793cc" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:37.233705 11744 raft_consensus.cc:3060] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:37.233944 11469 raft_consensus.cc:3055] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:37.234385 11469 raft_consensus.cc:740] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:37.235234 11469 consensus_queue.cc:260] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:37.236613 11469 raft_consensus.cc:3060] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:37.242259 11469 raft_consensus.cc:2468] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 2.
I20260709 13:04:37.242796 11744 raft_consensus.cc:2468] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 2.
I20260709 13:04:37.243353 11508 leader_election.cc:304] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, a0690c7fd263405a9adb36b9bd4f2f27; no voters: 
I20260709 13:04:37.244165 11803 raft_consensus.cc:2804] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:37.244690 11803 raft_consensus.cc:697] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [term 2 LEADER]: Becoming Leader. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:37.245641 11803 consensus_queue.cc:237] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:37.254531 11289 catalog_manager.cc:5697] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 reported cstate change: term changed from 1 to 2, leader changed from a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194) to 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "3ece719b875a475eaa0c79ffde68cb20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:37.369649 11542 raft_consensus.cc:1275] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Refusing update from remote peer a0690c7fd263405a9adb36b9bd4f2f27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:37.371928 11790 consensus_queue.cc:1048] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:37.381783 11744 raft_consensus.cc:1275] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Refusing update from remote peer a0690c7fd263405a9adb36b9bd4f2f27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:37.385103 11805 consensus_queue.cc:1048] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:37.627883 11744 raft_consensus.cc:1275] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Refusing update from remote peer 3ece719b875a475eaa0c79ffde68cb20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:37.629336 11803 consensus_queue.cc:1048] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:37.641966 11469 raft_consensus.cc:1275] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Refusing update from remote peer 3ece719b875a475eaa0c79ffde68cb20: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:37.643765 11789 consensus_queue.cc:1048] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:37.758677 11340 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:04:37.895514   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:37.904059 11542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92cd1ee5127a4f578b1f2f9125999904"
dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
mode: GRACEFUL
new_leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
 from {username='slave'} at 127.0.0.1:53360
I20260709 13:04:37.904693 11542 raft_consensus.cc:606] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Received request to transfer leadership to TS a0690c7fd263405a9adb36b9bd4f2f27
I20260709 13:04:37.907452 11469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab"
dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
mode: GRACEFUL
new_leader_uuid: "3ece719b875a475eaa0c79ffde68cb20"
 from {username='slave'} at 127.0.0.1:37842
I20260709 13:04:37.907991 11469 raft_consensus.cc:606] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Received request to transfer leadership to TS 3ece719b875a475eaa0c79ffde68cb20
I20260709 13:04:37.909366   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:37.910027   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:37.910638   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:37.949139 11799 raft_consensus.cc:993] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27: : Instructing follower 3ece719b875a475eaa0c79ffde68cb20 to start an election
I20260709 13:04:37.949777 11805 raft_consensus.cc:1081] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Signalling peer 3ece719b875a475eaa0c79ffde68cb20 to start an election
I20260709 13:04:37.951391 11542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab"
dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
 from {username='slave'} at 127.0.0.1:51290
I20260709 13:04:37.951911 11542 raft_consensus.cc:493] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:37.952225 11542 raft_consensus.cc:3060] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:37.957350 11542 raft_consensus.cc:515] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:37.959266 11542 leader_election.cc:290] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 3 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:37.960139 11744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:37.960350 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:37.960803 11744 raft_consensus.cc:3060] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:37.961059 11469 raft_consensus.cc:3055] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:37.961547 11469 raft_consensus.cc:740] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:37.962445 11469 consensus_queue.cc:260] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:37.963866 11469 raft_consensus.cc:3060] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:37.966492 11744 raft_consensus.cc:2468] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 3.
I20260709 13:04:37.967717 11508 leader_election.cc:304] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:37.968402 11789 raft_consensus.cc:2804] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:37.968861 11789 raft_consensus.cc:697] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 3 LEADER]: Becoming Leader. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:37.969709 11789 consensus_queue.cc:237] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:37.972450 11469 raft_consensus.cc:2468] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 3.
I20260709 13:04:37.977097 11289 catalog_manager.cc:5697] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 reported cstate change: term changed from 2 to 3, leader changed from a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194) to 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "3ece719b875a475eaa0c79ffde68cb20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:38.225893 11834 raft_consensus.cc:993] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20: : Instructing follower a0690c7fd263405a9adb36b9bd4f2f27 to start an election
I20260709 13:04:38.226402 11834 raft_consensus.cc:1081] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Signalling peer a0690c7fd263405a9adb36b9bd4f2f27 to start an election
I20260709 13:04:38.228060 11469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92cd1ee5127a4f578b1f2f9125999904"
dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
 from {username='slave'} at 127.0.0.1:56446
I20260709 13:04:38.228696 11469 raft_consensus.cc:493] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:38.229135 11469 raft_consensus.cc:3060] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:38.237802 11469 raft_consensus.cc:515] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:38.240147 11469 leader_election.cc:290] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:38.240932 11744 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92cd1ee5127a4f578b1f2f9125999904" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:38.241206 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92cd1ee5127a4f578b1f2f9125999904" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:38.241659 11744 raft_consensus.cc:3060] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:38.241794 11542 raft_consensus.cc:3055] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:38.242251 11542 raft_consensus.cc:740] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:38.243116 11542 consensus_queue.cc:260] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:38.244431 11542 raft_consensus.cc:3060] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:38.249245 11744 raft_consensus.cc:2468] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 2.
I20260709 13:04:38.250655 11434 leader_election.cc:304] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:38.251680 11805 raft_consensus.cc:2804] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:38.252178 11805 raft_consensus.cc:697] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Becoming Leader. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:38.253015 11805 consensus_queue.cc:237] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:38.253223 11542 raft_consensus.cc:2468] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 2.
I20260709 13:04:38.262343 11289 catalog_manager.cc:5697] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 reported cstate change: term changed from 1 to 2, leader changed from 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195) to a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:38.576188 11744 raft_consensus.cc:1275] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Refusing update from remote peer 3ece719b875a475eaa0c79ffde68cb20: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:38.576376 11469 raft_consensus.cc:1275] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Refusing update from remote peer 3ece719b875a475eaa0c79ffde68cb20: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:38.578931 11834 consensus_queue.cc:1048] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:38.580477 11633 consensus_queue.cc:1048] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:04:38.760221 11340 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:04:38.866420 11744 raft_consensus.cc:1275] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Refusing update from remote peer a0690c7fd263405a9adb36b9bd4f2f27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:38.868453 11799 consensus_queue.cc:1048] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:38.869555 11542 raft_consensus.cc:1275] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Refusing update from remote peer a0690c7fd263405a9adb36b9bd4f2f27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:38.871173 11799 consensus_queue.cc:1048] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:38.918715   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:38.929986 11469 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb34dc48d7194d7bb15ecb7094dabf43"
dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
mode: GRACEFUL
new_leader_uuid: "3ece719b875a475eaa0c79ffde68cb20"
 from {username='slave'} at 127.0.0.1:37842
I20260709 13:04:38.930665 11469 raft_consensus.cc:606] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Received request to transfer leadership to TS 3ece719b875a475eaa0c79ffde68cb20
I20260709 13:04:38.933849 11542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd340b80e0bf42e080b36269ec8b5c4a"
dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
mode: GRACEFUL
new_leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
 from {username='slave'} at 127.0.0.1:53360
I20260709 13:04:38.934412 11542 raft_consensus.cc:606] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Received request to transfer leadership to TS a0690c7fd263405a9adb36b9bd4f2f27
I20260709 13:04:38.936048   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:38.936630   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:38.937655   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:39.216976 11799 raft_consensus.cc:993] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27: : Instructing follower 3ece719b875a475eaa0c79ffde68cb20 to start an election
I20260709 13:04:39.217643 11845 raft_consensus.cc:1081] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Signalling peer 3ece719b875a475eaa0c79ffde68cb20 to start an election
I20260709 13:04:39.219612 11542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb34dc48d7194d7bb15ecb7094dabf43"
dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
 from {username='slave'} at 127.0.0.1:51290
I20260709 13:04:39.220228 11542 raft_consensus.cc:493] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:39.220625 11542 raft_consensus.cc:3060] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:39.230937 11542 raft_consensus.cc:515] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:39.233129 11542 leader_election.cc:290] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 3 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491)
I20260709 13:04:39.234184 11746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb34dc48d7194d7bb15ecb7094dabf43" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:39.235044 11746 raft_consensus.cc:3060] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:39.235268 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb34dc48d7194d7bb15ecb7094dabf43" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:39.235929 11469 raft_consensus.cc:3055] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:39.236248 11469 raft_consensus.cc:740] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:39.237545 11469 consensus_queue.cc:260] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:39.239198 11469 raft_consensus.cc:3060] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:39.245481 11746 raft_consensus.cc:2468] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 3.
I20260709 13:04:39.247289 11508 leader_election.cc:304] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:39.248194 11633 raft_consensus.cc:2804] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:39.248883 11469 raft_consensus.cc:2468] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 3.
I20260709 13:04:39.249902 11633 raft_consensus.cc:697] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 LEADER]: Becoming Leader. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:39.251011 11633 consensus_queue.cc:237] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:39.259493 11291 catalog_manager.cc:5697] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 reported cstate change: term changed from 2 to 3, leader changed from a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194) to 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "3ece719b875a475eaa0c79ffde68cb20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:39.337602 11789 raft_consensus.cc:993] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20: : Instructing follower a0690c7fd263405a9adb36b9bd4f2f27 to start an election
I20260709 13:04:39.338179 11633 raft_consensus.cc:1081] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Signalling peer a0690c7fd263405a9adb36b9bd4f2f27 to start an election
I20260709 13:04:39.340080 11469 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd340b80e0bf42e080b36269ec8b5c4a"
dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
 from {username='slave'} at 127.0.0.1:56446
I20260709 13:04:39.340606 11469 raft_consensus.cc:493] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:39.340978 11469 raft_consensus.cc:3060] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.347177 11469 raft_consensus.cc:515] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:39.348812 11469 leader_election.cc:290] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:39.349851 11746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd340b80e0bf42e080b36269ec8b5c4a" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:39.350164 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd340b80e0bf42e080b36269ec8b5c4a" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:39.350543 11746 raft_consensus.cc:3060] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.350924 11542 raft_consensus.cc:3055] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:39.351337 11542 raft_consensus.cc:740] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:39.352511 11542 consensus_queue.cc:260] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:39.355504 11542 raft_consensus.cc:3060] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:39.358431 11746 raft_consensus.cc:2468] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 2.
I20260709 13:04:39.362551 11434 leader_election.cc:304] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:39.363538 11845 raft_consensus.cc:2804] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:39.364054 11845 raft_consensus.cc:697] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Becoming Leader. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:39.365442 11542 raft_consensus.cc:2468] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 2.
I20260709 13:04:39.365350 11845 consensus_queue.cc:237] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:39.374462 11291 catalog_manager.cc:5697] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 reported cstate change: term changed from 1 to 2, leader changed from 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195) to a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:39.684773 11468 raft_consensus.cc:1275] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Refusing update from remote peer 3ece719b875a475eaa0c79ffde68cb20: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:39.686226 11789 consensus_queue.cc:1048] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:39.705180 11746 raft_consensus.cc:1275] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Refusing update from remote peer 3ece719b875a475eaa0c79ffde68cb20: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:04:39.706660 11789 consensus_queue.cc:1048] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:04:39.761391 11340 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:04:39.785770 11542 raft_consensus.cc:1275] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Refusing update from remote peer a0690c7fd263405a9adb36b9bd4f2f27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.787178 11845 consensus_queue.cc:1048] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:39.801715 11746 raft_consensus.cc:1275] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Refusing update from remote peer a0690c7fd263405a9adb36b9bd4f2f27: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:39.803303 11845 consensus_queue.cc:1048] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:39.945062   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:39.952590 11468 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "92cd1ee5127a4f578b1f2f9125999904"
dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
mode: GRACEFUL
new_leader_uuid: "3ece719b875a475eaa0c79ffde68cb20"
 from {username='slave'} at 127.0.0.1:37842
I20260709 13:04:39.953194 11468 raft_consensus.cc:606] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Received request to transfer leadership to TS 3ece719b875a475eaa0c79ffde68cb20
I20260709 13:04:39.956023 11542 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab"
dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
mode: GRACEFUL
new_leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
 from {username='slave'} at 127.0.0.1:53360
I20260709 13:04:39.956576 11542 raft_consensus.cc:606] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 3 LEADER]: Received request to transfer leadership to TS a0690c7fd263405a9adb36b9bd4f2f27
I20260709 13:04:39.957908   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:39.958423   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:39.959002   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:40.071877 11789 raft_consensus.cc:993] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20: : Instructing follower a0690c7fd263405a9adb36b9bd4f2f27 to start an election
I20260709 13:04:40.072523 11633 raft_consensus.cc:1081] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 3 LEADER]: Signalling peer a0690c7fd263405a9adb36b9bd4f2f27 to start an election
I20260709 13:04:40.074146 11468 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab"
dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
 from {username='slave'} at 127.0.0.1:56446
I20260709 13:04:40.074659 11468 raft_consensus.cc:493] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:40.075009 11468 raft_consensus.cc:3060] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:40.082298 11468 raft_consensus.cc:515] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:40.084299 11468 leader_election.cc:290] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 4 election: Requested vote from peers dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057), 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195:46191)
I20260709 13:04:40.085215 11746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:40.085984 11746 raft_consensus.cc:3060] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:40.086445 11542 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4891fec6e634dc7ade7db078a5f3bab" candidate_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
I20260709 13:04:40.087106 11542 raft_consensus.cc:3055] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:04:40.087407 11542 raft_consensus.cc:740] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:40.088379 11542 consensus_queue.cc:260] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:40.089829 11542 raft_consensus.cc:3060] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:04:40.094102 11746 raft_consensus.cc:2468] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 4.
I20260709 13:04:40.095466 11434 leader_election.cc:304] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0690c7fd263405a9adb36b9bd4f2f27, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:40.096323 11845 raft_consensus.cc:2804] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:04:40.096800 11845 raft_consensus.cc:697] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 4 LEADER]: Becoming Leader. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:40.097505 11542 raft_consensus.cc:2468] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0690c7fd263405a9adb36b9bd4f2f27 in term 4.
I20260709 13:04:40.097854 11845 consensus_queue.cc:237] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:40.106442 11291 catalog_manager.cc:5697] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 reported cstate change: term changed from 3 to 4, leader changed from 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195) to a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194). New cstate: current_term: 4 leader_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:40.346089 11799 raft_consensus.cc:993] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27: : Instructing follower 3ece719b875a475eaa0c79ffde68cb20 to start an election
I20260709 13:04:40.346534 11799 raft_consensus.cc:1081] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Signalling peer 3ece719b875a475eaa0c79ffde68cb20 to start an election
I20260709 13:04:40.347956 11542 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "92cd1ee5127a4f578b1f2f9125999904"
dest_uuid: "3ece719b875a475eaa0c79ffde68cb20"
 from {username='slave'} at 127.0.0.1:51290
I20260709 13:04:40.348398 11542 raft_consensus.cc:493] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:40.348645 11542 raft_consensus.cc:3060] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:40.355137 11542 raft_consensus.cc:515] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:40.356714 11542 leader_election.cc:290] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 3 election: Requested vote from peers a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491), dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057)
I20260709 13:04:40.357600 11469 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92cd1ee5127a4f578b1f2f9125999904" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a0690c7fd263405a9adb36b9bd4f2f27"
I20260709 13:04:40.357957 11746 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92cd1ee5127a4f578b1f2f9125999904" candidate_uuid: "3ece719b875a475eaa0c79ffde68cb20" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dc3230c8b0704c5ab54fd82d58545db2"
I20260709 13:04:40.358237 11469 raft_consensus.cc:3055] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:40.358577 11746 raft_consensus.cc:3060] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:40.358592 11469 raft_consensus.cc:740] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a0690c7fd263405a9adb36b9bd4f2f27, State: Running, Role: LEADER
I20260709 13:04:40.359557 11469 consensus_queue.cc:260] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:40.360486 11469 raft_consensus.cc:3060] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:40.364418 11746 raft_consensus.cc:2468] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 3.
I20260709 13:04:40.365756 11508 leader_election.cc:304] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ece719b875a475eaa0c79ffde68cb20, dc3230c8b0704c5ab54fd82d58545db2; no voters: 
I20260709 13:04:40.365772 11469 raft_consensus.cc:2468] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ece719b875a475eaa0c79ffde68cb20 in term 3.
I20260709 13:04:40.366753 11865 raft_consensus.cc:2804] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:40.367187 11865 raft_consensus.cc:697] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 LEADER]: Becoming Leader. State: Replica: 3ece719b875a475eaa0c79ffde68cb20, State: Running, Role: LEADER
I20260709 13:04:40.368098 11865 consensus_queue.cc:237] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } }
I20260709 13:04:40.374754 11291 catalog_manager.cc:5697] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 reported cstate change: term changed from 2 to 3, leader changed from a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194) to 3ece719b875a475eaa0c79ffde68cb20 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "3ece719b875a475eaa0c79ffde68cb20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:40.674256 11746 raft_consensus.cc:1275] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 4 FOLLOWER]: Refusing update from remote peer a0690c7fd263405a9adb36b9bd4f2f27: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:40.675668 11866 consensus_queue.cc:1048] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:40.686278 11542 raft_consensus.cc:1275] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 4 FOLLOWER]: Refusing update from remote peer a0690c7fd263405a9adb36b9bd4f2f27: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:04:40.688913 11866 consensus_queue.cc:1048] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ece719b875a475eaa0c79ffde68cb20" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 46191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:04:40.762408 11340 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:04:40.784099 11746 raft_consensus.cc:1275] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Refusing update from remote peer 3ece719b875a475eaa0c79ffde68cb20: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:40.785542 11865 consensus_queue.cc:1048] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc3230c8b0704c5ab54fd82d58545db2" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:40.794337 11469 raft_consensus.cc:1275] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Refusing update from remote peer 3ece719b875a475eaa0c79ffde68cb20: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:40.796392 11875 consensus_queue.cc:1048] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0690c7fd263405a9adb36b9bd4f2f27" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 45491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:04:40.966327   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:40.995535   431 tablet_server.cc:179] TabletServer@127.0.107.193:38057 shutting down...
I20260709 13:04:41.028275   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:41.029138   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:41.030203   431 raft_consensus.cc:2243] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.030717   431 raft_consensus.cc:2272] T fd340b80e0bf42e080b36269ec8b5c4a P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.033216   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:41.033962   431 raft_consensus.cc:2243] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.034482   431 raft_consensus.cc:2272] T 92b5d1a8d996415982df5ed08c242522 P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.036810   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:41.037410   431 raft_consensus.cc:2243] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.037912   431 raft_consensus.cc:2272] T a2130ecf60b94295aff5ea3f1aeafd98 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.040367   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:41.040956   431 raft_consensus.cc:2243] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.041374   431 raft_consensus.cc:2272] T bc846bd430d248fd812392acd98e9ac1 P dc3230c8b0704c5ab54fd82d58545db2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.043867   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:41.044416   431 raft_consensus.cc:2243] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.044879   431 raft_consensus.cc:2272] T d4cc664f62b248ec92a2cef483c2c665 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:41.047783 11508 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 92 similar messages]
I20260709 13:04:41.048509   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:41.049467   431 raft_consensus.cc:2243] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.050060   431 raft_consensus.cc:2272] T f4891fec6e634dc7ade7db078a5f3bab P dc3230c8b0704c5ab54fd82d58545db2 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:41.051048 11508 consensus_peers.cc:597] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:41.053946   431 tablet_replica.cc:333] stopping tablet replica
W20260709 13:04:41.054804 11434 consensus_peers.cc:597] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:41.054905   431 raft_consensus.cc:2243] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.055533   431 raft_consensus.cc:2272] T f7a03a51a3ed458baa6e9dbc849793cc P dc3230c8b0704c5ab54fd82d58545db2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.058158   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:41.058761   431 raft_consensus.cc:2243] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.059195   431 raft_consensus.cc:2272] T 92cd1ee5127a4f578b1f2f9125999904 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.061991   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:41.062578   431 raft_consensus.cc:2243] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.063043   431 raft_consensus.cc:2272] T fb34dc48d7194d7bb15ecb7094dabf43 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.065625   431 tablet_replica.cc:333] stopping tablet replica
I20260709 13:04:41.066176   431 raft_consensus.cc:2243] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.066697   431 raft_consensus.cc:2272] T 1f216ed20c2043bf82cedeeeab582b59 P dc3230c8b0704c5ab54fd82d58545db2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.092749   431 tablet_server.cc:196] TabletServer@127.0.107.193:38057 shutdown complete.
I20260709 13:04:41.121276   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:04:41.126597 11508 consensus_peers.cc:597] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 -> Peer a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491): Couldn't send request to peer a0690c7fd263405a9adb36b9bd4f2f27. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:41.145331   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:41.146224   431 tablet_replica.cc:333] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27: stopping tablet replica
I20260709 13:04:41.147011   431 raft_consensus.cc:2243] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.147554   431 raft_consensus.cc:2272] T a2130ecf60b94295aff5ea3f1aeafd98 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.150082   431 tablet_replica.cc:333] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27: stopping tablet replica
I20260709 13:04:41.150763   431 raft_consensus.cc:2243] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.151276   431 raft_consensus.cc:2272] T 92b5d1a8d996415982df5ed08c242522 P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.153646   431 tablet_replica.cc:333] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27: stopping tablet replica
I20260709 13:04:41.154186   431 raft_consensus.cc:2243] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.155084   431 raft_consensus.cc:2272] T 1f216ed20c2043bf82cedeeeab582b59 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.157133   431 tablet_replica.cc:333] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27: stopping tablet replica
I20260709 13:04:41.157722   431 raft_consensus.cc:2243] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.158701   431 raft_consensus.cc:2272] T bc846bd430d248fd812392acd98e9ac1 P a0690c7fd263405a9adb36b9bd4f2f27 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.161131   431 tablet_replica.cc:333] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27: stopping tablet replica
I20260709 13:04:41.161756   431 raft_consensus.cc:2243] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.162657   431 raft_consensus.cc:2272] T d4cc664f62b248ec92a2cef483c2c665 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.164696   431 tablet_replica.cc:333] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27: stopping tablet replica
I20260709 13:04:41.165251   431 raft_consensus.cc:2243] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.165716   431 raft_consensus.cc:2272] T fb34dc48d7194d7bb15ecb7094dabf43 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.167737   431 tablet_replica.cc:333] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27: stopping tablet replica
I20260709 13:04:41.168311   431 raft_consensus.cc:2243] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.168834   431 raft_consensus.cc:2272] T f7a03a51a3ed458baa6e9dbc849793cc P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.171224   431 tablet_replica.cc:333] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27: stopping tablet replica
I20260709 13:04:41.171737   431 raft_consensus.cc:2243] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.172164   431 raft_consensus.cc:2272] T 92cd1ee5127a4f578b1f2f9125999904 P a0690c7fd263405a9adb36b9bd4f2f27 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.174953   431 tablet_replica.cc:333] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27: stopping tablet replica
I20260709 13:04:41.175479   431 raft_consensus.cc:2243] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.176275   431 raft_consensus.cc:2272] T fd340b80e0bf42e080b36269ec8b5c4a P a0690c7fd263405a9adb36b9bd4f2f27 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.178391   431 tablet_replica.cc:333] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27: stopping tablet replica
I20260709 13:04:41.178854   431 raft_consensus.cc:2243] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.179613   431 raft_consensus.cc:2272] T f4891fec6e634dc7ade7db078a5f3bab P a0690c7fd263405a9adb36b9bd4f2f27 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:41.193090 11508 consensus_peers.cc:597] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:41.215593   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:04:41.230682 11508 consensus_peers.cc:597] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:41.244225   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:04:41.263590 11508 consensus_peers.cc:597] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 -> Peer a0690c7fd263405a9adb36b9bd4f2f27 (127.0.107.194:45491): Couldn't send request to peer a0690c7fd263405a9adb36b9bd4f2f27. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:45491: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:41.278191   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:41.278869   431 tablet_replica.cc:333] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20: stopping tablet replica
I20260709 13:04:41.279642   431 raft_consensus.cc:2243] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.280644   431 raft_consensus.cc:2272] T a2130ecf60b94295aff5ea3f1aeafd98 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.283730   431 tablet_replica.cc:333] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20: stopping tablet replica
I20260709 13:04:41.284688   431 raft_consensus.cc:2243] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.285974   431 raft_consensus.cc:2272] T 92b5d1a8d996415982df5ed08c242522 P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:41.286753 11508 consensus_peers.cc:597] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 -> Peer dc3230c8b0704c5ab54fd82d58545db2 (127.0.107.193:38057): Couldn't send request to peer dc3230c8b0704c5ab54fd82d58545db2. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:38057: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:41.288600   431 tablet_replica.cc:333] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20: stopping tablet replica
I20260709 13:04:41.289343   431 raft_consensus.cc:2243] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.289839   431 raft_consensus.cc:2272] T fd340b80e0bf42e080b36269ec8b5c4a P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.292106   431 tablet_replica.cc:333] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20: stopping tablet replica
I20260709 13:04:41.292797   431 raft_consensus.cc:2243] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.293339   431 raft_consensus.cc:2272] T d4cc664f62b248ec92a2cef483c2c665 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.295666   431 tablet_replica.cc:333] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20: stopping tablet replica
I20260709 13:04:41.296243   431 raft_consensus.cc:2243] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.296729   431 raft_consensus.cc:2272] T bc846bd430d248fd812392acd98e9ac1 P 3ece719b875a475eaa0c79ffde68cb20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.299044   431 tablet_replica.cc:333] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20: stopping tablet replica
I20260709 13:04:41.299743   431 raft_consensus.cc:2243] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.300297   431 raft_consensus.cc:2272] T f4891fec6e634dc7ade7db078a5f3bab P 3ece719b875a475eaa0c79ffde68cb20 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.303113   431 tablet_replica.cc:333] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20: stopping tablet replica
I20260709 13:04:41.303699   431 raft_consensus.cc:2243] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.304520   431 raft_consensus.cc:2272] T fb34dc48d7194d7bb15ecb7094dabf43 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.306659   431 tablet_replica.cc:333] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20: stopping tablet replica
I20260709 13:04:41.307190   431 raft_consensus.cc:2243] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.307988   431 raft_consensus.cc:2272] T 92cd1ee5127a4f578b1f2f9125999904 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.310583   431 tablet_replica.cc:333] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20: stopping tablet replica
I20260709 13:04:41.311125   431 raft_consensus.cc:2243] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.312052   431 raft_consensus.cc:2272] T f7a03a51a3ed458baa6e9dbc849793cc P 3ece719b875a475eaa0c79ffde68cb20 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.314308   431 tablet_replica.cc:333] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20: stopping tablet replica
I20260709 13:04:41.314810   431 raft_consensus.cc:2243] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:41.315375   431 raft_consensus.cc:2272] T 1f216ed20c2043bf82cedeeeab582b59 P 3ece719b875a475eaa0c79ffde68cb20 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.338581   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:04:41.362756   431 master.cc:562] Master@127.0.107.254:38461 shutting down...
I20260709 13:04:41.384485   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:41.385047   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:41.385313   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 94f0028b98c7416eb5e86fe165dc3e20: stopping tablet replica
I20260709 13:04:41.424639   431 master.cc:584] Master@127.0.107.254:38461 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (14071 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:04:41.475193   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:34079
I20260709 13:04:41.476322   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:41.482048 11882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:41.482618 11883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:41.484543   431 server_base.cc:1061] running on GCE node
W20260709 13:04:41.484828 11885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:41.485900   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:41.486136   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:41.486299   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602281486278 us; error 0 us; skew 500 ppm
I20260709 13:04:41.486832   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:41.489279   431 webserver.cc:533] Webserver started at http://127.0.107.254:40477/ using document root <none> and password file <none>
I20260709 13:04:41.489854   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:41.490062   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:41.490338   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:41.491540   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/master-0-root/instance:
uuid: "0e70ea34673d4753b38665330e3cd308"
format_stamp: "Formatted at 2026-07-09 13:04:41 on dist-test-slave-jk05"
I20260709 13:04:41.496418   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:04:41.500165 11890 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:41.501045   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:04:41.501362   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/master-0-root
uuid: "0e70ea34673d4753b38665330e3cd308"
format_stamp: "Formatted at 2026-07-09 13:04:41 on dist-test-slave-jk05"
I20260709 13:04:41.501735   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:41.516216   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:41.517637   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:41.556685   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:34079
I20260709 13:04:41.556813 11941 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:34079 every 8 connection(s)
I20260709 13:04:41.561144 11942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:41.572822 11942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308: Bootstrap starting.
I20260709 13:04:41.577816 11942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:41.582201 11942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308: No bootstrap required, opened a new log
I20260709 13:04:41.584429 11942 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e70ea34673d4753b38665330e3cd308" member_type: VOTER }
I20260709 13:04:41.584805 11942 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:41.585110 11942 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e70ea34673d4753b38665330e3cd308, State: Initialized, Role: FOLLOWER
I20260709 13:04:41.585799 11942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e70ea34673d4753b38665330e3cd308" member_type: VOTER }
I20260709 13:04:41.586269 11942 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:41.586506 11942 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:41.586748 11942 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:41.591763 11942 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e70ea34673d4753b38665330e3cd308" member_type: VOTER }
I20260709 13:04:41.592326 11942 leader_election.cc:304] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e70ea34673d4753b38665330e3cd308; no voters: 
I20260709 13:04:41.593572 11942 leader_election.cc:290] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:41.593997 11945 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:41.595394 11945 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [term 1 LEADER]: Becoming Leader. State: Replica: 0e70ea34673d4753b38665330e3cd308, State: Running, Role: LEADER
I20260709 13:04:41.596122 11945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e70ea34673d4753b38665330e3cd308" member_type: VOTER }
I20260709 13:04:41.596611 11942 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:41.601655 11947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e70ea34673d4753b38665330e3cd308. Latest consensus state: current_term: 1 leader_uuid: "0e70ea34673d4753b38665330e3cd308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e70ea34673d4753b38665330e3cd308" member_type: VOTER } }
I20260709 13:04:41.602545 11947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:41.603469 11946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e70ea34673d4753b38665330e3cd308" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e70ea34673d4753b38665330e3cd308" member_type: VOTER } }
I20260709 13:04:41.604084 11946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:41.605611 11953 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:41.610633 11953 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:41.614100   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:41.619596 11953 catalog_manager.cc:1383] Generated new cluster ID: 558d10b10d074aaa910dd508379d360f
I20260709 13:04:41.619886 11953 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:41.639600 11953 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:41.640934 11953 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:41.658100 11953 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308: Generated new TSK 0
I20260709 13:04:41.658739 11953 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:41.681149   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:41.688218 11963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:41.688687 11964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:41.690532 11966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:41.691141   431 server_base.cc:1061] running on GCE node
I20260709 13:04:41.692080   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:41.692289   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:41.692484   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602281692465 us; error 0 us; skew 500 ppm
I20260709 13:04:41.693054   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:41.695477   431 webserver.cc:533] Webserver started at http://127.0.107.193:40587/ using document root <none> and password file <none>
I20260709 13:04:41.696002   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:41.696209   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:41.696509   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:41.697767   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-0-root/instance:
uuid: "4a888e812fbc4f778328c798d0a54fd8"
format_stamp: "Formatted at 2026-07-09 13:04:41 on dist-test-slave-jk05"
I20260709 13:04:41.702622   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:04:41.706188 11971 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:41.707135   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.002s
I20260709 13:04:41.707412   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-0-root
uuid: "4a888e812fbc4f778328c798d0a54fd8"
format_stamp: "Formatted at 2026-07-09 13:04:41 on dist-test-slave-jk05"
I20260709 13:04:41.707655   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:41.718137   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:41.719362   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:41.720810   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:41.723318   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:41.723507   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:41.723824   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:41.724004   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:41.765473   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:37693
I20260709 13:04:41.765694 12033 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:37693 every 8 connection(s)
I20260709 13:04:41.770859   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:41.779023 12038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:41.779290 12039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:41.788932 12034 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34079
I20260709 13:04:41.788950   431 server_base.cc:1061] running on GCE node
I20260709 13:04:41.789881 12034 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:41.790982 12034 heartbeater.cc:507] Master 127.0.107.254:34079 requested a full tablet report, sending...
W20260709 13:04:41.791883 12041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:41.793077   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:41.793357   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:41.793609   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602281793587 us; error 0 us; skew 500 ppm
I20260709 13:04:41.793605 11907 ts_manager.cc:194] Registered new tserver with Master: 4a888e812fbc4f778328c798d0a54fd8 (127.0.107.193:37693)
I20260709 13:04:41.794742   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:41.796480 11907 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46472
I20260709 13:04:41.797843   431 webserver.cc:533] Webserver started at http://127.0.107.194:46723/ using document root <none> and password file <none>
I20260709 13:04:41.798597   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:41.798828   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:41.799103   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:41.800308   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-1-root/instance:
uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d"
format_stamp: "Formatted at 2026-07-09 13:04:41 on dist-test-slave-jk05"
I20260709 13:04:41.805019   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 13:04:41.808384 12046 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:41.809284   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:04:41.809615   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-1-root
uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d"
format_stamp: "Formatted at 2026-07-09 13:04:41 on dist-test-slave-jk05"
I20260709 13:04:41.809861   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:41.837118   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:41.838347   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:41.839792   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:41.842154   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:41.842340   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:41.842648   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:41.842787   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:41.883797   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:36007
I20260709 13:04:41.883912 12108 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:36007 every 8 connection(s)
I20260709 13:04:41.888682   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:41.895640 12112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:41.898169 12113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:41.898736   431 server_base.cc:1061] running on GCE node
W20260709 13:04:41.900286 12116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:41.901660   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:41.901991   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:41.901985 12109 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34079
I20260709 13:04:41.902215   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602281902199 us; error 0 us; skew 500 ppm
I20260709 13:04:41.902601 12109 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:41.903061   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:41.903441 12109 heartbeater.cc:507] Master 127.0.107.254:34079 requested a full tablet report, sending...
I20260709 13:04:41.906070 11907 ts_manager.cc:194] Registered new tserver with Master: 41ea81cd9c1e42ae96cbf2ef7bf0288d (127.0.107.194:36007)
I20260709 13:04:41.906212   431 webserver.cc:533] Webserver started at http://127.0.107.195:37095/ using document root <none> and password file <none>
I20260709 13:04:41.906939   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:41.907191   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:41.907534   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:41.908090 11907 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46478
I20260709 13:04:41.909212   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-2-root/instance:
uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae"
format_stamp: "Formatted at 2026-07-09 13:04:41 on dist-test-slave-jk05"
I20260709 13:04:41.914494   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:04:41.917779 12120 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:41.918627   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:04:41.918957   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-2-root
uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae"
format_stamp: "Formatted at 2026-07-09 13:04:41 on dist-test-slave-jk05"
I20260709 13:04:41.919265   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783601935286233-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:41.932233   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:41.933562   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:41.935029   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:41.937223   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:41.937484   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:04:41.937732   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:41.937906   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:41.979331   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:37791
I20260709 13:04:41.979424 12182 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:37791 every 8 connection(s)
I20260709 13:04:41.993593 12183 heartbeater.cc:344] Connected to a master server at 127.0.107.254:34079
I20260709 13:04:41.994028 12183 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:41.994783 12183 heartbeater.cc:507] Master 127.0.107.254:34079 requested a full tablet report, sending...
I20260709 13:04:41.996778 11907 ts_manager.cc:194] Registered new tserver with Master: fbffff52f38b445cb8e5f6d4b26ac6ae (127.0.107.195:37791)
I20260709 13:04:41.997538   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014755449s
I20260709 13:04:41.998301 11907 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46484
I20260709 13:04:42.023555 11907 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46490:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:42.026114 11907 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:42.077628 12148 tablet_service.cc:1511] Processing CreateTablet for tablet 39dc0e28fb114cfaa18fe28fdbcea8b9 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=edbb78e43d024e7ab880450e2d6ab734]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:04:42.077965 12147 tablet_service.cc:1511] Processing CreateTablet for tablet d5500d216f6641c99012087e68d6826e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=edbb78e43d024e7ab880450e2d6ab734]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:04:42.079280 12148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39dc0e28fb114cfaa18fe28fdbcea8b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:42.079907 12147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5500d216f6641c99012087e68d6826e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:42.082062 12074 tablet_service.cc:1511] Processing CreateTablet for tablet 39dc0e28fb114cfaa18fe28fdbcea8b9 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=edbb78e43d024e7ab880450e2d6ab734]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:04:42.083693 12074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39dc0e28fb114cfaa18fe28fdbcea8b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:42.085886 12073 tablet_service.cc:1511] Processing CreateTablet for tablet d5500d216f6641c99012087e68d6826e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=edbb78e43d024e7ab880450e2d6ab734]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:04:42.087399 12073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5500d216f6641c99012087e68d6826e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:42.100972 11999 tablet_service.cc:1511] Processing CreateTablet for tablet 39dc0e28fb114cfaa18fe28fdbcea8b9 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=edbb78e43d024e7ab880450e2d6ab734]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:04:42.102480 11999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39dc0e28fb114cfaa18fe28fdbcea8b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:42.107300 12198 tablet_bootstrap.cc:492] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae: Bootstrap starting.
I20260709 13:04:42.109670 11998 tablet_service.cc:1511] Processing CreateTablet for tablet d5500d216f6641c99012087e68d6826e (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=edbb78e43d024e7ab880450e2d6ab734]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:04:42.110602 12199 tablet_bootstrap.cc:492] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Bootstrap starting.
I20260709 13:04:42.111117 11998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5500d216f6641c99012087e68d6826e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:42.116677 12198 tablet_bootstrap.cc:654] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.118335 12199 tablet_bootstrap.cc:654] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.129403 12198 tablet_bootstrap.cc:492] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae: No bootstrap required, opened a new log
I20260709 13:04:42.130152 12198 ts_tablet_manager.cc:1403] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 13:04:42.134727 12198 raft_consensus.cc:359] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } }
I20260709 13:04:42.135807 12198 raft_consensus.cc:385] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.136274 12198 raft_consensus.cc:740] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbffff52f38b445cb8e5f6d4b26ac6ae, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.137122 12198 consensus_queue.cc:260] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } }
I20260709 13:04:42.138104 12202 tablet_bootstrap.cc:492] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8: Bootstrap starting.
I20260709 13:04:42.139463 12199 tablet_bootstrap.cc:492] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: No bootstrap required, opened a new log
I20260709 13:04:42.139834 12183 heartbeater.cc:499] Master 127.0.107.254:34079 was elected leader, sending a full tablet report...
I20260709 13:04:42.140038 12199 ts_tablet_manager.cc:1403] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.008s
I20260709 13:04:42.140340 12198 ts_tablet_manager.cc:1434] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:04:42.142156 12198 tablet_bootstrap.cc:492] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae: Bootstrap starting.
I20260709 13:04:42.143092 12199 raft_consensus.cc:359] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } }
I20260709 13:04:42.143939 12199 raft_consensus.cc:385] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.144235 12199 raft_consensus.cc:740] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ea81cd9c1e42ae96cbf2ef7bf0288d, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.145105 12199 consensus_queue.cc:260] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } }
I20260709 13:04:42.146287 12202 tablet_bootstrap.cc:654] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.151126 12198 tablet_bootstrap.cc:654] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.161752 12109 heartbeater.cc:499] Master 127.0.107.254:34079 was elected leader, sending a full tablet report...
I20260709 13:04:42.162235 12199 ts_tablet_manager.cc:1434] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.013s
I20260709 13:04:42.163421 12199 tablet_bootstrap.cc:492] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Bootstrap starting.
I20260709 13:04:42.164067 12202 tablet_bootstrap.cc:492] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8: No bootstrap required, opened a new log
I20260709 13:04:42.165597 12202 ts_tablet_manager.cc:1403] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.010s
I20260709 13:04:42.167742 12198 tablet_bootstrap.cc:492] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae: No bootstrap required, opened a new log
I20260709 13:04:42.168358 12198 ts_tablet_manager.cc:1403] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.005s
I20260709 13:04:42.169066 12202 raft_consensus.cc:359] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } }
I20260709 13:04:42.169968 12202 raft_consensus.cc:385] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.170327 12202 raft_consensus.cc:740] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a888e812fbc4f778328c798d0a54fd8, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.170642 12199 tablet_bootstrap.cc:654] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.171221 12202 consensus_queue.cc:260] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } }
I20260709 13:04:42.171747 12198 raft_consensus.cc:359] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.173812 12198 raft_consensus.cc:385] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.174247 12198 raft_consensus.cc:740] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbffff52f38b445cb8e5f6d4b26ac6ae, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.175485 12198 consensus_queue.cc:260] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.178999 12198 ts_tablet_manager.cc:1434] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.005s
I20260709 13:04:42.187005 12202 ts_tablet_manager.cc:1434] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8: Time spent starting tablet: real 0.021s	user 0.015s	sys 0.005s
I20260709 13:04:42.187695 12034 heartbeater.cc:499] Master 127.0.107.254:34079 was elected leader, sending a full tablet report...
I20260709 13:04:42.188364 12202 tablet_bootstrap.cc:492] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8: Bootstrap starting.
I20260709 13:04:42.189055 12199 tablet_bootstrap.cc:492] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d: No bootstrap required, opened a new log
I20260709 13:04:42.189670 12199 ts_tablet_manager.cc:1403] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.011s
I20260709 13:04:42.192176 12199 raft_consensus.cc:359] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.192804 12199 raft_consensus.cc:385] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.193073 12199 raft_consensus.cc:740] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ea81cd9c1e42ae96cbf2ef7bf0288d, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.193832 12199 consensus_queue.cc:260] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.195122 12202 tablet_bootstrap.cc:654] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.195847 12199 ts_tablet_manager.cc:1434] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:42.200819 12202 tablet_bootstrap.cc:492] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8: No bootstrap required, opened a new log
I20260709 13:04:42.201362 12202 ts_tablet_manager.cc:1403] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:04:42.204092 12202 raft_consensus.cc:359] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.204640 12202 raft_consensus.cc:385] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.204869 12202 raft_consensus.cc:740] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a888e812fbc4f778328c798d0a54fd8, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.205648 12202 consensus_queue.cc:260] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.207665 12202 ts_tablet_manager.cc:1434] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:42.448060 12204 raft_consensus.cc:493] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.448673 12204 raft_consensus.cc:515] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } }
I20260709 13:04:42.450881 12204 leader_election.cc:290] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbffff52f38b445cb8e5f6d4b26ac6ae (127.0.107.195:37791), 4a888e812fbc4f778328c798d0a54fd8 (127.0.107.193:37693)
I20260709 13:04:42.461624 12158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dc0e28fb114cfaa18fe28fdbcea8b9" candidate_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" is_pre_election: true
I20260709 13:04:42.461836 12009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dc0e28fb114cfaa18fe28fdbcea8b9" candidate_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a888e812fbc4f778328c798d0a54fd8" is_pre_election: true
I20260709 13:04:42.462394 12158 raft_consensus.cc:2468] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41ea81cd9c1e42ae96cbf2ef7bf0288d in term 0.
I20260709 13:04:42.462440 12009 raft_consensus.cc:2468] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41ea81cd9c1e42ae96cbf2ef7bf0288d in term 0.
I20260709 13:04:42.463716 12047 leader_election.cc:304] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ea81cd9c1e42ae96cbf2ef7bf0288d, fbffff52f38b445cb8e5f6d4b26ac6ae; no voters: 
I20260709 13:04:42.464461 12204 raft_consensus.cc:2804] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:42.464857 12204 raft_consensus.cc:493] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.465137 12204 raft_consensus.cc:3060] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.470649 12204 raft_consensus.cc:515] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } }
I20260709 13:04:42.472477 12204 leader_election.cc:290] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [CANDIDATE]: Term 1 election: Requested vote from peers fbffff52f38b445cb8e5f6d4b26ac6ae (127.0.107.195:37791), 4a888e812fbc4f778328c798d0a54fd8 (127.0.107.193:37693)
I20260709 13:04:42.473237 12158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dc0e28fb114cfaa18fe28fdbcea8b9" candidate_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae"
I20260709 13:04:42.473574 12009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dc0e28fb114cfaa18fe28fdbcea8b9" candidate_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a888e812fbc4f778328c798d0a54fd8"
I20260709 13:04:42.473953 12158 raft_consensus.cc:3060] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.474174 12009 raft_consensus.cc:3060] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.481781 12158 raft_consensus.cc:2468] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ea81cd9c1e42ae96cbf2ef7bf0288d in term 1.
I20260709 13:04:42.481781 12009 raft_consensus.cc:2468] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ea81cd9c1e42ae96cbf2ef7bf0288d in term 1.
I20260709 13:04:42.483301 12047 leader_election.cc:304] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ea81cd9c1e42ae96cbf2ef7bf0288d, fbffff52f38b445cb8e5f6d4b26ac6ae; no voters: 
I20260709 13:04:42.484017 12204 raft_consensus.cc:2804] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:42.485069 12204 raft_consensus.cc:697] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 LEADER]: Becoming Leader. State: Replica: 41ea81cd9c1e42ae96cbf2ef7bf0288d, State: Running, Role: LEADER
I20260709 13:04:42.485879 12204 consensus_queue.cc:237] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } }
I20260709 13:04:42.492720 12204 raft_consensus.cc:493] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.493191 12204 raft_consensus.cc:515] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.494906 12204 leader_election.cc:290] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a888e812fbc4f778328c798d0a54fd8 (127.0.107.193:37693), fbffff52f38b445cb8e5f6d4b26ac6ae (127.0.107.195:37791)
I20260709 13:04:42.494473 11907 catalog_manager.cc:5697] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d reported cstate change: term changed from 0 to 1, leader changed from <none> to 41ea81cd9c1e42ae96cbf2ef7bf0288d (127.0.107.194). New cstate: current_term: 1 leader_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:42.495828 12009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5500d216f6641c99012087e68d6826e" candidate_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a888e812fbc4f778328c798d0a54fd8" is_pre_election: true
I20260709 13:04:42.496064 12158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5500d216f6641c99012087e68d6826e" candidate_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" is_pre_election: true
I20260709 13:04:42.496459 12009 raft_consensus.cc:2468] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41ea81cd9c1e42ae96cbf2ef7bf0288d in term 0.
I20260709 13:04:42.496802 12158 raft_consensus.cc:2468] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41ea81cd9c1e42ae96cbf2ef7bf0288d in term 0.
I20260709 13:04:42.497697 12050 leader_election.cc:304] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ea81cd9c1e42ae96cbf2ef7bf0288d, 4a888e812fbc4f778328c798d0a54fd8; no voters: 
I20260709 13:04:42.498610 12204 raft_consensus.cc:2804] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:42.498950 12204 raft_consensus.cc:493] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.499230 12204 raft_consensus.cc:3060] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.505599 12204 raft_consensus.cc:515] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.507215 12204 leader_election.cc:290] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [CANDIDATE]: Term 1 election: Requested vote from peers 4a888e812fbc4f778328c798d0a54fd8 (127.0.107.193:37693), fbffff52f38b445cb8e5f6d4b26ac6ae (127.0.107.195:37791)
I20260709 13:04:42.508006 12009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5500d216f6641c99012087e68d6826e" candidate_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a888e812fbc4f778328c798d0a54fd8"
I20260709 13:04:42.508210 12158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5500d216f6641c99012087e68d6826e" candidate_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae"
I20260709 13:04:42.508605 12009 raft_consensus.cc:3060] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.508893 12158 raft_consensus.cc:3060] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.514050 12009 raft_consensus.cc:2468] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ea81cd9c1e42ae96cbf2ef7bf0288d in term 1.
I20260709 13:04:42.514163 12158 raft_consensus.cc:2468] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ea81cd9c1e42ae96cbf2ef7bf0288d in term 1.
I20260709 13:04:42.515281 12047 leader_election.cc:304] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ea81cd9c1e42ae96cbf2ef7bf0288d, fbffff52f38b445cb8e5f6d4b26ac6ae; no voters: 
I20260709 13:04:42.516269 12204 raft_consensus.cc:2804] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:42.516701 12204 raft_consensus.cc:697] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 LEADER]: Becoming Leader. State: Replica: 41ea81cd9c1e42ae96cbf2ef7bf0288d, State: Running, Role: LEADER
I20260709 13:04:42.517349 12204 consensus_queue.cc:237] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.525635 11907 catalog_manager.cc:5697] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d reported cstate change: term changed from 0 to 1, leader changed from <none> to 41ea81cd9c1e42ae96cbf2ef7bf0288d (127.0.107.194). New cstate: current_term: 1 leader_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:42.568636 11907 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46492:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:42.570595 11907 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:42.594946 11999 tablet_service.cc:1511] Processing CreateTablet for tablet bf5d4391c27648ca84e5b3bf064a3c86 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=1ac1c1663691413e8c19df678a860e7e]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:04:42.596347 11999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5d4391c27648ca84e5b3bf064a3c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:42.596084 12073 tablet_service.cc:1511] Processing CreateTablet for tablet bf5d4391c27648ca84e5b3bf064a3c86 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=1ac1c1663691413e8c19df678a860e7e]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:04:42.597491 12073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5d4391c27648ca84e5b3bf064a3c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:42.597584 12147 tablet_service.cc:1511] Processing CreateTablet for tablet bf5d4391c27648ca84e5b3bf064a3c86 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=1ac1c1663691413e8c19df678a860e7e]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:04:42.598874 12147 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf5d4391c27648ca84e5b3bf064a3c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:42.600136 11998 tablet_service.cc:1511] Processing CreateTablet for tablet ca0b6466af8a4c24843aeced8f502cb8 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=1ac1c1663691413e8c19df678a860e7e]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:04:42.601521 11998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca0b6466af8a4c24843aeced8f502cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:42.603144 12148 tablet_service.cc:1511] Processing CreateTablet for tablet ca0b6466af8a4c24843aeced8f502cb8 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=1ac1c1663691413e8c19df678a860e7e]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:04:42.604586 12148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca0b6466af8a4c24843aeced8f502cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:42.604278 12074 tablet_service.cc:1511] Processing CreateTablet for tablet ca0b6466af8a4c24843aeced8f502cb8 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=1ac1c1663691413e8c19df678a860e7e]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:04:42.605751 12074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ca0b6466af8a4c24843aeced8f502cb8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:04:42.608580 11955 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:42.622205 12199 tablet_bootstrap.cc:492] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Bootstrap starting.
I20260709 13:04:42.624007 12198 tablet_bootstrap.cc:492] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae: Bootstrap starting.
I20260709 13:04:42.625959 12202 tablet_bootstrap.cc:492] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8: Bootstrap starting.
I20260709 13:04:42.628512 12199 tablet_bootstrap.cc:654] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.632405 12202 tablet_bootstrap.cc:654] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.634148 12198 tablet_bootstrap.cc:654] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.644773 12199 tablet_bootstrap.cc:492] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: No bootstrap required, opened a new log
I20260709 13:04:42.645327 12199 ts_tablet_manager.cc:1403] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.009s
I20260709 13:04:42.648414 12199 raft_consensus.cc:359] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.649299 12199 raft_consensus.cc:385] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.650117 12202 tablet_bootstrap.cc:492] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8: No bootstrap required, opened a new log
I20260709 13:04:42.650719 12202 ts_tablet_manager.cc:1403] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.005s
I20260709 13:04:42.651854 12198 tablet_bootstrap.cc:492] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae: No bootstrap required, opened a new log
I20260709 13:04:42.649616 12199 raft_consensus.cc:740] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ea81cd9c1e42ae96cbf2ef7bf0288d, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.652613 12198 ts_tablet_manager.cc:1403] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.005s
I20260709 13:04:42.653167 12199 consensus_queue.cc:260] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.653545 12202 raft_consensus.cc:359] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.654192 12202 raft_consensus.cc:385] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.654513 12202 raft_consensus.cc:740] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a888e812fbc4f778328c798d0a54fd8, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.655256 12202 consensus_queue.cc:260] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.655632 12199 ts_tablet_manager.cc:1434] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:04:42.657356 12202 ts_tablet_manager.cc:1434] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:04:42.657797 12198 raft_consensus.cc:359] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.658803 12198 raft_consensus.cc:385] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.659137 12198 raft_consensus.cc:740] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbffff52f38b445cb8e5f6d4b26ac6ae, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.659220 12199 tablet_bootstrap.cc:492] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Bootstrap starting.
I20260709 13:04:42.658468 12202 tablet_bootstrap.cc:492] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8: Bootstrap starting.
I20260709 13:04:42.661800 12198 consensus_queue.cc:260] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.664796 12198 ts_tablet_manager.cc:1434] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:04:42.666040 12198 tablet_bootstrap.cc:492] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae: Bootstrap starting.
I20260709 13:04:42.667457 12199 tablet_bootstrap.cc:654] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.673628 12198 tablet_bootstrap.cc:654] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.682251 12202 tablet_bootstrap.cc:654] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:42.683166 12199 tablet_bootstrap.cc:492] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: No bootstrap required, opened a new log
I20260709 13:04:42.683650 12199 ts_tablet_manager.cc:1403] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.013s
I20260709 13:04:42.683780 12198 tablet_bootstrap.cc:492] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae: No bootstrap required, opened a new log
I20260709 13:04:42.684361 12198 ts_tablet_manager.cc:1403] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.007s
I20260709 13:04:42.686479 12199 raft_consensus.cc:359] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.687111 12199 raft_consensus.cc:385] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.687361 12199 raft_consensus.cc:740] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ea81cd9c1e42ae96cbf2ef7bf0288d, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.688207 12198 raft_consensus.cc:359] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.687868 12202 tablet_bootstrap.cc:492] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8: No bootstrap required, opened a new log
I20260709 13:04:42.689146 12198 raft_consensus.cc:385] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.688748 12199 consensus_queue.cc:260] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.689469 12202 ts_tablet_manager.cc:1403] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.004s
I20260709 13:04:42.689652 12198 raft_consensus.cc:740] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbffff52f38b445cb8e5f6d4b26ac6ae, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.690680 12198 consensus_queue.cc:260] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.691627 12199 ts_tablet_manager.cc:1434] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:04:42.692866 12198 ts_tablet_manager.cc:1434] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:04:42.694965 12202 raft_consensus.cc:359] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.695919 12202 raft_consensus.cc:385] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:42.696269 12202 raft_consensus.cc:740] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a888e812fbc4f778328c798d0a54fd8, State: Initialized, Role: FOLLOWER
I20260709 13:04:42.697091 12202 consensus_queue.cc:260] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.700630 12202 ts_tablet_manager.cc:1434] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:04:42.918288 12207 raft_consensus.cc:493] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.918928 12207 raft_consensus.cc:515] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.921016 12207 leader_election.cc:290] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41ea81cd9c1e42ae96cbf2ef7bf0288d (127.0.107.194:36007), fbffff52f38b445cb8e5f6d4b26ac6ae (127.0.107.195:37791)
I20260709 13:04:42.933936 12158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5d4391c27648ca84e5b3bf064a3c86" candidate_uuid: "4a888e812fbc4f778328c798d0a54fd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" is_pre_election: true
I20260709 13:04:42.934361 12084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5d4391c27648ca84e5b3bf064a3c86" candidate_uuid: "4a888e812fbc4f778328c798d0a54fd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" is_pre_election: true
I20260709 13:04:42.934805 12158 raft_consensus.cc:2468] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a888e812fbc4f778328c798d0a54fd8 in term 0.
I20260709 13:04:42.935029 12084 raft_consensus.cc:2468] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a888e812fbc4f778328c798d0a54fd8 in term 0.
I20260709 13:04:42.936125 11972 leader_election.cc:304] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a888e812fbc4f778328c798d0a54fd8, fbffff52f38b445cb8e5f6d4b26ac6ae; no voters: 
I20260709 13:04:42.936942 12207 raft_consensus.cc:2804] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:42.937242 12207 raft_consensus.cc:493] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.937541 12207 raft_consensus.cc:3060] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.943104 12207 raft_consensus.cc:515] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.944860 12207 leader_election.cc:290] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [CANDIDATE]: Term 1 election: Requested vote from peers 41ea81cd9c1e42ae96cbf2ef7bf0288d (127.0.107.194:36007), fbffff52f38b445cb8e5f6d4b26ac6ae (127.0.107.195:37791)
I20260709 13:04:42.945727 12084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5d4391c27648ca84e5b3bf064a3c86" candidate_uuid: "4a888e812fbc4f778328c798d0a54fd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d"
I20260709 13:04:42.945909 12158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5d4391c27648ca84e5b3bf064a3c86" candidate_uuid: "4a888e812fbc4f778328c798d0a54fd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae"
I20260709 13:04:42.946286 12084 raft_consensus.cc:3060] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.946504 12158 raft_consensus.cc:3060] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.952112 12084 raft_consensus.cc:2468] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a888e812fbc4f778328c798d0a54fd8 in term 1.
I20260709 13:04:42.952112 12158 raft_consensus.cc:2468] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a888e812fbc4f778328c798d0a54fd8 in term 1.
I20260709 13:04:42.953354 11973 leader_election.cc:304] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ea81cd9c1e42ae96cbf2ef7bf0288d, 4a888e812fbc4f778328c798d0a54fd8; no voters: 
I20260709 13:04:42.954123 12207 raft_consensus.cc:2804] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:42.955243 12207 raft_consensus.cc:697] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 LEADER]: Becoming Leader. State: Replica: 4a888e812fbc4f778328c798d0a54fd8, State: Running, Role: LEADER
I20260709 13:04:42.956032 12207 consensus_queue.cc:237] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.963454 11905 catalog_manager.cc:5697] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a888e812fbc4f778328c798d0a54fd8 (127.0.107.193). New cstate: current_term: 1 leader_uuid: "4a888e812fbc4f778328c798d0a54fd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:42.971675 12204 raft_consensus.cc:493] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:42.972226 12204 raft_consensus.cc:515] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.973932 12204 leader_election.cc:290] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a888e812fbc4f778328c798d0a54fd8 (127.0.107.193:37693), fbffff52f38b445cb8e5f6d4b26ac6ae (127.0.107.195:37791)
I20260709 13:04:42.974720 12009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b6466af8a4c24843aeced8f502cb8" candidate_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a888e812fbc4f778328c798d0a54fd8" is_pre_election: true
I20260709 13:04:42.974879 12158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b6466af8a4c24843aeced8f502cb8" candidate_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" is_pre_election: true
I20260709 13:04:42.975385 12009 raft_consensus.cc:2468] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41ea81cd9c1e42ae96cbf2ef7bf0288d in term 0.
I20260709 13:04:42.975523 12158 raft_consensus.cc:2468] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41ea81cd9c1e42ae96cbf2ef7bf0288d in term 0.
I20260709 13:04:42.976425 12050 leader_election.cc:304] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ea81cd9c1e42ae96cbf2ef7bf0288d, 4a888e812fbc4f778328c798d0a54fd8; no voters: 
I20260709 13:04:42.977185 12204 raft_consensus.cc:2804] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:42.977531 12204 raft_consensus.cc:493] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:42.977818 12204 raft_consensus.cc:3060] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.982861 12204 raft_consensus.cc:515] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:42.984494 12204 leader_election.cc:290] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [CANDIDATE]: Term 1 election: Requested vote from peers 4a888e812fbc4f778328c798d0a54fd8 (127.0.107.193:37693), fbffff52f38b445cb8e5f6d4b26ac6ae (127.0.107.195:37791)
I20260709 13:04:42.985673 12158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b6466af8a4c24843aeced8f502cb8" candidate_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae"
I20260709 13:04:42.985913 12009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b6466af8a4c24843aeced8f502cb8" candidate_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a888e812fbc4f778328c798d0a54fd8"
I20260709 13:04:42.986469 12158 raft_consensus.cc:3060] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.986632 12009 raft_consensus.cc:3060] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:42.992063 12009 raft_consensus.cc:2468] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ea81cd9c1e42ae96cbf2ef7bf0288d in term 1.
I20260709 13:04:42.992270 12158 raft_consensus.cc:2468] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41ea81cd9c1e42ae96cbf2ef7bf0288d in term 1.
I20260709 13:04:42.993317 12050 leader_election.cc:304] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ea81cd9c1e42ae96cbf2ef7bf0288d, 4a888e812fbc4f778328c798d0a54fd8; no voters: 
I20260709 13:04:42.994114 12204 raft_consensus.cc:2804] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:42.994573 12204 raft_consensus.cc:697] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 LEADER]: Becoming Leader. State: Replica: 41ea81cd9c1e42ae96cbf2ef7bf0288d, State: Running, Role: LEADER
I20260709 13:04:42.995361 12204 consensus_queue.cc:237] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:43.002221 11905 catalog_manager.cc:5697] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d reported cstate change: term changed from 0 to 1, leader changed from <none> to 41ea81cd9c1e42ae96cbf2ef7bf0288d (127.0.107.194). New cstate: current_term: 1 leader_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:43.005704 12204 consensus_queue.cc:1048] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:43.022011 12204 consensus_queue.cc:1048] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:43.035421 12084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39dc0e28fb114cfaa18fe28fdbcea8b9"
dest_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d"
mode: GRACEFUL
new_leader_uuid: "4a888e812fbc4f778328c798d0a54fd8"
 from {username='slave'} at 127.0.0.1:52894
I20260709 13:04:43.036019 12084 raft_consensus.cc:606] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 LEADER]: Received request to transfer leadership to TS 4a888e812fbc4f778328c798d0a54fd8
I20260709 13:04:43.048168 12084 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ca0b6466af8a4c24843aeced8f502cb8"
dest_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d"
mode: GRACEFUL
new_leader_uuid: "4a888e812fbc4f778328c798d0a54fd8"
 from {username='slave'} at 127.0.0.1:52894
I20260709 13:04:43.048739 12084 raft_consensus.cc:606] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 LEADER]: Received request to transfer leadership to TS 4a888e812fbc4f778328c798d0a54fd8
I20260709 13:04:43.077277 12204 consensus_queue.cc:1048] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:43.089515 12240 consensus_queue.cc:1048] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:43.418517 12207 consensus_queue.cc:1048] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:43.429230 12207 consensus_queue.cc:1048] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:43.478502 12240 consensus_queue.cc:1048] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:43.491977 12246 consensus_queue.cc:1048] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:43.502113 12204 raft_consensus.cc:993] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: : Instructing follower 4a888e812fbc4f778328c798d0a54fd8 to start an election
I20260709 13:04:43.502631 12204 raft_consensus.cc:1081] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 LEADER]: Signalling peer 4a888e812fbc4f778328c798d0a54fd8 to start an election
I20260709 13:04:43.504302 12009 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ca0b6466af8a4c24843aeced8f502cb8"
dest_uuid: "4a888e812fbc4f778328c798d0a54fd8"
 from {username='slave'} at 127.0.0.1:52626
I20260709 13:04:43.504877 12009 raft_consensus.cc:493] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:43.505206 12009 raft_consensus.cc:3060] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.511632 12009 raft_consensus.cc:515] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:43.513597 12009 leader_election.cc:290] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [CANDIDATE]: Term 2 election: Requested vote from peers 41ea81cd9c1e42ae96cbf2ef7bf0288d (127.0.107.194:36007), fbffff52f38b445cb8e5f6d4b26ac6ae (127.0.107.195:37791)
I20260709 13:04:43.514497 12084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b6466af8a4c24843aeced8f502cb8" candidate_uuid: "4a888e812fbc4f778328c798d0a54fd8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d"
I20260709 13:04:43.514663 12158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ca0b6466af8a4c24843aeced8f502cb8" candidate_uuid: "4a888e812fbc4f778328c798d0a54fd8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae"
I20260709 13:04:43.515190 12084 raft_consensus.cc:3055] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:43.515354 12158 raft_consensus.cc:3060] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.515616 12084 raft_consensus.cc:740] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41ea81cd9c1e42ae96cbf2ef7bf0288d, State: Running, Role: LEADER
I20260709 13:04:43.516571 12084 consensus_queue.cc:260] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:43.517691 12084 raft_consensus.cc:3060] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.522231 12158 raft_consensus.cc:2468] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a888e812fbc4f778328c798d0a54fd8 in term 2.
I20260709 13:04:43.523509 11972 leader_election.cc:304] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a888e812fbc4f778328c798d0a54fd8, fbffff52f38b445cb8e5f6d4b26ac6ae; no voters: 
I20260709 13:04:43.523576 12084 raft_consensus.cc:2468] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a888e812fbc4f778328c798d0a54fd8 in term 2.
I20260709 13:04:43.524477 12207 raft_consensus.cc:2804] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:43.524951 12207 raft_consensus.cc:697] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [term 2 LEADER]: Becoming Leader. State: Replica: 4a888e812fbc4f778328c798d0a54fd8, State: Running, Role: LEADER
I20260709 13:04:43.525719 12207 consensus_queue.cc:237] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:43.531793 11907 catalog_manager.cc:5697] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 reported cstate change: term changed from 1 to 2, leader changed from 41ea81cd9c1e42ae96cbf2ef7bf0288d (127.0.107.194) to 4a888e812fbc4f778328c798d0a54fd8 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "4a888e812fbc4f778328c798d0a54fd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:43.620756 12240 raft_consensus.cc:993] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: : Instructing follower 4a888e812fbc4f778328c798d0a54fd8 to start an election
I20260709 13:04:43.621284 12204 raft_consensus.cc:1081] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 LEADER]: Signalling peer 4a888e812fbc4f778328c798d0a54fd8 to start an election
I20260709 13:04:43.623235 12009 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39dc0e28fb114cfaa18fe28fdbcea8b9"
dest_uuid: "4a888e812fbc4f778328c798d0a54fd8"
 from {username='slave'} at 127.0.0.1:52626
I20260709 13:04:43.623768 12009 raft_consensus.cc:493] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:43.624138 12009 raft_consensus.cc:3060] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.629402 12009 raft_consensus.cc:515] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } }
I20260709 13:04:43.631973 12009 leader_election.cc:290] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [CANDIDATE]: Term 2 election: Requested vote from peers fbffff52f38b445cb8e5f6d4b26ac6ae (127.0.107.195:37791), 41ea81cd9c1e42ae96cbf2ef7bf0288d (127.0.107.194:36007)
I20260709 13:04:43.632833 12158 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dc0e28fb114cfaa18fe28fdbcea8b9" candidate_uuid: "4a888e812fbc4f778328c798d0a54fd8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae"
I20260709 13:04:43.633208 12084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39dc0e28fb114cfaa18fe28fdbcea8b9" candidate_uuid: "4a888e812fbc4f778328c798d0a54fd8" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d"
I20260709 13:04:43.633687 12158 raft_consensus.cc:3060] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.633841 12084 raft_consensus.cc:3055] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:43.634264 12084 raft_consensus.cc:740] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41ea81cd9c1e42ae96cbf2ef7bf0288d, State: Running, Role: LEADER
I20260709 13:04:43.635119 12084 consensus_queue.cc:260] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } }
I20260709 13:04:43.636413 12084 raft_consensus.cc:3060] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.641317 12158 raft_consensus.cc:2468] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a888e812fbc4f778328c798d0a54fd8 in term 2.
I20260709 13:04:43.642694 11972 leader_election.cc:304] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a888e812fbc4f778328c798d0a54fd8, fbffff52f38b445cb8e5f6d4b26ac6ae; no voters: 
I20260709 13:04:43.643551 12207 raft_consensus.cc:2804] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:43.643779 12084 raft_consensus.cc:2468] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a888e812fbc4f778328c798d0a54fd8 in term 2.
I20260709 13:04:43.644120 12207 raft_consensus.cc:697] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [term 2 LEADER]: Becoming Leader. State: Replica: 4a888e812fbc4f778328c798d0a54fd8, State: Running, Role: LEADER
I20260709 13:04:43.645020 12207 consensus_queue.cc:237] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } }
I20260709 13:04:43.651922 11907 catalog_manager.cc:5697] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 reported cstate change: term changed from 1 to 2, leader changed from 41ea81cd9c1e42ae96cbf2ef7bf0288d (127.0.107.194) to 4a888e812fbc4f778328c798d0a54fd8 (127.0.107.193). New cstate: current_term: 2 leader_uuid: "4a888e812fbc4f778328c798d0a54fd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:43.783800   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:04:43.785497   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:04:43.785923   431 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:04:43.798226 12009 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf5d4391c27648ca84e5b3bf064a3c86"
dest_uuid: "4a888e812fbc4f778328c798d0a54fd8"
mode: GRACEFUL
new_leader_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae"
 from {username='slave'} at 127.0.0.1:39118
I20260709 13:04:43.798756 12009 raft_consensus.cc:606] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 LEADER]: Received request to transfer leadership to TS fbffff52f38b445cb8e5f6d4b26ac6ae
I20260709 13:04:43.800233   431 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:04:43.961167 12243 raft_consensus.cc:993] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8: : Instructing follower fbffff52f38b445cb8e5f6d4b26ac6ae to start an election
I20260709 13:04:43.961704 12253 raft_consensus.cc:1081] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 LEADER]: Signalling peer fbffff52f38b445cb8e5f6d4b26ac6ae to start an election
I20260709 13:04:43.963485 12158 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf5d4391c27648ca84e5b3bf064a3c86"
dest_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae"
 from {username='slave'} at 127.0.0.1:43502
I20260709 13:04:43.964100 12158 raft_consensus.cc:493] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:43.964434 12158 raft_consensus.cc:3060] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.969552 12158 raft_consensus.cc:515] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:43.971719 12158 leader_election.cc:290] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [CANDIDATE]: Term 2 election: Requested vote from peers 4a888e812fbc4f778328c798d0a54fd8 (127.0.107.193:37693), 41ea81cd9c1e42ae96cbf2ef7bf0288d (127.0.107.194:36007)
I20260709 13:04:43.983493 12009 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5d4391c27648ca84e5b3bf064a3c86" candidate_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a888e812fbc4f778328c798d0a54fd8"
I20260709 13:04:43.984174 12009 raft_consensus.cc:3055] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:43.984558 12009 raft_consensus.cc:740] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4a888e812fbc4f778328c798d0a54fd8, State: Running, Role: LEADER
I20260709 13:04:43.985064 12084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf5d4391c27648ca84e5b3bf064a3c86" candidate_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d"
I20260709 13:04:43.985739 12084 raft_consensus.cc:3060] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.985443 12009 consensus_queue.cc:260] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:43.986829 12009 raft_consensus.cc:3060] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:43.988325 12083 raft_consensus.cc:1275] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 2 FOLLOWER]: Refusing update from remote peer 4a888e812fbc4f778328c798d0a54fd8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:43.989867 12253 consensus_queue.cc:1048] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:43.993510 12084 raft_consensus.cc:2468] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbffff52f38b445cb8e5f6d4b26ac6ae in term 2.
I20260709 13:04:43.994019 12009 raft_consensus.cc:2468] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbffff52f38b445cb8e5f6d4b26ac6ae in term 2.
I20260709 13:04:43.995165 12122 leader_election.cc:304] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41ea81cd9c1e42ae96cbf2ef7bf0288d, fbffff52f38b445cb8e5f6d4b26ac6ae; no voters: 
I20260709 13:04:43.997364 12257 raft_consensus.cc:2804] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:44.001816 12257 raft_consensus.cc:697] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 LEADER]: Becoming Leader. State: Replica: fbffff52f38b445cb8e5f6d4b26ac6ae, State: Running, Role: LEADER
I20260709 13:04:44.002914 12257 consensus_queue.cc:237] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } }
I20260709 13:04:44.004772 12158 raft_consensus.cc:1275] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 FOLLOWER]: Refusing update from remote peer 4a888e812fbc4f778328c798d0a54fd8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:44.007251 12253 consensus_queue.cc:1048] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:44.012315 11905 catalog_manager.cc:5697] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae reported cstate change: term changed from 1 to 2, leader changed from 4a888e812fbc4f778328c798d0a54fd8 (127.0.107.193) to fbffff52f38b445cb8e5f6d4b26ac6ae (127.0.107.195). New cstate: current_term: 2 leader_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a888e812fbc4f778328c798d0a54fd8" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 37693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:44.063983 12084 raft_consensus.cc:1275] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 2 FOLLOWER]: Refusing update from remote peer 4a888e812fbc4f778328c798d0a54fd8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:44.065364 12253 consensus_queue.cc:1048] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41ea81cd9c1e42ae96cbf2ef7bf0288d" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:44.074705 12158 raft_consensus.cc:1275] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 FOLLOWER]: Refusing update from remote peer 4a888e812fbc4f778328c798d0a54fd8: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:44.076586 12253 consensus_queue.cc:1048] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbffff52f38b445cb8e5f6d4b26ac6ae" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 37791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:44.164016   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:04:44.182677   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:44.183336   431 tablet_replica.cc:333] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8: stopping tablet replica
I20260709 13:04:44.183945   431 raft_consensus.cc:2243] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:44.184414   431 raft_consensus.cc:2272] T d5500d216f6641c99012087e68d6826e P 4a888e812fbc4f778328c798d0a54fd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.186396   431 tablet_replica.cc:333] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8: stopping tablet replica
I20260709 13:04:44.186947   431 raft_consensus.cc:2243] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:44.187747   431 raft_consensus.cc:2272] T ca0b6466af8a4c24843aeced8f502cb8 P 4a888e812fbc4f778328c798d0a54fd8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.190142   431 tablet_replica.cc:333] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8: stopping tablet replica
I20260709 13:04:44.190683   431 raft_consensus.cc:2243] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:44.191087   431 raft_consensus.cc:2272] T bf5d4391c27648ca84e5b3bf064a3c86 P 4a888e812fbc4f778328c798d0a54fd8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.193171   431 tablet_replica.cc:333] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8: stopping tablet replica
I20260709 13:04:44.193723   431 raft_consensus.cc:2243] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:44.194569   431 raft_consensus.cc:2272] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 4a888e812fbc4f778328c798d0a54fd8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.208807   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:04:44.221758   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
I20260709 13:04:44.240536   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:44.241200   431 tablet_replica.cc:333] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d: stopping tablet replica
I20260709 13:04:44.241804   431 raft_consensus.cc:2243] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:44.242604   431 raft_consensus.cc:2272] T d5500d216f6641c99012087e68d6826e P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.244540   431 tablet_replica.cc:333] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: stopping tablet replica
I20260709 13:04:44.245074   431 raft_consensus.cc:2243] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:44.245571   431 raft_consensus.cc:2272] T ca0b6466af8a4c24843aeced8f502cb8 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.247700   431 tablet_replica.cc:333] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: stopping tablet replica
I20260709 13:04:44.248624   431 raft_consensus.cc:2243] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:44.249133   431 raft_consensus.cc:2272] T bf5d4391c27648ca84e5b3bf064a3c86 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.251682   431 tablet_replica.cc:333] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d: stopping tablet replica
I20260709 13:04:44.252316   431 raft_consensus.cc:2243] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:44.252779   431 raft_consensus.cc:2272] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P 41ea81cd9c1e42ae96cbf2ef7bf0288d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.275906   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
I20260709 13:04:44.288930   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
I20260709 13:04:44.308528   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:44.309227   431 tablet_replica.cc:333] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae: stopping tablet replica
I20260709 13:04:44.309893   431 raft_consensus.cc:2243] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:44.310356   431 raft_consensus.cc:2272] T d5500d216f6641c99012087e68d6826e P fbffff52f38b445cb8e5f6d4b26ac6ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.312270   431 tablet_replica.cc:333] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae: stopping tablet replica
I20260709 13:04:44.312806   431 raft_consensus.cc:2243] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:44.313241   431 raft_consensus.cc:2272] T ca0b6466af8a4c24843aeced8f502cb8 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.315516   431 tablet_replica.cc:333] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae: stopping tablet replica
I20260709 13:04:44.316039   431 raft_consensus.cc:2243] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:44.316636   431 pending_rounds.cc:70] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae: Trying to abort 1 pending ops.
I20260709 13:04:44.316831   431 pending_rounds.cc:77] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305634955287154688 op_type: NO_OP noop_request { }
I20260709 13:04:44.317128   431 raft_consensus.cc:2889] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:44.317394   431 raft_consensus.cc:2272] T bf5d4391c27648ca84e5b3bf064a3c86 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.320206   431 tablet_replica.cc:333] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae: stopping tablet replica
I20260709 13:04:44.320705   431 raft_consensus.cc:2243] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:44.321127   431 raft_consensus.cc:2272] T 39dc0e28fb114cfaa18fe28fdbcea8b9 P fbffff52f38b445cb8e5f6d4b26ac6ae [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.334455   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:04:44.347244   431 master.cc:562] Master@127.0.107.254:34079 shutting down...
I20260709 13:04:44.363229   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:44.363765   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:44.364033   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0e70ea34673d4753b38665330e3cd308: stopping tablet replica
I20260709 13:04:44.382668   431 master.cc:584] Master@127.0.107.254:34079 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2943 ms)
[----------] 6 tests from LeaderRebalancerTest (348964 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:04:44.418355   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:35413
I20260709 13:04:44.419486   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:44.424729 12264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:44.425791 12263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:44.426314 12266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:44.427059   431 server_base.cc:1061] running on GCE node
I20260709 13:04:44.427928   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:44.428102   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:44.428215   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602284428204 us; error 0 us; skew 500 ppm
I20260709 13:04:44.428668   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:44.431175   431 webserver.cc:533] Webserver started at http://127.0.107.254:34713/ using document root <none> and password file <none>
I20260709 13:04:44.431623   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:44.431779   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:44.432000   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:44.433022   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/master-0-root/instance:
uuid: "da2bdb6bde834ca588d7e7a427254aa5"
format_stamp: "Formatted at 2026-07-09 13:04:44 on dist-test-slave-jk05"
I20260709 13:04:44.437691   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:04:44.441092 12271 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:44.441939   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:04:44.442255   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/master-0-root
uuid: "da2bdb6bde834ca588d7e7a427254aa5"
format_stamp: "Formatted at 2026-07-09 13:04:44 on dist-test-slave-jk05"
I20260709 13:04:44.442543   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:44.459414   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:44.460496   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:44.495978   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:35413
I20260709 13:04:44.496279 12322 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:35413 every 8 connection(s)
I20260709 13:04:44.499946 12323 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:44.511560 12323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5: Bootstrap starting.
I20260709 13:04:44.516436 12323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:44.521320 12323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5: No bootstrap required, opened a new log
I20260709 13:04:44.524853 12323 raft_consensus.cc:359] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2bdb6bde834ca588d7e7a427254aa5" member_type: VOTER }
I20260709 13:04:44.525504 12323 raft_consensus.cc:385] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:44.525820 12323 raft_consensus.cc:740] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2bdb6bde834ca588d7e7a427254aa5, State: Initialized, Role: FOLLOWER
I20260709 13:04:44.526636 12323 consensus_queue.cc:260] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2bdb6bde834ca588d7e7a427254aa5" member_type: VOTER }
I20260709 13:04:44.527263 12323 raft_consensus.cc:399] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:44.527573 12323 raft_consensus.cc:493] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:44.527942 12323 raft_consensus.cc:3060] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:44.534247 12323 raft_consensus.cc:515] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2bdb6bde834ca588d7e7a427254aa5" member_type: VOTER }
I20260709 13:04:44.534768 12323 leader_election.cc:304] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da2bdb6bde834ca588d7e7a427254aa5; no voters: 
I20260709 13:04:44.536005 12323 leader_election.cc:290] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:44.536360 12326 raft_consensus.cc:2804] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:44.537814 12326 raft_consensus.cc:697] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [term 1 LEADER]: Becoming Leader. State: Replica: da2bdb6bde834ca588d7e7a427254aa5, State: Running, Role: LEADER
I20260709 13:04:44.538537 12326 consensus_queue.cc:237] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2bdb6bde834ca588d7e7a427254aa5" member_type: VOTER }
I20260709 13:04:44.538975 12323 sys_catalog.cc:565] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:44.541294 12327 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da2bdb6bde834ca588d7e7a427254aa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2bdb6bde834ca588d7e7a427254aa5" member_type: VOTER } }
I20260709 13:04:44.541324 12328 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da2bdb6bde834ca588d7e7a427254aa5. Latest consensus state: current_term: 1 leader_uuid: "da2bdb6bde834ca588d7e7a427254aa5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2bdb6bde834ca588d7e7a427254aa5" member_type: VOTER } }
I20260709 13:04:44.542132 12327 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:44.542194 12328 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:44.547675 12331 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:44.555465 12331 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:44.556516   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:44.564976 12331 catalog_manager.cc:1383] Generated new cluster ID: 35f522eb359b48059d1257ad286a340c
I20260709 13:04:44.565328 12331 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:44.583621 12331 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:44.585003 12331 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:44.598420 12331 catalog_manager.cc:6070] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5: Generated new TSK 0
I20260709 13:04:44.599092 12331 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:04:44.623365   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:44.629244 12344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:44.630650 12345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:44.632320 12347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:44.632490   431 server_base.cc:1061] running on GCE node
I20260709 13:04:44.633664   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:44.633877   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:44.634078   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602284634059 us; error 0 us; skew 500 ppm
I20260709 13:04:44.634642   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:44.636984   431 webserver.cc:533] Webserver started at http://127.0.107.193:39811/ using document root <none> and password file <none>
I20260709 13:04:44.637563   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:44.637766   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:44.638072   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:44.639159   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-0-root/instance:
uuid: "9b85e1f979194d6889a0490e970ebfde"
format_stamp: "Formatted at 2026-07-09 13:04:44 on dist-test-slave-jk05"
I20260709 13:04:44.643921   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 13:04:44.647466 12352 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:44.648232   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:04:44.648545   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-0-root
uuid: "9b85e1f979194d6889a0490e970ebfde"
format_stamp: "Formatted at 2026-07-09 13:04:44 on dist-test-slave-jk05"
I20260709 13:04:44.648835   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:44.666293   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:44.667541   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:44.669071   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:44.671556   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:44.671797   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:44.672053   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:44.672223   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:44.711619   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:46243
I20260709 13:04:44.711724 12414 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:46243 every 8 connection(s)
I20260709 13:04:44.716571   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:44.724565 12419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:44.726255 12420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:44.729283 12422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:44.730325   431 server_base.cc:1061] running on GCE node
I20260709 13:04:44.730341 12415 heartbeater.cc:344] Connected to a master server at 127.0.107.254:35413
I20260709 13:04:44.731019 12415 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:44.731807 12415 heartbeater.cc:507] Master 127.0.107.254:35413 requested a full tablet report, sending...
I20260709 13:04:44.732074   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:44.732357   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:44.732545   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602284732527 us; error 0 us; skew 500 ppm
I20260709 13:04:44.733143   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:44.734097 12288 ts_manager.cc:194] Registered new tserver with Master: 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243)
I20260709 13:04:44.735919   431 webserver.cc:533] Webserver started at http://127.0.107.194:40169/ using document root <none> and password file <none>
I20260709 13:04:44.736382 12288 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48368
I20260709 13:04:44.736479   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:44.736820   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:44.737097   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:44.738323   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-1-root/instance:
uuid: "e82db363f6484aa3be77e60927c02fa9"
format_stamp: "Formatted at 2026-07-09 13:04:44 on dist-test-slave-jk05"
I20260709 13:04:44.743099   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:04:44.746714 12427 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:44.747572   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:04:44.747892   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-1-root
uuid: "e82db363f6484aa3be77e60927c02fa9"
format_stamp: "Formatted at 2026-07-09 13:04:44 on dist-test-slave-jk05"
I20260709 13:04:44.748208   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:44.767282   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:44.768381   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:44.769876   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:44.772161   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:44.772397   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:44.772652   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:44.772828   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:44.811687   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:35975
I20260709 13:04:44.811789 12489 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:35975 every 8 connection(s)
I20260709 13:04:44.816792   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:44.823659 12493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:44.824988 12494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:44.827903 12496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:44.828081 12490 heartbeater.cc:344] Connected to a master server at 127.0.107.254:35413
I20260709 13:04:44.828788 12490 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:44.829413   431 server_base.cc:1061] running on GCE node
I20260709 13:04:44.829968 12490 heartbeater.cc:507] Master 127.0.107.254:35413 requested a full tablet report, sending...
I20260709 13:04:44.830452   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:44.830742   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:44.830941   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602284830922 us; error 0 us; skew 500 ppm
I20260709 13:04:44.831565   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:44.832258 12288 ts_manager.cc:194] Registered new tserver with Master: e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:44.834020 12288 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48378
I20260709 13:04:44.834208   431 webserver.cc:533] Webserver started at http://127.0.107.195:37809/ using document root <none> and password file <none>
I20260709 13:04:44.834877   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:44.835101   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:44.835379   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:44.836488   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-2-root/instance:
uuid: "a0184626ff924a26959a272550a00c05"
format_stamp: "Formatted at 2026-07-09 13:04:44 on dist-test-slave-jk05"
I20260709 13:04:44.841029   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:04:44.844651 12501 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:44.845652   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:04:44.845975   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-2-root
uuid: "a0184626ff924a26959a272550a00c05"
format_stamp: "Formatted at 2026-07-09 13:04:44 on dist-test-slave-jk05"
I20260709 13:04:44.846287   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783601935286233-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:44.873533   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:44.874644   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:44.876084   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:04:44.878360   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:04:44.878611   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:44.878857   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:04:44.879040   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:44.918984   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:36193
I20260709 13:04:44.919095 12563 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:36193 every 8 connection(s)
I20260709 13:04:44.933081 12564 heartbeater.cc:344] Connected to a master server at 127.0.107.254:35413
I20260709 13:04:44.933631 12564 heartbeater.cc:461] Registering TS with master...
I20260709 13:04:44.934360 12564 heartbeater.cc:507] Master 127.0.107.254:35413 requested a full tablet report, sending...
I20260709 13:04:44.936371 12288 ts_manager.cc:194] Registered new tserver with Master: a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:44.937218   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014834372s
I20260709 13:04:44.938110 12288 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48384
I20260709 13:04:44.969357 12288 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48392:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:44.971966 12288 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:45.055673 12452 tablet_service.cc:1511] Processing CreateTablet for tablet 5ead0cc836794861a8c8c15929eb6e3a (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:45.056725 12454 tablet_service.cc:1511] Processing CreateTablet for tablet 75504590d7e64c50bbb8f946ea1fc1c3 (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:45.057283 12452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ead0cc836794861a8c8c15929eb6e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.059103 12453 tablet_service.cc:1511] Processing CreateTablet for tablet 679e72f804654636aca1ad4dc51900c6 (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:45.060571 12453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679e72f804654636aca1ad4dc51900c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.055673 12455 tablet_service.cc:1511] Processing CreateTablet for tablet 2c059adc4ffb476596307903819bb1c0 (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:45.062904 12455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c059adc4ffb476596307903819bb1c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.064350 12529 tablet_service.cc:1511] Processing CreateTablet for tablet 2c059adc4ffb476596307903819bb1c0 (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:45.065752 12529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c059adc4ffb476596307903819bb1c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.067530 12450 tablet_service.cc:1511] Processing CreateTablet for tablet ee78aec6998e4a29a17f470c6d8f0cd1 (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:45.067728 12451 tablet_service.cc:1511] Processing CreateTablet for tablet 5a44c20b12ff46099f11c28404c1d89b (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:45.068953 12454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75504590d7e64c50bbb8f946ea1fc1c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.069665 12451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a44c20b12ff46099f11c28404c1d89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.070539 12528 tablet_service.cc:1511] Processing CreateTablet for tablet 75504590d7e64c50bbb8f946ea1fc1c3 (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:45.071946 12528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75504590d7e64c50bbb8f946ea1fc1c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.072667 12450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee78aec6998e4a29a17f470c6d8f0cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.074993 12525 tablet_service.cc:1511] Processing CreateTablet for tablet 5a44c20b12ff46099f11c28404c1d89b (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:45.076452 12525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a44c20b12ff46099f11c28404c1d89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.078516 12526 tablet_service.cc:1511] Processing CreateTablet for tablet 5ead0cc836794861a8c8c15929eb6e3a (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:45.079968 12526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ead0cc836794861a8c8c15929eb6e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.082572 12527 tablet_service.cc:1511] Processing CreateTablet for tablet 679e72f804654636aca1ad4dc51900c6 (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:45.084021 12527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679e72f804654636aca1ad4dc51900c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.085626 12524 tablet_service.cc:1511] Processing CreateTablet for tablet ee78aec6998e4a29a17f470c6d8f0cd1 (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:45.087105 12524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee78aec6998e4a29a17f470c6d8f0cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.110709 12380 tablet_service.cc:1511] Processing CreateTablet for tablet 2c059adc4ffb476596307903819bb1c0 (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:45.112807 12380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c059adc4ffb476596307903819bb1c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.115010 12379 tablet_service.cc:1511] Processing CreateTablet for tablet 75504590d7e64c50bbb8f946ea1fc1c3 (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:45.118964 12378 tablet_service.cc:1511] Processing CreateTablet for tablet 679e72f804654636aca1ad4dc51900c6 (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:45.119567 12579 tablet_bootstrap.cc:492] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:45.117187 12379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 75504590d7e64c50bbb8f946ea1fc1c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.121469 12377 tablet_service.cc:1511] Processing CreateTablet for tablet 5ead0cc836794861a8c8c15929eb6e3a (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:45.121469 12580 tablet_bootstrap.cc:492] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:45.123413 12377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ead0cc836794861a8c8c15929eb6e3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.125820 12378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 679e72f804654636aca1ad4dc51900c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.134552 12580 tablet_bootstrap.cc:654] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.134662 12376 tablet_service.cc:1511] Processing CreateTablet for tablet 5a44c20b12ff46099f11c28404c1d89b (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:45.140795 12579 tablet_bootstrap.cc:654] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.141065 12376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a44c20b12ff46099f11c28404c1d89b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.137054 12375 tablet_service.cc:1511] Processing CreateTablet for tablet ee78aec6998e4a29a17f470c6d8f0cd1 (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:45.144764 12375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee78aec6998e4a29a17f470c6d8f0cd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.172096 12374 tablet_service.cc:1511] Processing CreateTablet for tablet b2723322211a40479ee5ef03f2d6b1ef (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:45.173687 12374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2723322211a40479ee5ef03f2d6b1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.180020 12583 tablet_bootstrap.cc:492] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:45.187569 12583 tablet_bootstrap.cc:654] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.194233 12452 tablet_service.cc:1511] Processing CreateTablet for tablet b2723322211a40479ee5ef03f2d6b1ef (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:45.196259 12452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2723322211a40479ee5ef03f2d6b1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.197172 12528 tablet_service.cc:1511] Processing CreateTablet for tablet b2723322211a40479ee5ef03f2d6b1ef (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:45.198565 12528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2723322211a40479ee5ef03f2d6b1ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.209160 12380 tablet_service.cc:1511] Processing CreateTablet for tablet 66dc07eb98b744208bc1ea38b3b814ad (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:45.210678 12380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66dc07eb98b744208bc1ea38b3b814ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.275027 12452 tablet_service.cc:1511] Processing CreateTablet for tablet 66dc07eb98b744208bc1ea38b3b814ad (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:45.277011 12452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66dc07eb98b744208bc1ea38b3b814ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.277915 12528 tablet_service.cc:1511] Processing CreateTablet for tablet 66dc07eb98b744208bc1ea38b3b814ad (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:45.279325 12528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66dc07eb98b744208bc1ea38b3b814ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.291157 12579 tablet_bootstrap.cc:492] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:45.291841 12579 ts_tablet_manager.cc:1403] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.173s	user 0.026s	sys 0.123s
I20260709 13:04:45.295995 12580 tablet_bootstrap.cc:492] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:45.296794 12580 ts_tablet_manager.cc:1403] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.176s	user 0.038s	sys 0.047s
I20260709 13:04:45.299309 12579 raft_consensus.cc:359] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.300148 12579 raft_consensus.cc:385] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.300448 12579 raft_consensus.cc:740] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.300771 12580 raft_consensus.cc:359] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.304849 12583 tablet_bootstrap.cc:492] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:45.301855 12580 raft_consensus.cc:385] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.305156 12380 tablet_service.cc:1511] Processing CreateTablet for tablet 97cb4f0685fa44f79e19ae9b9eabf3fa (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:45.301378 12579 consensus_queue.cc:260] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.305810 12580 raft_consensus.cc:740] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.306527 12380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97cb4f0685fa44f79e19ae9b9eabf3fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.306612 12583 ts_tablet_manager.cc:1403] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.127s	user 0.033s	sys 0.031s
I20260709 13:04:45.310009 12583 raft_consensus.cc:359] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } }
I20260709 13:04:45.312110 12583 raft_consensus.cc:385] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.312855 12583 raft_consensus.cc:740] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.306667 12580 consensus_queue.cc:260] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.314579 12583 consensus_queue.cc:260] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } }
I20260709 13:04:45.319754 12579 ts_tablet_manager.cc:1434] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.028s	user 0.004s	sys 0.010s
I20260709 13:04:45.320710 12564 heartbeater.cc:499] Master 127.0.107.254:35413 was elected leader, sending a full tablet report...
I20260709 13:04:45.321935 12580 ts_tablet_manager.cc:1434] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.025s	user 0.011s	sys 0.002s
I20260709 13:04:45.323081 12580 tablet_bootstrap.cc:492] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:45.323644 12490 heartbeater.cc:499] Master 127.0.107.254:35413 was elected leader, sending a full tablet report...
I20260709 13:04:45.322753 12452 tablet_service.cc:1511] Processing CreateTablet for tablet 97cb4f0685fa44f79e19ae9b9eabf3fa (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:45.328019 12452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97cb4f0685fa44f79e19ae9b9eabf3fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.330453 12580 tablet_bootstrap.cc:654] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.338832 12415 heartbeater.cc:499] Master 127.0.107.254:35413 was elected leader, sending a full tablet report...
I20260709 13:04:45.342027 12524 tablet_service.cc:1511] Processing CreateTablet for tablet 97cb4f0685fa44f79e19ae9b9eabf3fa (DEFAULT_TABLE table=test-workload [id=d2f6cf3b4cbc4075824899f7b7e89b2b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:45.343416 12524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97cb4f0685fa44f79e19ae9b9eabf3fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:45.356271 12579 tablet_bootstrap.cc:492] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:45.362898 12585 raft_consensus.cc:493] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.363799 12585 raft_consensus.cc:515] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.371544 12583 ts_tablet_manager.cc:1434] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.064s	user 0.008s	sys 0.006s
I20260709 13:04:45.372725 12583 tablet_bootstrap.cc:492] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:45.375422 12579 tablet_bootstrap.cc:654] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.376976 12583 tablet_bootstrap.cc:654] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.381217 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75504590d7e64c50bbb8f946ea1fc1c3" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:45.385391 12580 tablet_bootstrap.cc:492] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:45.386241 12580 ts_tablet_manager.cc:1403] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.063s	user 0.019s	sys 0.030s
I20260709 13:04:45.390406 12580 raft_consensus.cc:359] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } }
I20260709 13:04:45.391474 12580 raft_consensus.cc:385] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.391855 12580 raft_consensus.cc:740] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.392829 12580 consensus_queue.cc:260] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } }
I20260709 13:04:45.395635 12580 ts_tablet_manager.cc:1434] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.003s
I20260709 13:04:45.396921 12580 tablet_bootstrap.cc:492] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:45.405440 12580 tablet_bootstrap.cc:654] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.409960 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75504590d7e64c50bbb8f946ea1fc1c3" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
W20260709 13:04:45.412484 12505 leader_election.cc:343] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:04:45.414713 12504 leader_election.cc:343] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.415671 12504 leader_election.cc:304] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0184626ff924a26959a272550a00c05; no voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:45.420678 12585 leader_election.cc:290] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:45.421584 12585 raft_consensus.cc:2749] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.424741 12579 tablet_bootstrap.cc:492] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:45.425320 12579 ts_tablet_manager.cc:1403] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.005s
I20260709 13:04:45.428160 12579 raft_consensus.cc:359] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.428870 12579 raft_consensus.cc:385] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.429159 12579 raft_consensus.cc:740] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.431922 12579 consensus_queue.cc:260] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.435465 12579 ts_tablet_manager.cc:1434] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:04:45.435657 12583 tablet_bootstrap.cc:492] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:45.436107 12580 tablet_bootstrap.cc:492] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:45.436211 12583 ts_tablet_manager.cc:1403] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.064s	user 0.028s	sys 0.028s
I20260709 13:04:45.436566 12579 tablet_bootstrap.cc:492] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:45.437002 12580 ts_tablet_manager.cc:1403] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.024s
I20260709 13:04:45.439642 12583 raft_consensus.cc:359] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.440506 12583 raft_consensus.cc:385] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.440109 12580 raft_consensus.cc:359] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.440889 12583 raft_consensus.cc:740] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.441258 12580 raft_consensus.cc:385] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.441671 12580 raft_consensus.cc:740] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.441941 12583 consensus_queue.cc:260] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.442519 12580 consensus_queue.cc:260] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.443651 12579 tablet_bootstrap.cc:654] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.445035 12580 ts_tablet_manager.cc:1434] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:45.446218 12580 tablet_bootstrap.cc:492] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:45.447192 12583 ts_tablet_manager.cc:1434] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:04:45.448707 12583 tablet_bootstrap.cc:492] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:45.453274 12580 tablet_bootstrap.cc:654] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.457643 12583 tablet_bootstrap.cc:654] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.463352 12579 tablet_bootstrap.cc:492] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:45.463871 12579 ts_tablet_manager.cc:1403] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.015s
I20260709 13:04:45.466897 12580 tablet_bootstrap.cc:492] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:45.466897 12579 raft_consensus.cc:359] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.467664 12580 ts_tablet_manager.cc:1403] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.005s
I20260709 13:04:45.467715 12579 raft_consensus.cc:385] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.468237 12579 raft_consensus.cc:740] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.468793 12583 tablet_bootstrap.cc:492] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:45.469336 12583 ts_tablet_manager.cc:1403] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.006s
I20260709 13:04:45.469076 12579 consensus_queue.cc:260] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.470876 12580 raft_consensus.cc:359] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.471418 12579 ts_tablet_manager.cc:1434] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:45.471701 12580 raft_consensus.cc:385] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.472000 12580 raft_consensus.cc:740] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.472514 12579 tablet_bootstrap.cc:492] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:45.472272 12583 raft_consensus.cc:359] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.472779 12580 consensus_queue.cc:260] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.473232 12583 raft_consensus.cc:385] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.473718 12583 raft_consensus.cc:740] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.474679 12583 consensus_queue.cc:260] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.475122 12580 ts_tablet_manager.cc:1434] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:45.475891 12587 raft_consensus.cc:493] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.476413 12587 raft_consensus.cc:515] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } }
I20260709 13:04:45.478590 12580 tablet_bootstrap.cc:492] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:45.479096 12587 leader_election.cc:290] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0184626ff924a26959a272550a00c05 (127.0.107.195:36193), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:45.479388 12579 tablet_bootstrap.cc:654] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.486706 12580 tablet_bootstrap.cc:654] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.489615 12583 ts_tablet_manager.cc:1434] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 13:04:45.494489 12583 tablet_bootstrap.cc:492] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:45.524612 12579 tablet_bootstrap.cc:492] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:45.525249 12579 ts_tablet_manager.cc:1403] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.053s	user 0.023s	sys 0.026s
I20260709 13:04:45.525499 12580 tablet_bootstrap.cc:492] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:45.526022 12580 ts_tablet_manager.cc:1403] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.048s	user 0.022s	sys 0.013s
I20260709 13:04:45.526651 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c059adc4ffb476596307903819bb1c0" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:45.526798 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c059adc4ffb476596307903819bb1c0" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:45.527542 12465 raft_consensus.cc:2468] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 0.
I20260709 13:04:45.527798 12583 tablet_bootstrap.cc:654] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.529081 12356 leader_election.cc:304] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:45.529071 12580 raft_consensus.cc:359] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.530157 12580 raft_consensus.cc:385] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.530388 12587 raft_consensus.cc:2804] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.530526 12580 raft_consensus.cc:740] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.531005 12587 raft_consensus.cc:493] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.531458 12580 consensus_queue.cc:260] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.531505 12587 raft_consensus.cc:3060] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.533895 12580 ts_tablet_manager.cc:1434] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 13:04:45.534212 12579 raft_consensus.cc:359] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } }
I20260709 13:04:45.534963 12579 raft_consensus.cc:385] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.535236 12579 raft_consensus.cc:740] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.536120 12579 consensus_queue.cc:260] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } }
I20260709 13:04:45.536844 12580 tablet_bootstrap.cc:492] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
W20260709 13:04:45.531867 12355 leader_election.cc:343] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0184626ff924a26959a272550a00c05 (127.0.107.195:36193): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.542753 12579 ts_tablet_manager.cc:1434] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 13:04:45.543941 12579 tablet_bootstrap.cc:492] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:45.544353 12580 tablet_bootstrap.cc:654] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:45.549460 12336 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:45.557524 12587 raft_consensus.cc:515] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } }
I20260709 13:04:45.561502 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c059adc4ffb476596307903819bb1c0" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:45.562158 12539 raft_consensus.cc:3060] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.562922 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c059adc4ffb476596307903819bb1c0" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:45.563403 12579 tablet_bootstrap.cc:654] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.563628 12465 raft_consensus.cc:3060] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.569787 12539 raft_consensus.cc:2468] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 1.
I20260709 13:04:45.571527 12355 leader_election.cc:304] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05; no voters: 
I20260709 13:04:45.572336 12587 leader_election.cc:290] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Requested vote from peers a0184626ff924a26959a272550a00c05 (127.0.107.195:36193), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:45.576820 12592 raft_consensus.cc:493] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.577838 12592 raft_consensus.cc:515] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.563160 12580 tablet_bootstrap.cc:492] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:45.579512 12580 ts_tablet_manager.cc:1403] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.010s
I20260709 13:04:45.580921 12583 tablet_bootstrap.cc:492] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:45.581590 12583 ts_tablet_manager.cc:1403] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.087s	user 0.020s	sys 0.037s
I20260709 13:04:45.582021 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc07eb98b744208bc1ea38b3b814ad" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:45.582270 12592 leader_election.cc:290] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:45.582966 12580 raft_consensus.cc:359] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.577847 12587 raft_consensus.cc:2804] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.583729 12580 raft_consensus.cc:385] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.577019 12465 raft_consensus.cc:2468] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 1.
W20260709 13:04:45.584486 12504 leader_election.cc:343] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.584299 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc07eb98b744208bc1ea38b3b814ad" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:45.584142 12587 raft_consensus.cc:697] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Becoming Leader. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
W20260709 13:04:45.586329 12505 leader_election.cc:343] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.586290 12587 consensus_queue.cc:237] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } }
I20260709 13:04:45.586678 12583 raft_consensus.cc:359] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.586910 12505 leader_election.cc:304] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0184626ff924a26959a272550a00c05; no voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:45.584165 12580 raft_consensus.cc:740] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.587536 12583 raft_consensus.cc:385] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.588225 12583 raft_consensus.cc:740] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.588455 12592 raft_consensus.cc:2749] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.589394 12583 consensus_queue.cc:260] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.588402 12580 consensus_queue.cc:260] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.592679 12580 ts_tablet_manager.cc:1434] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 13:04:45.598104 12580 tablet_bootstrap.cc:492] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:45.606951 12580 tablet_bootstrap.cc:654] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.607530 12583 ts_tablet_manager.cc:1434] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.026s	user 0.011s	sys 0.010s
I20260709 13:04:45.608783 12583 tablet_bootstrap.cc:492] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:45.611840 12579 tablet_bootstrap.cc:492] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:45.612427 12579 ts_tablet_manager.cc:1403] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.069s	user 0.012s	sys 0.014s
I20260709 13:04:45.614151 12583 tablet_bootstrap.cc:654] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.615590 12579 raft_consensus.cc:359] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.616487 12579 raft_consensus.cc:385] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.616815 12579 raft_consensus.cc:740] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.616225 12287 catalog_manager.cc:5697] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b85e1f979194d6889a0490e970ebfde (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9b85e1f979194d6889a0490e970ebfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:45.617615 12579 consensus_queue.cc:260] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.619789 12580 tablet_bootstrap.cc:492] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:45.620671 12580 ts_tablet_manager.cc:1403] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.000s
I20260709 13:04:45.624287 12580 raft_consensus.cc:359] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.625097 12580 raft_consensus.cc:385] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.625466 12580 raft_consensus.cc:740] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.625535 12583 tablet_bootstrap.cc:492] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:45.626224 12583 ts_tablet_manager.cc:1403] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.018s	user 0.003s	sys 0.013s
I20260709 13:04:45.626433 12580 consensus_queue.cc:260] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.629380 12583 raft_consensus.cc:359] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.629932 12579 ts_tablet_manager.cc:1434] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 13:04:45.630211 12583 raft_consensus.cc:385] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.630579 12583 raft_consensus.cc:740] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.631543 12580 ts_tablet_manager.cc:1434] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:04:45.631510 12583 consensus_queue.cc:260] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.632885 12580 tablet_bootstrap.cc:492] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:45.634815 12583 ts_tablet_manager.cc:1434] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:04:45.636023 12583 tablet_bootstrap.cc:492] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:45.640746 12580 tablet_bootstrap.cc:654] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.643024 12583 tablet_bootstrap.cc:654] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.644517 12579 tablet_bootstrap.cc:492] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:45.656776 12580 tablet_bootstrap.cc:492] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:45.657413 12580 ts_tablet_manager.cc:1403] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.011s
I20260709 13:04:45.658277 12579 tablet_bootstrap.cc:654] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.658620 12587 raft_consensus.cc:493] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.659180 12587 raft_consensus.cc:515] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.660287 12580 raft_consensus.cc:359] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.661051 12580 raft_consensus.cc:385] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.661401 12580 raft_consensus.cc:740] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.662362 12580 consensus_queue.cc:260] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.663336 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc07eb98b744208bc1ea38b3b814ad" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:45.664085 12539 raft_consensus.cc:2468] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 0.
I20260709 13:04:45.665608 12580 ts_tablet_manager.cc:1434] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:04:45.665958 12355 leader_election.cc:304] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05; no voters: 
I20260709 13:04:45.666867 12587 leader_election.cc:290] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.666980 12580 tablet_bootstrap.cc:492] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:45.667070 12604 raft_consensus.cc:2804] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.667752 12604 raft_consensus.cc:493] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.668190 12604 raft_consensus.cc:3060] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.668633 12587 raft_consensus.cc:493] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.669183 12587 raft_consensus.cc:515] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.670298 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc07eb98b744208bc1ea38b3b814ad" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:45.671011 12465 raft_consensus.cc:2468] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 0.
I20260709 13:04:45.673017 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679e72f804654636aca1ad4dc51900c6" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:45.673504 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679e72f804654636aca1ad4dc51900c6" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:45.674170 12465 raft_consensus.cc:2468] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 0.
W20260709 13:04:45.674561 12355 leader_election.cc:343] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0184626ff924a26959a272550a00c05 (127.0.107.195:36193): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.675181 12587 leader_election.cc:290] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.675590 12356 leader_election.cc:304] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: a0184626ff924a26959a272550a00c05
I20260709 13:04:45.676730 12587 raft_consensus.cc:2804] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.677094 12587 raft_consensus.cc:493] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.677507 12587 raft_consensus.cc:3060] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.678886 12580 tablet_bootstrap.cc:654] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.684486 12604 raft_consensus.cc:515] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.684933 12587 raft_consensus.cc:515] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.686960 12586 raft_consensus.cc:493] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.690653 12586 raft_consensus.cc:515] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.693655 12586 leader_election.cc:290] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.695511 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc07eb98b744208bc1ea38b3b814ad" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:45.696218 12465 raft_consensus.cc:3060] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.696995 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679e72f804654636aca1ad4dc51900c6" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
W20260709 13:04:45.698559 12355 leader_election.cc:343] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a0184626ff924a26959a272550a00c05 (127.0.107.195:36193): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.699148 12587 leader_election.cc:290] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Requested vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.699795 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679e72f804654636aca1ad4dc51900c6" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:45.700747 12464 raft_consensus.cc:3060] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.708890 12580 tablet_bootstrap.cc:492] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:45.709864 12580 ts_tablet_manager.cc:1403] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.043s	user 0.028s	sys 0.009s
I20260709 13:04:45.714146 12580 raft_consensus.cc:359] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.715265 12580 raft_consensus.cc:385] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.715699 12580 raft_consensus.cc:740] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.716717 12580 consensus_queue.cc:260] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.719718 12580 ts_tablet_manager.cc:1434] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:04:45.725939 12586 raft_consensus.cc:493] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.726785 12586 raft_consensus.cc:515] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.729835 12586 leader_election.cc:290] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.733839 12464 raft_consensus.cc:2468] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 1.
I20260709 13:04:45.735718 12356 leader_election.cc:304] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: a0184626ff924a26959a272550a00c05
I20260709 13:04:45.737577 12587 raft_consensus.cc:2804] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.738319 12587 raft_consensus.cc:697] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Becoming Leader. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
I20260709 13:04:45.739430 12604 leader_election.cc:290] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Requested vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.739439 12587 consensus_queue.cc:237] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.740779 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc07eb98b744208bc1ea38b3b814ad" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:45.741477 12539 raft_consensus.cc:3060] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.747012 12465 raft_consensus.cc:2468] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 1.
I20260709 13:04:45.748993 12604 raft_consensus.cc:493] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.752722 12604 raft_consensus.cc:515] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.750540 12356 leader_election.cc:304] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:45.754315 12579 tablet_bootstrap.cc:492] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:45.754845 12603 raft_consensus.cc:2804] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.755102 12579 ts_tablet_manager.cc:1403] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.111s	user 0.030s	sys 0.062s
I20260709 13:04:45.756393 12538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
W20260709 13:04:45.758422 12355 leader_election.cc:343] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0184626ff924a26959a272550a00c05 (127.0.107.195:36193): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.759589 12579 raft_consensus.cc:359] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.760627 12539 raft_consensus.cc:2468] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 1.
I20260709 13:04:45.761802 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:45.762569 12464 raft_consensus.cc:2468] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 0.
I20260709 13:04:45.763747 12356 leader_election.cc:304] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: a0184626ff924a26959a272550a00c05
I20260709 13:04:45.760679 12579 raft_consensus.cc:385] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.765059 12579 raft_consensus.cc:740] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.765914 12604 leader_election.cc:290] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.765949 12579 consensus_queue.cc:260] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.768445 12583 tablet_bootstrap.cc:492] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:45.767954 12587 raft_consensus.cc:2804] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.768970 12587 raft_consensus.cc:493] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.769114 12583 ts_tablet_manager.cc:1403] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.133s	user 0.035s	sys 0.033s
I20260709 13:04:45.769379 12587 raft_consensus.cc:3060] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.770277 12579 ts_tablet_manager.cc:1434] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:04:45.771431 12579 tablet_bootstrap.cc:492] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:45.774039 12583 raft_consensus.cc:359] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.774878 12583 raft_consensus.cc:385] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.775234 12583 raft_consensus.cc:740] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.776005 12583 consensus_queue.cc:260] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.778410 12583 ts_tablet_manager.cc:1434] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:04:45.779563 12583 tablet_bootstrap.cc:492] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:45.780063 12603 raft_consensus.cc:697] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Becoming Leader. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
I20260709 13:04:45.781121 12603 consensus_queue.cc:237] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.782691 12579 tablet_bootstrap.cc:654] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.786495 12583 tablet_bootstrap.cc:654] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.792192 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ead0cc836794861a8c8c15929eb6e3a" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:45.793251 12539 raft_consensus.cc:2468] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:45.795662 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a44c20b12ff46099f11c28404c1d89b" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:45.796584 12539 raft_consensus.cc:2468] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:45.798974 12430 leader_election.cc:304] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:45.800863 12586 raft_consensus.cc:2804] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.801482 12586 raft_consensus.cc:493] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.801185 12587 raft_consensus.cc:515] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.802114 12586 raft_consensus.cc:3060] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.802929 12430 leader_election.cc:304] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:45.812305 12622 raft_consensus.cc:2804] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.812909 12622 raft_consensus.cc:493] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.816903 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:45.817806 12464 raft_consensus.cc:3060] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.829000 12287 catalog_manager.cc:5697] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b85e1f979194d6889a0490e970ebfde (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9b85e1f979194d6889a0490e970ebfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:45.831872 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:45.831918 12287 catalog_manager.cc:5697] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b85e1f979194d6889a0490e970ebfde (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9b85e1f979194d6889a0490e970ebfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
W20260709 13:04:45.834106 12355 leader_election.cc:343] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a0184626ff924a26959a272550a00c05 (127.0.107.195:36193): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:45.828931 12464 raft_consensus.cc:2468] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 1.
I20260709 13:04:45.839886 12579 tablet_bootstrap.cc:492] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:45.837409 12586 raft_consensus.cc:515] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.839784 12356 leader_election.cc:304] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: a0184626ff924a26959a272550a00c05
I20260709 13:04:45.841005 12603 raft_consensus.cc:2804] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.841557 12603 raft_consensus.cc:697] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Becoming Leader. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
I20260709 13:04:45.842129 12586 leader_election.cc:290] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.842865 12586 raft_consensus.cc:493] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.842424 12603 consensus_queue.cc:237] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.843354 12586 raft_consensus.cc:515] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.845572 12586 leader_election.cc:290] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.836740 12587 leader_election.cc:290] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Requested vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.813339 12622 raft_consensus.cc:3060] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.848421 12579 ts_tablet_manager.cc:1403] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.077s	user 0.034s	sys 0.029s
I20260709 13:04:45.851257 12583 tablet_bootstrap.cc:492] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:45.851817 12583 ts_tablet_manager.cc:1403] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.072s	user 0.021s	sys 0.035s
I20260709 13:04:45.852131 12579 raft_consensus.cc:359] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.852941 12579 raft_consensus.cc:385] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.853245 12579 raft_consensus.cc:740] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.854077 12579 consensus_queue.cc:260] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.848280 12586 raft_consensus.cc:493] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.855937 12586 raft_consensus.cc:515] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.855046 12583 raft_consensus.cc:359] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.856379 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ead0cc836794861a8c8c15929eb6e3a" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:45.856788 12583 raft_consensus.cc:385] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.857159 12583 raft_consensus.cc:740] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.859947 12583 consensus_queue.cc:260] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.860463 12579 ts_tablet_manager.cc:1434] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 13:04:45.861064 12390 raft_consensus.cc:2468] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:45.861749 12579 tablet_bootstrap.cc:492] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:45.862790 12583 ts_tablet_manager.cc:1434] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:04:45.863922 12583 tablet_bootstrap.cc:492] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:45.868242 12587 raft_consensus.cc:493] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.868865 12587 raft_consensus.cc:515] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.858302 12586 leader_election.cc:290] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.869644 12579 tablet_bootstrap.cc:654] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.871268 12583 tablet_bootstrap.cc:654] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.872200 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a44c20b12ff46099f11c28404c1d89b" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:45.873123 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a44c20b12ff46099f11c28404c1d89b" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:45.873492 12539 raft_consensus.cc:2468] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 0.
I20260709 13:04:45.873677 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a44c20b12ff46099f11c28404c1d89b" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:45.872417 12587 leader_election.cc:290] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.874377 12390 raft_consensus.cc:2468] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:45.874738 12355 leader_election.cc:304] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05; no voters: 
I20260709 13:04:45.876068 12603 raft_consensus.cc:2804] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.876528 12603 raft_consensus.cc:493] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.876922 12603 raft_consensus.cc:3060] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.879046 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ead0cc836794861a8c8c15929eb6e3a" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:45.879922 12583 tablet_bootstrap.cc:492] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:45.879973 12390 raft_consensus.cc:3060] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.880704 12583 ts_tablet_manager.cc:1403] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.001s
I20260709 13:04:45.883584 12583 raft_consensus.cc:359] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.884274 12583 raft_consensus.cc:385] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.884534 12583 raft_consensus.cc:740] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.885227 12583 consensus_queue.cc:260] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.886497 12622 raft_consensus.cc:515] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.887708 12583 ts_tablet_manager.cc:1434] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:04:45.887763 12464 raft_consensus.cc:2393] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9b85e1f979194d6889a0490e970ebfde in current term 1: Already voted for candidate e82db363f6484aa3be77e60927c02fa9 in this term.
I20260709 13:04:45.888991 12583 tablet_bootstrap.cc:492] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:45.891474 12603 raft_consensus.cc:515] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.891927 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ead0cc836794861a8c8c15929eb6e3a" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:45.894496 12539 raft_consensus.cc:3060] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.894369 12389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75504590d7e64c50bbb8f946ea1fc1c3" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:45.895673 12583 tablet_bootstrap.cc:654] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.895577 12389 raft_consensus.cc:2468] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:45.900395 12538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75504590d7e64c50bbb8f946ea1fc1c3" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:45.901314 12538 raft_consensus.cc:2468] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:45.904277 12389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2723322211a40479ee5ef03f2d6b1ef" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:45.893805 12622 leader_election.cc:290] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.905274 12389 raft_consensus.cc:2468] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:45.908390 12538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2723322211a40479ee5ef03f2d6b1ef" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:45.909344 12538 raft_consensus.cc:2468] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:45.912469 12389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a44c20b12ff46099f11c28404c1d89b" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:45.913667 12389 raft_consensus.cc:2393] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e82db363f6484aa3be77e60927c02fa9 in current term 1: Already voted for candidate 9b85e1f979194d6889a0490e970ebfde in this term.
I20260709 13:04:45.916265 12538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a44c20b12ff46099f11c28404c1d89b" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:45.917155 12538 raft_consensus.cc:3060] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.916411 12284 catalog_manager.cc:5697] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b85e1f979194d6889a0490e970ebfde (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9b85e1f979194d6889a0490e970ebfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:45.904594 12539 raft_consensus.cc:2468] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:45.924403 12537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a44c20b12ff46099f11c28404c1d89b" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:45.926980 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5a44c20b12ff46099f11c28404c1d89b" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:45.927793 12464 raft_consensus.cc:2393] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9b85e1f979194d6889a0490e970ebfde in current term 1: Already voted for candidate e82db363f6484aa3be77e60927c02fa9 in this term.
I20260709 13:04:45.928431 12579 tablet_bootstrap.cc:492] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:45.928941 12579 ts_tablet_manager.cc:1403] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.067s	user 0.034s	sys 0.021s
I20260709 13:04:45.929076 12356 leader_election.cc:304] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde; no voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:45.930060 12587 raft_consensus.cc:2749] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.931137 12583 tablet_bootstrap.cc:492] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:45.931676 12583 ts_tablet_manager.cc:1403] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.043s	user 0.031s	sys 0.007s
I20260709 13:04:45.931921 12579 raft_consensus.cc:359] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.932665 12579 raft_consensus.cc:385] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.932936 12579 raft_consensus.cc:740] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.925125 12603 leader_election.cc:290] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Requested vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.934396 12583 raft_consensus.cc:359] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.935210 12583 raft_consensus.cc:385] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:45.935464 12583 raft_consensus.cc:740] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:45.935322 12579 consensus_queue.cc:260] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.936235 12583 consensus_queue.cc:260] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.937345 12430 leader_election.cc:304] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:45.938251 12579 ts_tablet_manager.cc:1434] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:04:45.938331 12583 ts_tablet_manager.cc:1434] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:04:45.939555 12579 tablet_bootstrap.cc:492] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:45.940006 12390 raft_consensus.cc:2468] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:45.938318 12430 leader_election.cc:304] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:45.940807 12622 raft_consensus.cc:2804] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.941524 12622 raft_consensus.cc:493] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.941875 12624 raft_consensus.cc:2804] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:45.942329 12624 raft_consensus.cc:493] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:45.942638 12624 raft_consensus.cc:3060] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.944816 12538 raft_consensus.cc:2468] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:45.941915 12430 leader_election.cc:304] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:45.941921 12622 raft_consensus.cc:3060] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.945752 12592 raft_consensus.cc:493] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:45.946326 12586 raft_consensus.cc:2804] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.946830 12586 raft_consensus.cc:697] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:45.946519 12592 raft_consensus.cc:515] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.947659 12586 consensus_queue.cc:237] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.949559 12592 leader_election.cc:290] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:45.952491 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:45.955062 12464 raft_consensus.cc:2393] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0184626ff924a26959a272550a00c05 in current term 1: Already voted for candidate 9b85e1f979194d6889a0490e970ebfde in this term.
I20260709 13:04:45.956914 12622 raft_consensus.cc:515] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.957571 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:45.959334 12622 leader_election.cc:290] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.959090 12504 leader_election.cc:304] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0184626ff924a26959a272550a00c05; no voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:45.960266 12592 raft_consensus.cc:3060] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.962546 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75504590d7e64c50bbb8f946ea1fc1c3" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:45.963222 12390 raft_consensus.cc:3060] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.963646 12624 raft_consensus.cc:515] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.965953 12624 leader_election.cc:290] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:45.966954 12538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75504590d7e64c50bbb8f946ea1fc1c3" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:45.967275 12430 leader_election.cc:304] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:45.967653 12538 raft_consensus.cc:3060] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.968230 12624 raft_consensus.cc:2804] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.968719 12624 raft_consensus.cc:697] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:45.969604 12624 consensus_queue.cc:237] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.970275 12579 tablet_bootstrap.cc:654] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:45.969696 12592 raft_consensus.cc:2749] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:45.976895 12538 raft_consensus.cc:2468] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:45.983374 12389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2723322211a40479ee5ef03f2d6b1ef" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:45.984102 12389 raft_consensus.cc:3060] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.985576 12390 raft_consensus.cc:2468] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:45.986896 12538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b2723322211a40479ee5ef03f2d6b1ef" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:45.987293 12430 leader_election.cc:304] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:45.987589 12538 raft_consensus.cc:3060] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:45.990351 12624 raft_consensus.cc:2804] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.989830 12284 catalog_manager.cc:5697] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:45.990898 12624 raft_consensus.cc:697] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:45.991751 12624 consensus_queue.cc:237] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:45.992043 12389 raft_consensus.cc:2468] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:45.995514 12538 raft_consensus.cc:2468] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:45.996743 12430 leader_election.cc:304] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:45.996994 12579 tablet_bootstrap.cc:492] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:45.997583 12579 ts_tablet_manager.cc:1403] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.058s	user 0.009s	sys 0.009s
I20260709 13:04:45.997586 12586 raft_consensus.cc:2804] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:45.998476 12586 raft_consensus.cc:697] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:45.999434 12586 consensus_queue.cc:237] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.000641 12579 raft_consensus.cc:359] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.001345 12579 raft_consensus.cc:385] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:46.001636 12579 raft_consensus.cc:740] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:46.002344 12579 consensus_queue.cc:260] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.004515 12579 ts_tablet_manager.cc:1434] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:46.010300 12287 catalog_manager.cc:5697] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.011801 12287 catalog_manager.cc:5697] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.013348 12287 catalog_manager.cc:5697] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.047484 12586 raft_consensus.cc:493] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.047621 12592 raft_consensus.cc:493] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:46.048097 12586 raft_consensus.cc:515] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.048314 12592 raft_consensus.cc:515] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.050401 12592 leader_election.cc:290] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:46.050441 12586 leader_election.cc:290] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:46.051384 12389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cb4f0685fa44f79e19ae9b9eabf3fa" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:46.051615 12538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cb4f0685fa44f79e19ae9b9eabf3fa" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:46.051731 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679e72f804654636aca1ad4dc51900c6" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:46.051992 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679e72f804654636aca1ad4dc51900c6" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:46.052117 12389 raft_consensus.cc:2468] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:46.052196 12538 raft_consensus.cc:2468] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:46.052728 12464 raft_consensus.cc:2393] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0184626ff924a26959a272550a00c05 in current term 1: Already voted for candidate 9b85e1f979194d6889a0490e970ebfde in this term.
I20260709 13:04:46.053665 12430 leader_election.cc:304] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:46.053841 12505 leader_election.cc:304] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0184626ff924a26959a272550a00c05; no voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:46.054385 12586 raft_consensus.cc:2804] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:46.054626 12592 raft_consensus.cc:3060] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.054769 12586 raft_consensus.cc:493] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:46.055172 12586 raft_consensus.cc:3060] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.060004 12592 raft_consensus.cc:2749] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:46.060279 12586 raft_consensus.cc:515] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.062319 12586 leader_election.cc:290] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:46.063077 12389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cb4f0685fa44f79e19ae9b9eabf3fa" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:46.063349 12538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cb4f0685fa44f79e19ae9b9eabf3fa" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:46.063638 12389 raft_consensus.cc:3060] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.063925 12538 raft_consensus.cc:3060] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:46.068753 12389 raft_consensus.cc:2468] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:46.068753 12538 raft_consensus.cc:2468] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:46.070420 12430 leader_election.cc:304] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:46.071117 12586 raft_consensus.cc:2804] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:46.071599 12586 raft_consensus.cc:697] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:46.073004 12586 consensus_queue.cc:237] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.074940 12603 consensus_queue.cc:1048] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.083280 12287 catalog_manager.cc:5697] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.092319 12587 consensus_queue.cc:1048] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.127278 12587 consensus_queue.cc:1048] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.138252 12618 consensus_queue.cc:1048] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.190315 12464 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "75504590d7e64c50bbb8f946ea1fc1c3"
dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
mode: GRACEFUL
new_leader_uuid: "9b85e1f979194d6889a0490e970ebfde"
 from {username='slave'} at 127.0.0.1:55014
I20260709 13:04:46.190858 12464 raft_consensus.cc:606] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Received request to transfer leadership to TS 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:46.198761 12464 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ead0cc836794861a8c8c15929eb6e3a"
dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
mode: GRACEFUL
new_leader_uuid: "9b85e1f979194d6889a0490e970ebfde"
 from {username='slave'} at 127.0.0.1:55014
I20260709 13:04:46.199241 12464 raft_consensus.cc:606] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Received request to transfer leadership to TS 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:46.213930 12389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1"
dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
mode: GRACEFUL
new_leader_uuid: "e82db363f6484aa3be77e60927c02fa9"
 from {username='slave'} at 127.0.0.1:39424
I20260709 13:04:46.214432 12389 raft_consensus.cc:606] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Received request to transfer leadership to TS e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:46.223166 12389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "66dc07eb98b744208bc1ea38b3b814ad"
dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
mode: GRACEFUL
new_leader_uuid: "e82db363f6484aa3be77e60927c02fa9"
 from {username='slave'} at 127.0.0.1:39424
I20260709 13:04:46.223719 12389 raft_consensus.cc:606] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Received request to transfer leadership to TS e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:46.229033 12464 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "97cb4f0685fa44f79e19ae9b9eabf3fa"
dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
mode: GRACEFUL
new_leader_uuid: "a0184626ff924a26959a272550a00c05"
 from {username='slave'} at 127.0.0.1:55014
I20260709 13:04:46.229609 12464 raft_consensus.cc:606] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Received request to transfer leadership to TS a0184626ff924a26959a272550a00c05
I20260709 13:04:46.282788 12618 consensus_queue.cc:1048] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.288976 12603 raft_consensus.cc:993] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde: : Instructing follower e82db363f6484aa3be77e60927c02fa9 to start an election
I20260709 13:04:46.289527 12587 raft_consensus.cc:1081] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Signalling peer e82db363f6484aa3be77e60927c02fa9 to start an election
I20260709 13:04:46.291225 12465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "66dc07eb98b744208bc1ea38b3b814ad"
dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
 from {username='slave'} at 127.0.0.1:55000
I20260709 13:04:46.292162 12465 raft_consensus.cc:493] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:46.292593 12465 raft_consensus.cc:3060] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.296386 12587 consensus_queue.cc:1048] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.299613 12465 raft_consensus.cc:515] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.304621 12389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc07eb98b744208bc1ea38b3b814ad" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:46.305253 12389 raft_consensus.cc:3055] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:46.305353 12465 leader_election.cc:290] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:46.305678 12389 raft_consensus.cc:740] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
I20260709 13:04:46.306576 12538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "66dc07eb98b744208bc1ea38b3b814ad" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:46.307300 12538 raft_consensus.cc:3060] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.306960 12389 consensus_queue.cc:260] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.308867 12389 raft_consensus.cc:3060] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.314059 12538 raft_consensus.cc:2468] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 2.
I20260709 13:04:46.314059 12389 raft_consensus.cc:2468] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 2.
I20260709 13:04:46.315373 12430 leader_election.cc:304] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:46.316164 12586 raft_consensus.cc:2804] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:46.316668 12586 raft_consensus.cc:697] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:46.317523 12586 consensus_queue.cc:237] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.323949 12284 catalog_manager.cc:5697] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 1 to 2, leader changed from 9b85e1f979194d6889a0490e970ebfde (127.0.107.193) to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.372702 12586 consensus_queue.cc:1048] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.379563 12624 raft_consensus.cc:993] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9: : Instructing follower 9b85e1f979194d6889a0490e970ebfde to start an election
I20260709 13:04:46.380070 12622 raft_consensus.cc:1081] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Signalling peer 9b85e1f979194d6889a0490e970ebfde to start an election
I20260709 13:04:46.382371 12390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "75504590d7e64c50bbb8f946ea1fc1c3"
dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
 from {username='slave'} at 127.0.0.1:39408
I20260709 13:04:46.382972 12390 raft_consensus.cc:493] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:46.383368 12390 raft_consensus.cc:3060] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.387631 12622 consensus_queue.cc:1048] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.390228 12390 raft_consensus.cc:515] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.395383 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75504590d7e64c50bbb8f946ea1fc1c3" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:46.395812 12390 leader_election.cc:290] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 2 election: Requested vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:46.396426 12538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75504590d7e64c50bbb8f946ea1fc1c3" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:46.396050 12465 raft_consensus.cc:3055] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:46.397456 12465 raft_consensus.cc:740] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:46.397107 12538 raft_consensus.cc:3060] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.398523 12465 consensus_queue.cc:260] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.399938 12465 raft_consensus.cc:3060] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.403964 12538 raft_consensus.cc:2468] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 2.
I20260709 13:04:46.405184 12355 leader_election.cc:304] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05; no voters: 
I20260709 13:04:46.405983 12587 raft_consensus.cc:2804] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:46.406145 12465 raft_consensus.cc:2468] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 2.
I20260709 13:04:46.406477 12587 raft_consensus.cc:697] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [term 2 LEADER]: Becoming Leader. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
I20260709 13:04:46.407467 12587 consensus_queue.cc:237] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.414314 12287 catalog_manager.cc:5697] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde reported cstate change: term changed from 1 to 2, leader changed from e82db363f6484aa3be77e60927c02fa9 (127.0.107.194) to 9b85e1f979194d6889a0490e970ebfde (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9b85e1f979194d6889a0490e970ebfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.447489 12622 consensus_queue.cc:1048] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.451506 12587 consensus_queue.cc:1048] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.462489 12624 consensus_queue.cc:1048] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.469034 12618 consensus_queue.cc:1048] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.480913 12587 raft_consensus.cc:993] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde: : Instructing follower e82db363f6484aa3be77e60927c02fa9 to start an election
I20260709 13:04:46.481474 12618 raft_consensus.cc:1081] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Signalling peer e82db363f6484aa3be77e60927c02fa9 to start an election
I20260709 13:04:46.482401 12622 consensus_queue.cc:1048] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.483095 12465 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1"
dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
 from {username='slave'} at 127.0.0.1:55000
I20260709 13:04:46.483713 12465 raft_consensus.cc:493] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:46.484107 12465 raft_consensus.cc:3060] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.491189 12465 raft_consensus.cc:515] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.491307 12622 consensus_queue.cc:1048] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.493350 12465 leader_election.cc:290] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:46.495472 12538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:46.496197 12538 raft_consensus.cc:3060] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.497411 12388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:46.498116 12388 raft_consensus.cc:3055] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:46.498430 12388 raft_consensus.cc:740] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
I20260709 13:04:46.499176 12388 consensus_queue.cc:260] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.500353 12388 raft_consensus.cc:3060] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.503790 12538 raft_consensus.cc:2468] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 2.
I20260709 13:04:46.507208 12430 leader_election.cc:304] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:46.508339 12622 raft_consensus.cc:2804] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:46.508870 12622 raft_consensus.cc:697] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:46.509747 12622 consensus_queue.cc:237] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.512231 12388 raft_consensus.cc:2468] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 2.
I20260709 13:04:46.516539 12624 consensus_queue.cc:1048] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.517560 12586 raft_consensus.cc:993] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9: : Instructing follower 9b85e1f979194d6889a0490e970ebfde to start an election
I20260709 13:04:46.518077 12287 catalog_manager.cc:5697] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 1 to 2, leader changed from 9b85e1f979194d6889a0490e970ebfde (127.0.107.193) to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.518101 12624 raft_consensus.cc:1081] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Signalling peer 9b85e1f979194d6889a0490e970ebfde to start an election
I20260709 13:04:46.530802 12390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ead0cc836794861a8c8c15929eb6e3a"
dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
 from {username='slave'} at 127.0.0.1:39408
I20260709 13:04:46.531438 12390 raft_consensus.cc:493] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:46.532788 12390 raft_consensus.cc:3060] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.540867 12390 raft_consensus.cc:515] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.544188 12538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ead0cc836794861a8c8c15929eb6e3a" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:46.544736 12538 raft_consensus.cc:3060] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.546022 12390 leader_election.cc:290] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 2 election: Requested vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:46.549018 12655 consensus_queue.cc:1048] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:46.551249 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ead0cc836794861a8c8c15929eb6e3a" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:46.545358 12586 consensus_queue.cc:1048] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:04:46.552726 12465 raft_consensus.cc:3055] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:46.553103 12465 raft_consensus.cc:740] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:46.553983 12465 consensus_queue.cc:260] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.555160 12465 raft_consensus.cc:3060] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.555716 12622 consensus_queue.cc:1048] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:46.563571 12465 raft_consensus.cc:2468] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 2.
I20260709 13:04:46.564962 12356 leader_election.cc:304] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:46.565511 12538 raft_consensus.cc:2468] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 2.
I20260709 13:04:46.565982 12587 raft_consensus.cc:2804] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:46.566397 12537 raft_consensus.cc:1240] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Rejecting Update request from peer e82db363f6484aa3be77e60927c02fa9 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 13:04:46.566529 12587 raft_consensus.cc:697] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [term 2 LEADER]: Becoming Leader. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
I20260709 13:04:46.567544 12587 consensus_queue.cc:237] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.578171 12287 catalog_manager.cc:5697] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde reported cstate change: term changed from 1 to 2, leader changed from e82db363f6484aa3be77e60927c02fa9 (127.0.107.194) to 9b85e1f979194d6889a0490e970ebfde (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9b85e1f979194d6889a0490e970ebfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:46.598922 12655 raft_consensus.cc:993] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9: : Instructing follower a0184626ff924a26959a272550a00c05 to start an election
I20260709 13:04:46.599431 12655 raft_consensus.cc:1081] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Signalling peer a0184626ff924a26959a272550a00c05 to start an election
I20260709 13:04:46.600996 12537 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "97cb4f0685fa44f79e19ae9b9eabf3fa"
dest_uuid: "a0184626ff924a26959a272550a00c05"
 from {username='slave'} at 127.0.0.1:49538
I20260709 13:04:46.601624 12537 raft_consensus.cc:493] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:46.602002 12537 raft_consensus.cc:3060] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.608923 12537 raft_consensus.cc:515] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.611140 12537 leader_election.cc:290] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:46.611970 12388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cb4f0685fa44f79e19ae9b9eabf3fa" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:46.612215 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97cb4f0685fa44f79e19ae9b9eabf3fa" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:46.612672 12388 raft_consensus.cc:3060] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.612867 12465 raft_consensus.cc:3055] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:46.613310 12465 raft_consensus.cc:740] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:46.614131 12465 consensus_queue.cc:260] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.615137 12465 raft_consensus.cc:3060] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:46.619026 12388 raft_consensus.cc:2468] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 2.
I20260709 13:04:46.620401 12504 leader_election.cc:304] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05; no voters: 
I20260709 13:04:46.621270 12592 raft_consensus.cc:2804] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:46.621610 12465 raft_consensus.cc:2468] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 2.
I20260709 13:04:46.622663 12592 raft_consensus.cc:697] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [term 2 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:46.623631 12592 consensus_queue.cc:237] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:46.630182 12286 catalog_manager.cc:5697] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 1 to 2, leader changed from e82db363f6484aa3be77e60927c02fa9 (127.0.107.194) to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:46.771412 12388 raft_consensus.cc:1275] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Refusing update from remote peer e82db363f6484aa3be77e60927c02fa9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:46.772917 12655 consensus_queue.cc:1048] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:46.780820 12537 raft_consensus.cc:1275] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Refusing update from remote peer e82db363f6484aa3be77e60927c02fa9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:46.783458 12622 consensus_queue.cc:1048] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:46.791021 12537 raft_consensus.cc:1275] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Refusing update from remote peer 9b85e1f979194d6889a0490e970ebfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:46.792392 12603 consensus_queue.cc:1048] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:46.800383 12465 raft_consensus.cc:1275] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Refusing update from remote peer 9b85e1f979194d6889a0490e970ebfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:46.802371 12587 consensus_queue.cc:1048] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:46.964567 12537 raft_consensus.cc:1275] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Refusing update from remote peer e82db363f6484aa3be77e60927c02fa9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:46.965965 12622 consensus_queue.cc:1048] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:46.973906 12388 raft_consensus.cc:1275] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Refusing update from remote peer e82db363f6484aa3be77e60927c02fa9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:46.975689 12655 consensus_queue.cc:1048] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:47.019204 12537 raft_consensus.cc:1275] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Refusing update from remote peer 9b85e1f979194d6889a0490e970ebfde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:47.020643 12587 consensus_queue.cc:1048] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:47.030637 12465 raft_consensus.cc:1275] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Refusing update from remote peer 9b85e1f979194d6889a0490e970ebfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:47.031317 12388 raft_consensus.cc:1275] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:47.032701 12618 consensus_queue.cc:1048] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:47.032879 12592 consensus_queue.cc:1048] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:47.038491 12464 raft_consensus.cc:1275] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:47.040860 12658 consensus_queue.cc:1048] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:49.262977 12284 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48406:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:04:49.265863 12284 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:04:49.322799 12380 tablet_service.cc:1511] Processing CreateTablet for tablet 81c4673118cf4c2ab6f3a59d06d8ebc3 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:49.323928 12452 tablet_service.cc:1511] Processing CreateTablet for tablet 81c4673118cf4c2ab6f3a59d06d8ebc3 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:49.324739 12380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81c4673118cf4c2ab6f3a59d06d8ebc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.325368 12524 tablet_service.cc:1511] Processing CreateTablet for tablet 81c4673118cf4c2ab6f3a59d06d8ebc3 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:04:49.325906 12452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81c4673118cf4c2ab6f3a59d06d8ebc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.327131 12524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81c4673118cf4c2ab6f3a59d06d8ebc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.331277 12377 tablet_service.cc:1511] Processing CreateTablet for tablet aff0923e6e974f478093276d83c0a3e5 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:49.332891 12377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aff0923e6e974f478093276d83c0a3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.334367 12451 tablet_service.cc:1511] Processing CreateTablet for tablet aff0923e6e974f478093276d83c0a3e5 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:49.335871 12451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aff0923e6e974f478093276d83c0a3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.339534 12453 tablet_service.cc:1511] Processing CreateTablet for tablet 2f5b5c9972e84588bc82910949c271d5 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:49.341398 12453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5b5c9972e84588bc82910949c271d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.341959 12525 tablet_service.cc:1511] Processing CreateTablet for tablet aff0923e6e974f478093276d83c0a3e5 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:04:49.343539 12525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aff0923e6e974f478093276d83c0a3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.347463 12374 tablet_service.cc:1511] Processing CreateTablet for tablet 2f5b5c9972e84588bc82910949c271d5 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:49.349128 12374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5b5c9972e84588bc82910949c271d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.361156 12527 tablet_service.cc:1511] Processing CreateTablet for tablet 2f5b5c9972e84588bc82910949c271d5 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:04:49.362793 12527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f5b5c9972e84588bc82910949c271d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.363497 12679 tablet_bootstrap.cc:492] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:49.363240 12450 tablet_service.cc:1511] Processing CreateTablet for tablet 518f8726e0ee4660b8c8f05c648f05a4 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:49.364826 12450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 518f8726e0ee4660b8c8f05c648f05a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.367396 12454 tablet_service.cc:1511] Processing CreateTablet for tablet 8843d3f9e08d42a480f5d335516ab427 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:49.368216 12526 tablet_service.cc:1511] Processing CreateTablet for tablet 518f8726e0ee4660b8c8f05c648f05a4 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:49.369076 12454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8843d3f9e08d42a480f5d335516ab427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.369766 12526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 518f8726e0ee4660b8c8f05c648f05a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.374073 12679 tablet_bootstrap.cc:654] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.375999 12376 tablet_service.cc:1511] Processing CreateTablet for tablet 518f8726e0ee4660b8c8f05c648f05a4 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:04:49.377835 12376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 518f8726e0ee4660b8c8f05c648f05a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.379118 12375 tablet_service.cc:1511] Processing CreateTablet for tablet 8843d3f9e08d42a480f5d335516ab427 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:49.380712 12375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8843d3f9e08d42a480f5d335516ab427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.400427 12529 tablet_service.cc:1511] Processing CreateTablet for tablet 8843d3f9e08d42a480f5d335516ab427 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:04:49.402114 12529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8843d3f9e08d42a480f5d335516ab427. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.403086 12452 tablet_service.cc:1511] Processing CreateTablet for tablet 7b319df053e54d6dbaaf1e3f0eeb75b9 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:49.404587 12452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b319df053e54d6dbaaf1e3f0eeb75b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.406679 12379 tablet_service.cc:1511] Processing CreateTablet for tablet 7b319df053e54d6dbaaf1e3f0eeb75b9 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:49.408193 12379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b319df053e54d6dbaaf1e3f0eeb75b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.412928 12528 tablet_service.cc:1511] Processing CreateTablet for tablet 7b319df053e54d6dbaaf1e3f0eeb75b9 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:04:49.414503 12528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b319df053e54d6dbaaf1e3f0eeb75b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.432451 12523 tablet_service.cc:1511] Processing CreateTablet for tablet 9fa195d27155441086c43051a0eeaa78 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:49.434152 12523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fa195d27155441086c43051a0eeaa78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.436625 12451 tablet_service.cc:1511] Processing CreateTablet for tablet 9fa195d27155441086c43051a0eeaa78 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:49.438354 12451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fa195d27155441086c43051a0eeaa78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.438596 12522 tablet_service.cc:1511] Processing CreateTablet for tablet 887bbd0f3df74e8c890d6025d42553dc (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:49.438499 12450 tablet_service.cc:1511] Processing CreateTablet for tablet 887bbd0f3df74e8c890d6025d42553dc (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:49.440218 12522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887bbd0f3df74e8c890d6025d42553dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.440788 12450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887bbd0f3df74e8c890d6025d42553dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.443220 12378 tablet_service.cc:1511] Processing CreateTablet for tablet 887bbd0f3df74e8c890d6025d42553dc (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:04:49.444722 12378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 887bbd0f3df74e8c890d6025d42553dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.444587 12453 tablet_service.cc:1511] Processing CreateTablet for tablet 6bf95d8655624d84abcfb16f84c47bfd (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:49.446758 12373 tablet_service.cc:1511] Processing CreateTablet for tablet 6bf95d8655624d84abcfb16f84c47bfd (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:49.455324 12373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf95d8655624d84abcfb16f84c47bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.449395 12380 tablet_service.cc:1511] Processing CreateTablet for tablet 9fa195d27155441086c43051a0eeaa78 (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:04:49.462514 12380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9fa195d27155441086c43051a0eeaa78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.466172 12453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf95d8655624d84abcfb16f84c47bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.475544 12521 tablet_service.cc:1511] Processing CreateTablet for tablet 6bf95d8655624d84abcfb16f84c47bfd (DEFAULT_TABLE table=soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:04:49.477064 12521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bf95d8655624d84abcfb16f84c47bfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:49.492110 12679 tablet_bootstrap.cc:492] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:49.492789 12679 ts_tablet_manager.cc:1403] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.130s	user 0.007s	sys 0.007s
I20260709 13:04:49.524854 12682 tablet_bootstrap.cc:492] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:49.562304 12679 raft_consensus.cc:359] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.564325 12679 raft_consensus.cc:385] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.564757 12679 raft_consensus.cc:740] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.565779 12679 consensus_queue.cc:260] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.568274 12682 tablet_bootstrap.cc:654] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.571004 12679 ts_tablet_manager.cc:1434] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.078s	user 0.005s	sys 0.005s
I20260709 13:04:49.572237 12679 tablet_bootstrap.cc:492] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:49.572517 12681 tablet_bootstrap.cc:492] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:49.583531 12679 tablet_bootstrap.cc:654] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.594866 12682 tablet_bootstrap.cc:492] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:49.594897 12679 tablet_bootstrap.cc:492] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:49.595422 12682 ts_tablet_manager.cc:1403] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.071s	user 0.013s	sys 0.017s
I20260709 13:04:49.595952 12679 ts_tablet_manager.cc:1403] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.001s
I20260709 13:04:49.596074 12681 tablet_bootstrap.cc:654] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.598733 12682 raft_consensus.cc:359] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.599579 12682 raft_consensus.cc:385] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.599897 12682 raft_consensus.cc:740] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.599753 12679 raft_consensus.cc:359] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.600787 12682 consensus_queue.cc:260] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.600783 12679 raft_consensus.cc:385] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.602152 12679 raft_consensus.cc:740] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.602870 12679 consensus_queue.cc:260] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.605335 12679 ts_tablet_manager.cc:1434] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:04:49.606434 12679 tablet_bootstrap.cc:492] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:49.606773 12682 ts_tablet_manager.cc:1434] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:04:49.610030 12681 tablet_bootstrap.cc:492] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:49.610143 12682 tablet_bootstrap.cc:492] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:49.610780 12681 ts_tablet_manager.cc:1403] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.039s	user 0.019s	sys 0.000s
I20260709 13:04:49.613654 12679 tablet_bootstrap.cc:654] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.614208 12681 raft_consensus.cc:359] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.615139 12681 raft_consensus.cc:385] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.615461 12681 raft_consensus.cc:740] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.616268 12681 consensus_queue.cc:260] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.618310 12682 tablet_bootstrap.cc:654] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
W20260709 13:04:49.623157 12336 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:04:49.633371 12682 tablet_bootstrap.cc:492] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:49.634052 12681 ts_tablet_manager.cc:1434] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.001s
I20260709 13:04:49.634204 12682 ts_tablet_manager.cc:1403] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.026s	user 0.020s	sys 0.000s
I20260709 13:04:49.637691 12681 tablet_bootstrap.cc:492] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:49.637714 12682 raft_consensus.cc:359] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.638788 12682 raft_consensus.cc:385] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.639155 12682 raft_consensus.cc:740] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.640069 12682 consensus_queue.cc:260] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.642732 12682 ts_tablet_manager.cc:1434] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:04:49.652547 12682 tablet_bootstrap.cc:492] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:49.660359 12682 tablet_bootstrap.cc:654] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.663838 12681 tablet_bootstrap.cc:654] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.666373 12682 tablet_bootstrap.cc:492] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:49.667035 12682 ts_tablet_manager.cc:1403] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.008s
I20260709 13:04:49.670408 12682 raft_consensus.cc:359] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.671211 12682 raft_consensus.cc:385] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.671583 12682 raft_consensus.cc:740] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.672260 12679 tablet_bootstrap.cc:492] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:49.672875 12679 ts_tablet_manager.cc:1403] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.067s	user 0.030s	sys 0.024s
I20260709 13:04:49.672446 12682 consensus_queue.cc:260] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.675537 12682 ts_tablet_manager.cc:1434] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:49.676718 12682 tablet_bootstrap.cc:492] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:49.676376 12679 raft_consensus.cc:359] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.677189 12679 raft_consensus.cc:385] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.678188 12679 raft_consensus.cc:740] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.679116 12679 consensus_queue.cc:260] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.682736 12679 ts_tablet_manager.cc:1434] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.009s	user 0.000s	sys 0.008s
I20260709 13:04:49.683866 12679 tablet_bootstrap.cc:492] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:49.685215 12682 tablet_bootstrap.cc:654] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.692230 12679 tablet_bootstrap.cc:654] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.697839 12682 tablet_bootstrap.cc:492] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:49.698683 12682 ts_tablet_manager.cc:1403] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.022s	user 0.020s	sys 0.000s
I20260709 13:04:49.699393 12681 tablet_bootstrap.cc:492] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:49.699986 12681 ts_tablet_manager.cc:1403] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.024s
I20260709 13:04:49.703214 12681 raft_consensus.cc:359] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.703199 12682 raft_consensus.cc:359] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.704411 12681 raft_consensus.cc:385] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.704648 12682 raft_consensus.cc:385] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.705379 12682 raft_consensus.cc:740] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.704826 12681 raft_consensus.cc:740] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.706575 12682 consensus_queue.cc:260] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.706991 12681 consensus_queue.cc:260] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.709945 12682 ts_tablet_manager.cc:1434] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:04:49.710572 12681 ts_tablet_manager.cc:1434] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.002s
I20260709 13:04:49.711012 12682 tablet_bootstrap.cc:492] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:49.711597 12681 tablet_bootstrap.cc:492] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:49.719214 12681 tablet_bootstrap.cc:654] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.719980 12682 tablet_bootstrap.cc:654] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.725756 12679 tablet_bootstrap.cc:492] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:49.726524 12679 ts_tablet_manager.cc:1403] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.016s
I20260709 13:04:49.735957 12682 tablet_bootstrap.cc:492] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:49.730980 12679 raft_consensus.cc:359] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.735970 12681 tablet_bootstrap.cc:492] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:49.736662 12682 ts_tablet_manager.cc:1403] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.006s
I20260709 13:04:49.736697 12679 raft_consensus.cc:385] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.737361 12679 raft_consensus.cc:740] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.737504 12681 ts_tablet_manager.cc:1403] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.008s
I20260709 13:04:49.738422 12679 consensus_queue.cc:260] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.739928 12661 raft_consensus.cc:493] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.740615 12661 raft_consensus.cc:515] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.742152 12681 raft_consensus.cc:359] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.743006 12681 raft_consensus.cc:385] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.743144 12661 leader_election.cc:290] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:49.743321 12681 raft_consensus.cc:740] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.744262 12681 consensus_queue.cc:260] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.745391 12682 raft_consensus.cc:359] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.746201 12682 raft_consensus.cc:385] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.746506 12682 raft_consensus.cc:740] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.746959 12681 ts_tablet_manager.cc:1434] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 13:04:49.748121 12681 tablet_bootstrap.cc:492] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:49.749085 12679 ts_tablet_manager.cc:1434] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 13:04:49.747370 12537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81c4673118cf4c2ab6f3a59d06d8ebc3" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:49.750321 12679 tablet_bootstrap.cc:492] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:49.748874 12388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81c4673118cf4c2ab6f3a59d06d8ebc3" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:49.751101 12388 raft_consensus.cc:2468] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:49.750232 12537 raft_consensus.cc:2468] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:49.747370 12682 consensus_queue.cc:260] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.758205 12681 tablet_bootstrap.cc:654] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.760913 12682 ts_tablet_manager.cc:1434] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 13:04:49.762666 12682 tablet_bootstrap.cc:492] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:49.772094 12682 tablet_bootstrap.cc:654] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.772467 12679 tablet_bootstrap.cc:654] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.774379 12681 tablet_bootstrap.cc:492] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:49.775048 12681 ts_tablet_manager.cc:1403] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.027s	user 0.019s	sys 0.006s
I20260709 13:04:49.778410 12681 raft_consensus.cc:359] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.779322 12681 raft_consensus.cc:385] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.779683 12681 raft_consensus.cc:740] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.780472 12681 consensus_queue.cc:260] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.782584 12681 ts_tablet_manager.cc:1434] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:49.783722 12681 tablet_bootstrap.cc:492] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:49.784646 12430 leader_election.cc:304] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:49.785046 12682 tablet_bootstrap.cc:492] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:49.785591 12682 ts_tablet_manager.cc:1403] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.009s
I20260709 13:04:49.785604 12661 raft_consensus.cc:2804] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:49.787813 12661 raft_consensus.cc:493] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.788200 12661 raft_consensus.cc:3060] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.788810 12682 raft_consensus.cc:359] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.789595 12682 raft_consensus.cc:385] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.789884 12682 raft_consensus.cc:740] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.790689 12682 consensus_queue.cc:260] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.791291 12681 tablet_bootstrap.cc:654] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.793495 12679 tablet_bootstrap.cc:492] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:49.794042 12679 ts_tablet_manager.cc:1403] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.008s
I20260709 13:04:49.795101 12682 ts_tablet_manager.cc:1434] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:04:49.796200 12682 tablet_bootstrap.cc:492] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:49.797284 12679 raft_consensus.cc:359] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.798089 12679 raft_consensus.cc:385] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.798393 12679 raft_consensus.cc:740] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.799116 12679 consensus_queue.cc:260] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.801539 12679 ts_tablet_manager.cc:1434] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:04:49.802538 12679 tablet_bootstrap.cc:492] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:49.808432 12682 tablet_bootstrap.cc:654] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.809401 12679 tablet_bootstrap.cc:654] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.809252 12696 raft_consensus.cc:493] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.809953 12696 raft_consensus.cc:515] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.812520 12696 leader_election.cc:290] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:49.813814 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff0923e6e974f478093276d83c0a3e5" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:49.814586 12388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff0923e6e974f478093276d83c0a3e5" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:49.814636 12465 raft_consensus.cc:2468] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 0.
I20260709 13:04:49.816582 12505 leader_election.cc:304] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
W20260709 13:04:49.817340 12504 leader_election.cc:343] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.817593 12696 raft_consensus.cc:2804] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:49.818256 12696 raft_consensus.cc:493] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.818717 12696 raft_consensus.cc:3060] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.827631 12682 tablet_bootstrap.cc:492] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:49.828336 12682 ts_tablet_manager.cc:1403] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.005s
I20260709 13:04:49.828030 12696 raft_consensus.cc:515] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.830876 12696 leader_election.cc:290] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:49.832333 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff0923e6e974f478093276d83c0a3e5" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:49.832998 12464 raft_consensus.cc:3060] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.833737 12388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff0923e6e974f478093276d83c0a3e5" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:49.833321 12682 raft_consensus.cc:359] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.834408 12682 raft_consensus.cc:385] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.834781 12679 tablet_bootstrap.cc:492] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:49.834911 12682 raft_consensus.cc:740] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.835359 12679 ts_tablet_manager.cc:1403] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.033s	user 0.011s	sys 0.013s
W20260709 13:04:49.835693 12504 leader_election.cc:343] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.835829 12682 consensus_queue.cc:260] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.837069 12661 raft_consensus.cc:515] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.838490 12682 ts_tablet_manager.cc:1434] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:04:49.838660 12679 raft_consensus.cc:359] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.839581 12679 raft_consensus.cc:385] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.839974 12679 raft_consensus.cc:740] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.841842 12464 raft_consensus.cc:2468] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 1.
I20260709 13:04:49.841373 12679 consensus_queue.cc:260] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.842650 12388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81c4673118cf4c2ab6f3a59d06d8ebc3" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:49.843297 12388 raft_consensus.cc:3060] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.843477 12695 raft_consensus.cc:493] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.843374 12505 leader_election.cc:304] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:49.844062 12695 raft_consensus.cc:515] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.845858 12682 tablet_bootstrap.cc:492] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:49.846334 12695 leader_election.cc:290] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:49.848663 12696 raft_consensus.cc:2804] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.849412 12696 raft_consensus.cc:697] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:49.849774 12537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fa195d27155441086c43051a0eeaa78" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:49.852229 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fa195d27155441086c43051a0eeaa78" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:49.852519 12696 consensus_queue.cc:237] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
W20260709 13:04:49.853715 12430 leader_election.cc:343] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.853154 12537 raft_consensus.cc:2468] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:49.855254 12430 leader_election.cc:304] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:49.856709 12695 raft_consensus.cc:2804] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:49.857136 12695 raft_consensus.cc:493] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.857295 12682 tablet_bootstrap.cc:654] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.857657 12695 raft_consensus.cc:3060] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.861377 12388 raft_consensus.cc:2468] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:49.857604 12679 ts_tablet_manager.cc:1434] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 13:04:49.881842 12430 leader_election.cc:304] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:49.882619 12679 tablet_bootstrap.cc:492] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:49.872946 12284 catalog_manager.cc:5697] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:49.879933 12537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81c4673118cf4c2ab6f3a59d06d8ebc3" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:49.887802 12537 raft_consensus.cc:3060] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.865724 12695 raft_consensus.cc:515] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.894799 12695 leader_election.cc:290] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:49.880158 12661 leader_election.cc:290] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:49.898134 12661 raft_consensus.cc:2804] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.898598 12661 raft_consensus.cc:697] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:49.899427 12661 consensus_queue.cc:237] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.897048 12537 raft_consensus.cc:2468] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:49.919484 12388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fa195d27155441086c43051a0eeaa78" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:49.919773 12537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fa195d27155441086c43051a0eeaa78" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:49.920503 12537 raft_consensus.cc:3060] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:04:49.922580 12430 leader_election.cc:343] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.923825 12682 tablet_bootstrap.cc:492] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:49.924396 12682 ts_tablet_manager.cc:1403] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.079s	user 0.027s	sys 0.030s
I20260709 13:04:49.927346 12681 tablet_bootstrap.cc:492] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:49.928133 12681 ts_tablet_manager.cc:1403] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.145s	user 0.041s	sys 0.054s
I20260709 13:04:49.928570 12537 raft_consensus.cc:2468] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:49.931674 12679 tablet_bootstrap.cc:654] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.932112 12682 raft_consensus.cc:359] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.932809 12682 raft_consensus.cc:385] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.933097 12682 raft_consensus.cc:740] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.933868 12682 consensus_queue.cc:260] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.934662 12681 raft_consensus.cc:359] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.935696 12681 raft_consensus.cc:385] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:49.937382 12681 raft_consensus.cc:740] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:49.937938 12682 ts_tablet_manager.cc:1434] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:04:49.939024 12682 tablet_bootstrap.cc:492] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05: Bootstrap starting.
I20260709 13:04:49.938639 12681 consensus_queue.cc:260] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.945706 12430 leader_election.cc:304] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:49.946188 12667 raft_consensus.cc:493] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.947053 12706 raft_consensus.cc:2804] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:49.946928 12667 raft_consensus.cc:515] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.947697 12706 raft_consensus.cc:697] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:49.948678 12706 consensus_queue.cc:237] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.950711 12667 leader_election.cc:290] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:49.951985 12537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518f8726e0ee4660b8c8f05c648f05a4" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:49.952795 12537 raft_consensus.cc:2468] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 0.
I20260709 13:04:49.953498 12681 ts_tablet_manager.cc:1434] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.006s
I20260709 13:04:49.953775 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518f8726e0ee4660b8c8f05c648f05a4" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:49.954383 12355 leader_election.cc:304] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05; no voters: 
I20260709 13:04:49.954495 12464 raft_consensus.cc:2468] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 0.
I20260709 13:04:49.956552 12682 tablet_bootstrap.cc:654] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.957213 12667 raft_consensus.cc:2804] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:49.954694 12681 tablet_bootstrap.cc:492] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:49.957736 12667 raft_consensus.cc:493] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.958135 12667 raft_consensus.cc:3060] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.963007 12696 raft_consensus.cc:493] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:49.963596 12696 raft_consensus.cc:515] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.966019 12696 leader_election.cc:290] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:49.966632 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887bbd0f3df74e8c890d6025d42553dc" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:49.967329 12390 raft_consensus.cc:2468] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 0.
I20260709 13:04:49.968601 12504 leader_election.cc:304] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05; no voters: 
I20260709 13:04:49.969496 12696 raft_consensus.cc:2804] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:49.969985 12696 raft_consensus.cc:493] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:49.970391 12696 raft_consensus.cc:3060] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.971189 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887bbd0f3df74e8c890d6025d42553dc" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
W20260709 13:04:49.973230 12505 leader_election.cc:343] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:49.975754 12681 tablet_bootstrap.cc:654] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:49.981050 12667 raft_consensus.cc:515] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.981295 12286 catalog_manager.cc:5697] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:49.985594 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518f8726e0ee4660b8c8f05c648f05a4" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:49.986223 12464 raft_consensus.cc:3060] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.988607 12696 raft_consensus.cc:515] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.993521 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518f8726e0ee4660b8c8f05c648f05a4" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:49.994369 12539 raft_consensus.cc:3060] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.994632 12679 tablet_bootstrap.cc:492] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:49.995558 12679 ts_tablet_manager.cc:1403] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.113s	user 0.024s	sys 0.007s
I20260709 13:04:49.996423 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887bbd0f3df74e8c890d6025d42553dc" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:49.997299 12390 raft_consensus.cc:3060] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:49.999150 12679 raft_consensus.cc:359] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:49.999962 12679 raft_consensus.cc:385] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.000262 12679 raft_consensus.cc:740] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.001173 12679 consensus_queue.cc:260] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.002357 12667 leader_election.cc:290] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Requested vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:50.003270 12681 tablet_bootstrap.cc:492] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:50.003876 12681 ts_tablet_manager.cc:1403] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.000s
I20260709 13:04:50.003882 12679 ts_tablet_manager.cc:1434] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:04:50.005139 12679 tablet_bootstrap.cc:492] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:50.007190 12681 raft_consensus.cc:359] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.007966 12681 raft_consensus.cc:385] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.008277 12681 raft_consensus.cc:740] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.009143 12681 consensus_queue.cc:260] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.016168 12681 ts_tablet_manager.cc:1434] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:04:50.017378 12681 tablet_bootstrap.cc:492] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:50.015636 12682 tablet_bootstrap.cc:492] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05: No bootstrap required, opened a new log
I20260709 13:04:50.012265 12539 raft_consensus.cc:2468] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 1.
I20260709 13:04:50.018190 12682 ts_tablet_manager.cc:1403] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05: Time spent bootstrapping tablet: real 0.079s	user 0.012s	sys 0.022s
I20260709 13:04:50.014315 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887bbd0f3df74e8c890d6025d42553dc" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:50.019709 12355 leader_election.cc:304] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05; no voters: 
W20260709 13:04:50.020028 12505 leader_election.cc:343] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:04:50.012173 12679 tablet_bootstrap.cc:654] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.020716 12667 raft_consensus.cc:2804] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.012785 12696 leader_election.cc:290] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:50.021582 12667 raft_consensus.cc:697] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Becoming Leader. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
I20260709 13:04:50.021404 12682 raft_consensus.cc:359] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.023085 12682 raft_consensus.cc:385] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.023524 12682 raft_consensus.cc:740] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.023136 12667 consensus_queue.cc:237] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.024430 12682 consensus_queue.cc:260] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.025759 12681 tablet_bootstrap.cc:654] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.027113 12682 ts_tablet_manager.cc:1434] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 13:04:50.009320 12464 raft_consensus.cc:2468] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 1.
I20260709 13:04:50.015612 12390 raft_consensus.cc:2468] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 1.
I20260709 13:04:50.052455 12679 tablet_bootstrap.cc:492] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:50.052721 12504 leader_election.cc:304] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05; no voters: e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:50.053066 12679 ts_tablet_manager.cc:1403] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.048s	user 0.018s	sys 0.018s
I20260709 13:04:50.053032 12667 raft_consensus.cc:493] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.053624 12681 tablet_bootstrap.cc:492] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:50.054159 12681 ts_tablet_manager.cc:1403] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.019s
I20260709 13:04:50.053864 12667 raft_consensus.cc:515] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.056123 12667 leader_election.cc:290] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:50.056416 12679 raft_consensus.cc:359] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.057130 12679 raft_consensus.cc:385] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.057508 12679 raft_consensus.cc:740] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.057171 12681 raft_consensus.cc:359] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.057972 12681 raft_consensus.cc:385] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.058001 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5b5c9972e84588bc82910949c271d5" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:50.058382 12681 raft_consensus.cc:740] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.058323 12679 consensus_queue.cc:260] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.059420 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5b5c9972e84588bc82910949c271d5" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:50.059306 12681 consensus_queue.cc:260] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.060130 12539 raft_consensus.cc:2468] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 0.
I20260709 13:04:50.060721 12464 raft_consensus.cc:2468] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 0.
I20260709 13:04:50.061002 12679 ts_tablet_manager.cc:1434] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:04:50.062186 12356 leader_election.cc:304] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:50.062839 12696 raft_consensus.cc:2804] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.063222 12688 raft_consensus.cc:2804] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.063444 12696 raft_consensus.cc:697] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:50.063639 12688 raft_consensus.cc:493] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.064007 12688 raft_consensus.cc:3060] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.064396 12696 consensus_queue.cc:237] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.067873 12679 tablet_bootstrap.cc:492] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9: Bootstrap starting.
I20260709 13:04:50.070127 12286 catalog_manager.cc:5697] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.074962 12681 ts_tablet_manager.cc:1434] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:04:50.076146 12681 tablet_bootstrap.cc:492] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:50.080276 12688 raft_consensus.cc:515] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.083732 12679 tablet_bootstrap.cc:654] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.086036 12688 leader_election.cc:290] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Requested vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:50.086336 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5b5c9972e84588bc82910949c271d5" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:50.086966 12464 raft_consensus.cc:3060] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.087981 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5b5c9972e84588bc82910949c271d5" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:50.088613 12539 raft_consensus.cc:3060] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.094853 12696 raft_consensus.cc:493] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.095495 12696 raft_consensus.cc:515] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.097342 12464 raft_consensus.cc:2468] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 1.
I20260709 13:04:50.097904 12696 leader_election.cc:290] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:50.099082 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8843d3f9e08d42a480f5d335516ab427" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:50.099359 12356 leader_election.cc:304] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:50.099831 12465 raft_consensus.cc:2468] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 0.
I20260709 13:04:50.099798 12286 catalog_manager.cc:5697] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.100765 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8843d3f9e08d42a480f5d335516ab427" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:50.100971 12688 raft_consensus.cc:2804] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.101542 12390 raft_consensus.cc:2468] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 0.
I20260709 13:04:50.101589 12505 leader_election.cc:304] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:50.102520 12696 raft_consensus.cc:2804] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.102916 12696 raft_consensus.cc:493] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.101549 12688 raft_consensus.cc:697] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Becoming Leader. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
I20260709 13:04:50.095934 12288 catalog_manager.cc:5697] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b85e1f979194d6889a0490e970ebfde (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9b85e1f979194d6889a0490e970ebfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.104323 12688 consensus_queue.cc:237] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.104795 12681 tablet_bootstrap.cc:654] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.104749 12679 tablet_bootstrap.cc:492] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9: No bootstrap required, opened a new log
I20260709 13:04:50.106201 12679 ts_tablet_manager.cc:1403] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.004s
I20260709 13:04:50.103286 12696 raft_consensus.cc:3060] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.110541 12679 raft_consensus.cc:359] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.111606 12679 raft_consensus.cc:385] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.112036 12679 raft_consensus.cc:740] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.113240 12679 consensus_queue.cc:260] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.116305 12679 ts_tablet_manager.cc:1434] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:04:50.119786 12539 raft_consensus.cc:2468] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 1.
I20260709 13:04:50.123592 12696 raft_consensus.cc:515] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.127231 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8843d3f9e08d42a480f5d335516ab427" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:50.127941 12465 raft_consensus.cc:3060] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.128898 12681 tablet_bootstrap.cc:492] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:50.129509 12681 ts_tablet_manager.cc:1403] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.054s	user 0.024s	sys 0.004s
I20260709 13:04:50.129770 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8843d3f9e08d42a480f5d335516ab427" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:50.130453 12390 raft_consensus.cc:3060] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.132470 12681 raft_consensus.cc:359] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.133224 12681 raft_consensus.cc:385] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.133525 12681 raft_consensus.cc:740] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.134447 12681 consensus_queue.cc:260] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.136997 12681 ts_tablet_manager.cc:1434] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:04:50.137049 12465 raft_consensus.cc:2468] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 1.
I20260709 13:04:50.138095 12681 tablet_bootstrap.cc:492] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde: Bootstrap starting.
I20260709 13:04:50.138527 12505 leader_election.cc:304] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:50.139580 12665 raft_consensus.cc:2804] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.140746 12665 raft_consensus.cc:697] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:50.141654 12665 consensus_queue.cc:237] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.144052 12681 tablet_bootstrap.cc:654] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:50.146080 12390 raft_consensus.cc:2468] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 1.
I20260709 13:04:50.125996 12696 leader_election.cc:290] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:50.149111 12288 catalog_manager.cc:5697] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b85e1f979194d6889a0490e970ebfde (127.0.107.193). New cstate: current_term: 1 leader_uuid: "9b85e1f979194d6889a0490e970ebfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.151597 12681 tablet_bootstrap.cc:492] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde: No bootstrap required, opened a new log
I20260709 13:04:50.152133 12681 ts_tablet_manager.cc:1403] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 13:04:50.153891 12284 catalog_manager.cc:5697] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.154940 12681 raft_consensus.cc:359] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.155504 12681 raft_consensus.cc:385] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:50.155722 12681 raft_consensus.cc:740] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Initialized, Role: FOLLOWER
I20260709 13:04:50.156363 12681 consensus_queue.cc:260] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.158370 12681 ts_tablet_manager.cc:1434] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:04:50.162366 12665 raft_consensus.cc:493] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.162938 12665 raft_consensus.cc:515] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.164628 12665 leader_election.cc:290] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:50.165916 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b319df053e54d6dbaaf1e3f0eeb75b9" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:50.166666 12465 raft_consensus.cc:2468] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 0.
I20260709 13:04:50.165474 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b319df053e54d6dbaaf1e3f0eeb75b9" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:50.167773 12390 raft_consensus.cc:2468] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 0.
I20260709 13:04:50.168246 12505 leader_election.cc:304] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:50.169243 12665 raft_consensus.cc:2804] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.169718 12665 raft_consensus.cc:493] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.170110 12665 raft_consensus.cc:3060] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.175925 12665 raft_consensus.cc:515] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.178104 12665 leader_election.cc:290] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:50.178371 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b319df053e54d6dbaaf1e3f0eeb75b9" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:50.178691 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b319df053e54d6dbaaf1e3f0eeb75b9" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:50.179097 12390 raft_consensus.cc:3060] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.179334 12465 raft_consensus.cc:3060] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.186482 12390 raft_consensus.cc:2468] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 1.
I20260709 13:04:50.186482 12465 raft_consensus.cc:2468] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 1.
I20260709 13:04:50.187849 12505 leader_election.cc:304] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:50.188714 12665 raft_consensus.cc:2804] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.189230 12665 raft_consensus.cc:697] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:50.190109 12665 consensus_queue.cc:237] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.197964 12284 catalog_manager.cc:5697] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:50.307788 12390 raft_consensus.cc:3060] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.314704 12665 consensus_queue.cc:1048] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:50.319561 12665 consensus_queue.cc:1048] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:50.321918 12706 raft_consensus.cc:493] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.322552 12706 raft_consensus.cc:515] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.324862 12706 leader_election.cc:290] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:50.326376 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf95d8655624d84abcfb16f84c47bfd" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:50.327198 12539 raft_consensus.cc:2468] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:50.335510 12430 leader_election.cc:304] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:50.336056 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf95d8655624d84abcfb16f84c47bfd" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:50.336423 12706 raft_consensus.cc:2804] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:04:50.336787 12390 raft_consensus.cc:2468] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 0.
I20260709 13:04:50.336932 12706 raft_consensus.cc:493] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:04:50.337486 12706 raft_consensus.cc:3060] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.345609 12706 raft_consensus.cc:515] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.350608 12661 consensus_queue.cc:1048] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:50.350911 12706 leader_election.cc:290] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:50.352144 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf95d8655624d84abcfb16f84c47bfd" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:50.352823 12539 raft_consensus.cc:3060] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.355724 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf95d8655624d84abcfb16f84c47bfd" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:50.356464 12390 raft_consensus.cc:3060] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.360745 12539 raft_consensus.cc:2468] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:50.367679 12430 leader_election.cc:304] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:50.368515 12695 raft_consensus.cc:2804] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:50.369012 12695 raft_consensus.cc:697] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:50.369884 12695 consensus_queue.cc:237] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.372771 12661 consensus_queue.cc:1048] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:50.385349 12390 raft_consensus.cc:2468] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:50.389882 12288 catalog_manager.cc:5697] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:50.402258 12661 consensus_queue.cc:1048] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:50.412706 12695 raft_consensus.cc:493] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:50.413220 12695 raft_consensus.cc:515] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:50.413676 12390 raft_consensus.cc:3060] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.414896 12695 leader_election.cc:290] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:50.419235 12388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887bbd0f3df74e8c890d6025d42553dc" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b85e1f979194d6889a0490e970ebfde" is_pre_election: true
I20260709 13:04:50.420053 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887bbd0f3df74e8c890d6025d42553dc" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:50.420378 12388 raft_consensus.cc:2393] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e82db363f6484aa3be77e60927c02fa9 in current term 1: Already voted for candidate a0184626ff924a26959a272550a00c05 in this term.
I20260709 13:04:50.420667 12688 consensus_queue.cc:1048] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:50.426160 12430 leader_election.cc:304] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05
I20260709 13:04:50.429348 12695 raft_consensus.cc:3060] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:50.440106 12661 consensus_queue.cc:1048] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:50.458604 12695 raft_consensus.cc:2749] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:04:50.466142 12688 consensus_queue.cc:1048] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:50.978454 12665 consensus_queue.cc:1048] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.006203 12721 raft_consensus.cc:493] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:04:51.006789 12721 raft_consensus.cc:515] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.009577 12721 leader_election.cc:290] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:51.020123 12688 consensus_queue.cc:1048] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:04:51.035285 12539 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.003s	sys 0.006s
W20260709 13:04:51.035734 12539 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.003s	sys 0.006s
I20260709 13:04:51.054435 12741 consensus_queue.cc:1048] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.072193 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fa195d27155441086c43051a0eeaa78" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:51.078398 12739 consensus_queue.cc:1048] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.090229 12746 consensus_queue.cc:1048] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.095631 12753 consensus_queue.cc:1048] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.106204 12461 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9fa195d27155441086c43051a0eeaa78" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:51.112478 12744 consensus_queue.cc:1048] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.115386 12356 leader_election.cc:304] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde; no voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:51.120340 12739 consensus_queue.cc:1048] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.121933 12739 consensus_queue.cc:1048] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:04:51.124079 12688 raft_consensus.cc:2749] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:51.152158 12740 consensus_queue.cc:1048] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:04:51.203900 12461 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81c4673118cf4c2ab6f3a59d06d8ebc3"
dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
mode: GRACEFUL
new_leader_uuid: "9b85e1f979194d6889a0490e970ebfde"
 from {username='slave'} at 127.0.0.1:55014
I20260709 13:04:51.204679 12461 raft_consensus.cc:606] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Received request to transfer leadership to TS 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:51.267841 12537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aff0923e6e974f478093276d83c0a3e5"
dest_uuid: "a0184626ff924a26959a272550a00c05"
mode: GRACEFUL
new_leader_uuid: "9b85e1f979194d6889a0490e970ebfde"
 from {username='slave'} at 127.0.0.1:49554
I20260709 13:04:51.268514 12537 raft_consensus.cc:606] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Received request to transfer leadership to TS 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:51.281340 12537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8843d3f9e08d42a480f5d335516ab427"
dest_uuid: "a0184626ff924a26959a272550a00c05"
mode: GRACEFUL
new_leader_uuid: "e82db363f6484aa3be77e60927c02fa9"
 from {username='slave'} at 127.0.0.1:49554
I20260709 13:04:51.281988 12537 raft_consensus.cc:606] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Received request to transfer leadership to TS e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:51.288182 12537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b319df053e54d6dbaaf1e3f0eeb75b9"
dest_uuid: "a0184626ff924a26959a272550a00c05"
mode: GRACEFUL
new_leader_uuid: "e82db363f6484aa3be77e60927c02fa9"
 from {username='slave'} at 127.0.0.1:49554
I20260709 13:04:51.288836 12537 raft_consensus.cc:606] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Received request to transfer leadership to TS e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:51.298367 12537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "887bbd0f3df74e8c890d6025d42553dc"
dest_uuid: "a0184626ff924a26959a272550a00c05"
mode: GRACEFUL
new_leader_uuid: "e82db363f6484aa3be77e60927c02fa9"
 from {username='slave'} at 127.0.0.1:49554
I20260709 13:04:51.299119 12537 raft_consensus.cc:606] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Received request to transfer leadership to TS e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:51.304751 12464 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bf95d8655624d84abcfb16f84c47bfd"
dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
mode: GRACEFUL
new_leader_uuid: "a0184626ff924a26959a272550a00c05"
 from {username='slave'} at 127.0.0.1:55014
I20260709 13:04:51.305375 12464 raft_consensus.cc:606] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Received request to transfer leadership to TS a0184626ff924a26959a272550a00c05
I20260709 13:04:51.434543 12740 raft_consensus.cc:993] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9: : Instructing follower 9b85e1f979194d6889a0490e970ebfde to start an election
I20260709 13:04:51.435207 12695 raft_consensus.cc:1081] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Signalling peer 9b85e1f979194d6889a0490e970ebfde to start an election
I20260709 13:04:51.437078 12390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81c4673118cf4c2ab6f3a59d06d8ebc3"
dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
 from {username='slave'} at 127.0.0.1:39408
I20260709 13:04:51.437747 12390 raft_consensus.cc:493] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:51.438084 12390 raft_consensus.cc:3060] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.443243 12390 raft_consensus.cc:515] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.445178 12390 leader_election.cc:290] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 2 election: Requested vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:51.446506 12537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81c4673118cf4c2ab6f3a59d06d8ebc3" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:51.446436 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81c4673118cf4c2ab6f3a59d06d8ebc3" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:51.447273 12537 raft_consensus.cc:3060] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.447391 12464 raft_consensus.cc:3055] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:51.447881 12464 raft_consensus.cc:740] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:51.448798 12464 consensus_queue.cc:260] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.450134 12464 raft_consensus.cc:3060] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.455477 12537 raft_consensus.cc:2468] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 2.
I20260709 13:04:51.456872 12355 leader_election.cc:304] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05; no voters: 
I20260709 13:04:51.457715 12721 raft_consensus.cc:2804] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:51.458209 12721 raft_consensus.cc:697] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 2 LEADER]: Becoming Leader. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
I20260709 13:04:51.458930 12721 consensus_queue.cc:237] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.459529 12464 raft_consensus.cc:2468] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 2.
I20260709 13:04:51.467032 12288 catalog_manager.cc:5697] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde reported cstate change: term changed from 1 to 2, leader changed from e82db363f6484aa3be77e60927c02fa9 (127.0.107.194) to 9b85e1f979194d6889a0490e970ebfde (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9b85e1f979194d6889a0490e970ebfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.543249 12753 raft_consensus.cc:993] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05: : Instructing follower 9b85e1f979194d6889a0490e970ebfde to start an election
I20260709 13:04:51.543736 12744 raft_consensus.cc:1081] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Signalling peer 9b85e1f979194d6889a0490e970ebfde to start an election
I20260709 13:04:51.545356 12390 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aff0923e6e974f478093276d83c0a3e5"
dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
 from {username='slave'} at 127.0.0.1:39398
I20260709 13:04:51.545989 12390 raft_consensus.cc:493] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:51.546293 12390 raft_consensus.cc:3060] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.552389 12390 raft_consensus.cc:515] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.554021 12390 leader_election.cc:290] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 2 election: Requested vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:51.554896 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff0923e6e974f478093276d83c0a3e5" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:51.555065 12537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff0923e6e974f478093276d83c0a3e5" candidate_uuid: "9b85e1f979194d6889a0490e970ebfde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:51.555584 12464 raft_consensus.cc:3060] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.555691 12537 raft_consensus.cc:3055] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:51.556151 12537 raft_consensus.cc:740] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:51.556970 12537 consensus_queue.cc:260] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.558046 12537 raft_consensus.cc:3060] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.560988 12464 raft_consensus.cc:2468] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 2.
I20260709 13:04:51.562253 12356 leader_election.cc:304] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:51.562985 12721 raft_consensus.cc:2804] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:51.563395 12721 raft_consensus.cc:697] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [term 2 LEADER]: Becoming Leader. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
I20260709 13:04:51.563634 12537 raft_consensus.cc:2468] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9b85e1f979194d6889a0490e970ebfde in term 2.
I20260709 13:04:51.564126 12721 consensus_queue.cc:237] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.571806 12288 catalog_manager.cc:5697] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde reported cstate change: term changed from 1 to 2, leader changed from a0184626ff924a26959a272550a00c05 (127.0.107.195) to 9b85e1f979194d6889a0490e970ebfde (127.0.107.193). New cstate: current_term: 2 leader_uuid: "9b85e1f979194d6889a0490e970ebfde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.576421 12665 raft_consensus.cc:993] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05: : Instructing follower e82db363f6484aa3be77e60927c02fa9 to start an election
I20260709 13:04:51.576998 12744 raft_consensus.cc:1081] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Signalling peer e82db363f6484aa3be77e60927c02fa9 to start an election
I20260709 13:04:51.578830 12464 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "887bbd0f3df74e8c890d6025d42553dc"
dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
 from {username='slave'} at 127.0.0.1:54992
I20260709 13:04:51.579473 12464 raft_consensus.cc:493] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:51.579879 12464 raft_consensus.cc:3060] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.587199 12464 raft_consensus.cc:515] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.588994 12464 leader_election.cc:290] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:51.589751 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887bbd0f3df74e8c890d6025d42553dc" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:51.590075 12537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "887bbd0f3df74e8c890d6025d42553dc" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:51.590466 12390 raft_consensus.cc:3060] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.590790 12537 raft_consensus.cc:3055] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:51.591131 12537 raft_consensus.cc:740] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:51.591876 12537 consensus_queue.cc:260] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.592902 12537 raft_consensus.cc:3060] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.595708 12390 raft_consensus.cc:2468] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 2.
I20260709 13:04:51.596880 12430 leader_election.cc:304] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:51.597571 12695 raft_consensus.cc:2804] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:51.598013 12695 raft_consensus.cc:697] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:51.598482 12537 raft_consensus.cc:2468] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 2.
I20260709 13:04:51.598747 12695 consensus_queue.cc:237] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.606520 12288 catalog_manager.cc:5697] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 1 to 2, leader changed from a0184626ff924a26959a272550a00c05 (127.0.107.195) to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.668444 12665 raft_consensus.cc:993] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05: : Instructing follower e82db363f6484aa3be77e60927c02fa9 to start an election
I20260709 13:04:51.669062 12744 raft_consensus.cc:1081] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Signalling peer e82db363f6484aa3be77e60927c02fa9 to start an election
I20260709 13:04:51.670689 12464 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b319df053e54d6dbaaf1e3f0eeb75b9"
dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
 from {username='slave'} at 127.0.0.1:54992
I20260709 13:04:51.671293 12464 raft_consensus.cc:493] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:51.671689 12464 raft_consensus.cc:3060] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.677091 12464 raft_consensus.cc:515] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.678831 12464 leader_election.cc:290] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:51.679755 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b319df053e54d6dbaaf1e3f0eeb75b9" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:51.680094 12537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b319df053e54d6dbaaf1e3f0eeb75b9" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:51.680465 12390 raft_consensus.cc:3060] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.680727 12537 raft_consensus.cc:3055] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:51.681074 12537 raft_consensus.cc:740] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:51.681952 12537 consensus_queue.cc:260] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.683281 12537 raft_consensus.cc:3060] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.687932 12390 raft_consensus.cc:2468] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 2.
I20260709 13:04:51.689251 12430 leader_election.cc:304] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:51.690135 12695 raft_consensus.cc:2804] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:51.690569 12695 raft_consensus.cc:697] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:51.690737 12537 raft_consensus.cc:2468] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 2.
I20260709 13:04:51.691327 12695 consensus_queue.cc:237] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.698033 12288 catalog_manager.cc:5697] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 1 to 2, leader changed from a0184626ff924a26959a272550a00c05 (127.0.107.195) to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.702461 12771 raft_consensus.cc:993] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9: : Instructing follower a0184626ff924a26959a272550a00c05 to start an election
I20260709 13:04:51.703094 12695 raft_consensus.cc:1081] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Signalling peer a0184626ff924a26959a272550a00c05 to start an election
I20260709 13:04:51.704897 12539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6bf95d8655624d84abcfb16f84c47bfd"
dest_uuid: "a0184626ff924a26959a272550a00c05"
 from {username='slave'} at 127.0.0.1:49538
I20260709 13:04:51.705490 12539 raft_consensus.cc:493] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:51.705845 12539 raft_consensus.cc:3060] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.710819 12665 raft_consensus.cc:993] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05: : Instructing follower e82db363f6484aa3be77e60927c02fa9 to start an election
I20260709 13:04:51.711380 12665 raft_consensus.cc:1081] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Signalling peer e82db363f6484aa3be77e60927c02fa9 to start an election
I20260709 13:04:51.712255 12539 raft_consensus.cc:515] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.713022 12464 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8843d3f9e08d42a480f5d335516ab427"
dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
 from {username='slave'} at 127.0.0.1:54992
I20260709 13:04:51.713616 12464 raft_consensus.cc:493] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:51.713945 12464 raft_consensus.cc:3060] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.714514 12539 leader_election.cc:290] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:51.715392 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf95d8655624d84abcfb16f84c47bfd" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:51.716475 12390 raft_consensus.cc:3060] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.716950 12462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bf95d8655624d84abcfb16f84c47bfd" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:51.717567 12462 raft_consensus.cc:3055] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:51.717840 12462 raft_consensus.cc:740] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:51.718606 12462 consensus_queue.cc:260] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.719619 12462 raft_consensus.cc:3060] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.721371 12464 raft_consensus.cc:515] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.723707 12464 leader_election.cc:290] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:51.723912 12390 raft_consensus.cc:2468] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 2.
I20260709 13:04:51.725055 12504 leader_election.cc:304] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05; no voters: 
I20260709 13:04:51.725687 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8843d3f9e08d42a480f5d335516ab427" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:51.725860 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8843d3f9e08d42a480f5d335516ab427" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:51.726331 12390 raft_consensus.cc:3060] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.725943 12665 raft_consensus.cc:2804] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:51.726915 12539 raft_consensus.cc:3055] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:51.726996 12665 raft_consensus.cc:697] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [term 2 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:51.727402 12539 raft_consensus.cc:740] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:51.728013 12665 consensus_queue.cc:237] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.728482 12539 consensus_queue.cc:260] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.730057 12539 raft_consensus.cc:3060] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:51.733173 12390 raft_consensus.cc:2468] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 2.
I20260709 13:04:51.734346 12430 leader_election.cc:304] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:51.735114 12695 raft_consensus.cc:2804] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:51.725752 12462 raft_consensus.cc:2468] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 2.
I20260709 13:04:51.735647 12695 raft_consensus.cc:697] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:51.737083 12695 consensus_queue.cc:237] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:51.738201 12288 catalog_manager.cc:5697] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 1 to 2, leader changed from e82db363f6484aa3be77e60927c02fa9 (127.0.107.194) to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:51.739938 12539 raft_consensus.cc:2468] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 2.
I20260709 13:04:51.747027 12286 catalog_manager.cc:5697] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 1 to 2, leader changed from a0184626ff924a26959a272550a00c05 (127.0.107.195) to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:51.966711 12539 raft_consensus.cc:1275] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Refusing update from remote peer 9b85e1f979194d6889a0490e970ebfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:51.968119 12755 consensus_queue.cc:1048] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:51.977770 12539 raft_consensus.cc:1275] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Refusing update from remote peer 9b85e1f979194d6889a0490e970ebfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:51.979832 12755 consensus_queue.cc:1048] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:51.983438 12462 raft_consensus.cc:1275] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Refusing update from remote peer 9b85e1f979194d6889a0490e970ebfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:51.985886 12746 consensus_queue.cc:1048] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.000543 12464 raft_consensus.cc:1275] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Refusing update from remote peer 9b85e1f979194d6889a0490e970ebfde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.003007 12721 consensus_queue.cc:1048] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:52.045212 12560 debug-util.cc:398] Leaking SignalData structure 0x7b08001b3f20 after lost signal to thread 434
W20260709 13:04:52.046403 12560 debug-util.cc:398] Leaking SignalData structure 0x7b080031e240 after lost signal to thread 12322
W20260709 13:04:52.047508 12560 debug-util.cc:398] Leaking SignalData structure 0x7b08003825e0 after lost signal to thread 12414
W20260709 13:04:52.048496 12560 debug-util.cc:398] Leaking SignalData structure 0x7b08000edfe0 after lost signal to thread 12489
I20260709 13:04:52.125890 12390 raft_consensus.cc:1275] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Refusing update from remote peer e82db363f6484aa3be77e60927c02fa9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.128456 12771 consensus_queue.cc:1048] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.133929 12539 raft_consensus.cc:1275] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Refusing update from remote peer e82db363f6484aa3be77e60927c02fa9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.136493 12740 consensus_queue.cc:1048] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.140722 12537 raft_consensus.cc:1275] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Refusing update from remote peer e82db363f6484aa3be77e60927c02fa9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.146435 12771 consensus_queue.cc:1048] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.158964 12390 raft_consensus.cc:1275] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.160259 12386 raft_consensus.cc:1275] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Refusing update from remote peer e82db363f6484aa3be77e60927c02fa9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.160528 12744 consensus_queue.cc:1048] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:04:52.162523 12695 consensus_queue.cc:1048] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.178534 12462 raft_consensus.cc:1275] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.180608 12753 consensus_queue.cc:1048] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.203905 12537 raft_consensus.cc:1275] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Refusing update from remote peer e82db363f6484aa3be77e60927c02fa9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.205130 12695 consensus_queue.cc:1048] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:52.223047 12390 raft_consensus.cc:1275] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Refusing update from remote peer e82db363f6484aa3be77e60927c02fa9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:52.224915 12771 consensus_queue.cc:1048] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:54.370107 12288 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48406:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:04:54.398326 12288 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:48406:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:04:54.529562   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 13:04:54.557771 12800 log.cc:927] Time spent T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde: Append to log took a long time: real 0.056s	user 0.004s	sys 0.005s
W20260709 13:04:54.582850 12812 log.cc:927] Time spent T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:04:54.716379 12832 tablet.cc:1724] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:54.740546 12834 tablet.cc:1724] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:54.719556 12839 tablet.cc:1724] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:54.798461 12844 tablet.cc:1724] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:04:54.880198 12430 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:36193, user_credentials={real_user=slave}} blocked reactor thread for 54947.5us
W20260709 13:04:54.913863 12840 log.cc:927] Time spent T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05: Append to log took a long time: real 0.091s	user 0.001s	sys 0.010s
W20260709 13:04:54.967538 12837 log.cc:927] Time spent T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05: Append to log took a long time: real 0.133s	user 0.000s	sys 0.009s
I20260709 13:04:54.968693 12843 tablet.cc:1724] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.011858 12850 tablet.cc:1724] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:04:55.014742 12842 log.cc:927] Time spent T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:04:55.016144 12847 tablet.cc:1724] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.018376 12849 tablet.cc:1724] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.024421 12846 tablet.cc:1724] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.037000 12834 tablet.cc:1724] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.038040 12852 tablet.cc:1724] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:04:55.041839 12411 debug-util.cc:398] Leaking SignalData structure 0x7b080013ddc0 after lost signal to thread 434
W20260709 13:04:55.043066 12411 debug-util.cc:398] Leaking SignalData structure 0x7b08002104a0 after lost signal to thread 12322
W20260709 13:04:55.044519 12411 debug-util.cc:398] Leaking SignalData structure 0x7b080012c5a0 after lost signal to thread 12414
W20260709 13:04:55.046078 12411 debug-util.cc:398] Leaking SignalData structure 0x7b08001c9840 after lost signal to thread 12489
I20260709 13:04:55.046510 12832 tablet.cc:1724] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:04:55.047344 12411 debug-util.cc:398] Leaking SignalData structure 0x7b08002bdb80 after lost signal to thread 12563
I20260709 13:04:55.025156 12851 tablet.cc:1724] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:04:55.053859 12355 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:36193, user_credentials={real_user=slave}} blocked reactor thread for 289974us
I20260709 13:04:55.056919 12839 tablet.cc:1724] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.069584 12854 tablet.cc:1724] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.076956 12843 tablet.cc:1724] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.072474 12464 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b319df053e54d6dbaaf1e3f0eeb75b9"
dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
mode: GRACEFUL
new_leader_uuid: "a0184626ff924a26959a272550a00c05"
 from {username='slave'} at 127.0.0.1:59434
I20260709 13:04:55.083076 12464 raft_consensus.cc:606] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Received request to transfer leadership to TS a0184626ff924a26959a272550a00c05
W20260709 13:04:55.076359 12841 log.cc:927] Time spent T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 13:04:55.140291 12844 tablet.cc:1724] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.145838 12815 raft_consensus.cc:993] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9: : Instructing follower a0184626ff924a26959a272550a00c05 to start an election
I20260709 13:04:55.146767 12808 raft_consensus.cc:1081] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Signalling peer a0184626ff924a26959a272550a00c05 to start an election
W20260709 13:04:55.148128 12274 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.AlterSchema -> {remote=127.0.107.193:46243, user_credentials={real_user=slave}} blocked reactor thread for 46391.2us
W20260709 13:04:55.161625 12505 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:35975, user_credentials={real_user=slave}} blocked reactor thread for 59512.6us
I20260709 13:04:55.164157 12832 tablet.cc:1724] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.176401 12847 tablet.cc:1724] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.179540 12288 catalog_manager.cc:5789] TS 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243) does not have the latest schema for tablet 2f5b5c9972e84588bc82910949c271d5 (table soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]). Expected version 1 got 0
I20260709 13:04:55.180892 12288 catalog_manager.cc:5789] TS 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243) does not have the latest schema for tablet 81c4673118cf4c2ab6f3a59d06d8ebc3 (table soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]). Expected version 1 got 0
I20260709 13:04:55.181411 12390 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "81c4673118cf4c2ab6f3a59d06d8ebc3"
dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
mode: GRACEFUL
new_leader_uuid: "a0184626ff924a26959a272550a00c05"
 from {username='slave'} at 127.0.0.1:59256
I20260709 13:04:55.182108 12390 raft_consensus.cc:606] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 2 LEADER]: Received request to transfer leadership to TS a0184626ff924a26959a272550a00c05
I20260709 13:04:55.183599   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:04:55.184255   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:55.184916 12288 catalog_manager.cc:5789] TS 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243) does not have the latest schema for tablet 518f8726e0ee4660b8c8f05c648f05a4 (table soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663]). Expected version 1 got 0
I20260709 13:04:55.190603 12539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b319df053e54d6dbaaf1e3f0eeb75b9"
dest_uuid: "a0184626ff924a26959a272550a00c05"
 from {username='slave'} at 127.0.0.1:49538
I20260709 13:04:55.191211 12539 raft_consensus.cc:493] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:55.191546 12539 raft_consensus.cc:3060] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.179554 12846 tablet.cc:1724] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.198519 12539 raft_consensus.cc:515] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:55.198879 12844 tablet.cc:1724] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.201045 12539 leader_election.cc:290] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:55.201752 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b319df053e54d6dbaaf1e3f0eeb75b9" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:55.202418 12464 raft_consensus.cc:3055] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:55.202731 12464 raft_consensus.cc:740] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:55.203572 12385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c059adc4ffb476596307903819bb1c0"
dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
mode: GRACEFUL
new_leader_uuid: "a0184626ff924a26959a272550a00c05"
 from {username='slave'} at 127.0.0.1:59256
I20260709 13:04:55.204281 12385 raft_consensus.cc:606] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Received request to transfer leadership to TS a0184626ff924a26959a272550a00c05
I20260709 13:04:55.205495 12464 consensus_queue.cc:260] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:55.207329 12465 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1"
dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
mode: GRACEFUL
new_leader_uuid: "a0184626ff924a26959a272550a00c05"
 from {username='slave'} at 127.0.0.1:59434
I20260709 13:04:55.207983 12465 raft_consensus.cc:606] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Received request to transfer leadership to TS a0184626ff924a26959a272550a00c05
I20260709 13:04:55.207597 12854 tablet.cc:1724] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.209853   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:04:55.210561   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:55.210671 12852 tablet.cc:1724] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.212306 12850 tablet.cc:1724] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.206615 12464 raft_consensus.cc:3060] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.230042 12385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b319df053e54d6dbaaf1e3f0eeb75b9" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:55.229887 12851 tablet.cc:1724] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.230837 12385 raft_consensus.cc:3060] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.232478 12849 tablet.cc:1724] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.246423 12845 raft_consensus.cc:993] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde: : Instructing follower a0184626ff924a26959a272550a00c05 to start an election
I20260709 13:04:55.246927 12845 raft_consensus.cc:1081] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Signalling peer a0184626ff924a26959a272550a00c05 to start an election
I20260709 13:04:55.248943 12539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c059adc4ffb476596307903819bb1c0"
dest_uuid: "a0184626ff924a26959a272550a00c05"
 from {username='slave'} at 127.0.0.1:49526
I20260709 13:04:55.249524 12539 raft_consensus.cc:493] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:55.249863 12539 raft_consensus.cc:3060] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:55.257983 12539 raft_consensus.cc:515] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } }
I20260709 13:04:55.260591 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c059adc4ffb476596307903819bb1c0" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:55.261147 12465 raft_consensus.cc:3060] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:55.267515 12390 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c059adc4ffb476596307903819bb1c0" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:55.267874 12465 raft_consensus.cc:2468] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 2.
I20260709 13:04:55.268205 12390 raft_consensus.cc:3055] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:04:55.268680 12390 raft_consensus.cc:740] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
I20260709 13:04:55.269752 12505 leader_election.cc:304] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:55.270648 12744 raft_consensus.cc:2804] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:55.270861 12390 consensus_queue.cc:260] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } }
I20260709 13:04:55.271392 12744 raft_consensus.cc:697] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [term 2 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:55.272056 12390 raft_consensus.cc:3060] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:55.272526 12744 consensus_queue.cc:237] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } }
I20260709 13:04:55.277593 12539 leader_election.cc:290] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 election: Requested vote from peers e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975), 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243)
I20260709 13:04:55.280613 12390 raft_consensus.cc:2468] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 2.
I20260709 13:04:55.296016 12854 tablet.cc:1724] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:04:55.301834 12385 raft_consensus.cc:2468] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 3.
I20260709 13:04:55.303653 12504 leader_election.cc:304] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05; no voters: 
I20260709 13:04:55.305086 12848 raft_consensus.cc:2804] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:55.310390 12848 raft_consensus.cc:697] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 3 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:55.311371 12848 consensus_queue.cc:237] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:55.325537 12464 raft_consensus.cc:2468] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 3.
I20260709 13:04:55.344413 12288 catalog_manager.cc:6242] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5: soft_deleted_table [id=d3e63e7de39e4d86965f258deaa6a663] alter complete (version 1)
I20260709 13:04:55.359854 12284 catalog_manager.cc:5697] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 1 to 2, leader changed from 9b85e1f979194d6889a0490e970ebfde (127.0.107.193) to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:55.361936 12284 catalog_manager.cc:5697] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 2 to 3, leader changed from e82db363f6484aa3be77e60927c02fa9 (127.0.107.194) to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:55.477350 12771 raft_consensus.cc:993] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9: : Instructing follower a0184626ff924a26959a272550a00c05 to start an election
I20260709 13:04:55.477864 12771 raft_consensus.cc:1081] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Signalling peer a0184626ff924a26959a272550a00c05 to start an election
I20260709 13:04:55.479966 12539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1"
dest_uuid: "a0184626ff924a26959a272550a00c05"
 from {username='slave'} at 127.0.0.1:49538
I20260709 13:04:55.480548 12539 raft_consensus.cc:493] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:55.480918 12539 raft_consensus.cc:3060] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.506093 12539 raft_consensus.cc:515] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:55.509385 12385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:55.510124 12385 raft_consensus.cc:3060] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.518874 12385 raft_consensus.cc:2468] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 3.
I20260709 13:04:55.525713 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee78aec6998e4a29a17f470c6d8f0cd1" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:55.526909 12464 raft_consensus.cc:3055] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:55.527220 12464 raft_consensus.cc:740] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:55.528043 12464 consensus_queue.cc:260] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:55.529219 12464 raft_consensus.cc:3060] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.526295 12539 leader_election.cc:290] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:55.544425 12504 leader_election.cc:304] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05; no voters: 
I20260709 13:04:55.545476 12744 raft_consensus.cc:2804] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:55.546007 12744 raft_consensus.cc:697] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 3 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:55.546989 12744 consensus_queue.cc:237] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:55.565153 12464 raft_consensus.cc:2468] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 3.
I20260709 13:04:55.586706 12284 catalog_manager.cc:5697] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 2 to 3, leader changed from e82db363f6484aa3be77e60927c02fa9 (127.0.107.194) to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:55.636960 12845 raft_consensus.cc:993] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde: : Instructing follower a0184626ff924a26959a272550a00c05 to start an election
I20260709 13:04:55.641723 12818 raft_consensus.cc:1081] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 2 LEADER]: Signalling peer a0184626ff924a26959a272550a00c05 to start an election
I20260709 13:04:55.643883 12539 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "81c4673118cf4c2ab6f3a59d06d8ebc3"
dest_uuid: "a0184626ff924a26959a272550a00c05"
 from {username='slave'} at 127.0.0.1:49526
I20260709 13:04:55.644500 12539 raft_consensus.cc:493] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:04:55.644870 12539 raft_consensus.cc:3060] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.653144 12539 raft_consensus.cc:515] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:55.655256 12539 leader_election.cc:290] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:55.657094 12464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81c4673118cf4c2ab6f3a59d06d8ebc3" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:55.658211 12464 raft_consensus.cc:3060] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.663246 12385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81c4673118cf4c2ab6f3a59d06d8ebc3" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "9b85e1f979194d6889a0490e970ebfde"
I20260709 13:04:55.663547 12390 raft_consensus.cc:1275] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:55.664064 12385 raft_consensus.cc:3055] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:04:55.664462 12385 raft_consensus.cc:740] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9b85e1f979194d6889a0490e970ebfde, State: Running, Role: LEADER
I20260709 13:04:55.665781 12744 consensus_queue.cc:1048] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:04:55.672960 12464 raft_consensus.cc:2468] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 3.
I20260709 13:04:55.665381 12385 consensus_queue.cc:260] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:55.675233 12385 raft_consensus.cc:3060] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:55.677173 12505 leader_election.cc:304] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:55.678390 12848 raft_consensus.cc:2804] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:55.679286 12848 raft_consensus.cc:697] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 3 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:55.680269 12848 consensus_queue.cc:237] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:55.686846 12385 raft_consensus.cc:2468] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 3.
I20260709 13:04:55.716398 12288 catalog_manager.cc:5697] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 2 to 3, leader changed from 9b85e1f979194d6889a0490e970ebfde (127.0.107.193) to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:55.749795 12464 raft_consensus.cc:1275] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:55.756189 12848 consensus_queue.cc:1048] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:04:55.829656 12464 raft_consensus.cc:1275] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:55.831367 12848 consensus_queue.cc:1048] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:55.842716 12390 raft_consensus.cc:1275] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 3 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:55.844658 12744 consensus_queue.cc:1048] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:55.955856 12390 raft_consensus.cc:1275] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 3 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:55.957785 12744 consensus_queue.cc:1048] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:55.976240 12464 raft_consensus.cc:1275] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:04:55.979708 12853 consensus_queue.cc:1048] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:56.150900 12464 raft_consensus.cc:1275] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:56.152223 12744 consensus_queue.cc:1048] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:56.172179 12390 raft_consensus.cc:1275] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 3 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:56.174180 12848 consensus_queue.cc:1048] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:04:56.290226   431 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:04:56.298808   431 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:04:56.299436   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:04:56.307387   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:04:56.308038   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:04:57.341233   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
W20260709 13:04:57.355332 12430 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
W20260709 13:04:57.362746 12430 consensus_peers.cc:597] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:57.379752   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:57.380730   431 tablet_replica.cc:333] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.381350   431 raft_consensus.cc:2243] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.382340   431 raft_consensus.cc:2272] T aff0923e6e974f478093276d83c0a3e5 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.384294   431 tablet_replica.cc:333] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.384907   431 raft_consensus.cc:2243] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.385360   431 raft_consensus.cc:2272] T 6bf95d8655624d84abcfb16f84c47bfd P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.387481   431 tablet_replica.cc:333] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.388217   431 raft_consensus.cc:2243] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.388744   431 raft_consensus.cc:2272] T 8843d3f9e08d42a480f5d335516ab427 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.391203   431 tablet_replica.cc:333] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.391976   431 raft_consensus.cc:2243] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.392982   431 raft_consensus.cc:2272] T 2f5b5c9972e84588bc82910949c271d5 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.395545   431 tablet_replica.cc:333] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.396311   431 raft_consensus.cc:2243] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.396828   431 raft_consensus.cc:2272] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P 9b85e1f979194d6889a0490e970ebfde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.399264   431 tablet_replica.cc:333] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.400149   431 raft_consensus.cc:2243] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.400658   431 raft_consensus.cc:2272] T 2c059adc4ffb476596307903819bb1c0 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.403729   431 tablet_replica.cc:333] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.404520   431 raft_consensus.cc:2243] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.405570   431 raft_consensus.cc:2272] T 518f8726e0ee4660b8c8f05c648f05a4 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.408174   431 tablet_replica.cc:333] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.408964   431 raft_consensus.cc:2243] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.409574   431 raft_consensus.cc:2272] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P 9b85e1f979194d6889a0490e970ebfde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.412014   431 tablet_replica.cc:333] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.412626   431 raft_consensus.cc:2243] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.413096   431 raft_consensus.cc:2272] T 887bbd0f3df74e8c890d6025d42553dc P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.415066   431 tablet_replica.cc:333] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.415694   431 raft_consensus.cc:2243] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.416564   431 raft_consensus.cc:2272] T 5ead0cc836794861a8c8c15929eb6e3a P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.418946   431 tablet_replica.cc:333] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.419516   431 raft_consensus.cc:2243] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.420037   431 raft_consensus.cc:2272] T 97cb4f0685fa44f79e19ae9b9eabf3fa P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.422715   431 tablet_replica.cc:333] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.423393   431 raft_consensus.cc:2243] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.423956   431 raft_consensus.cc:2272] T b2723322211a40479ee5ef03f2d6b1ef P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.427047   431 tablet_replica.cc:333] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.427666   431 raft_consensus.cc:2243] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.428153   431 raft_consensus.cc:2272] T 9fa195d27155441086c43051a0eeaa78 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:57.428256 12504 consensus_peers.cc:597] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:57.430435   431 tablet_replica.cc:333] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.431298   431 raft_consensus.cc:2243] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.431792   431 raft_consensus.cc:2272] T 5a44c20b12ff46099f11c28404c1d89b P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.434882   431 tablet_replica.cc:333] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.435530   431 raft_consensus.cc:2243] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.436021   431 raft_consensus.cc:2272] T ee78aec6998e4a29a17f470c6d8f0cd1 P 9b85e1f979194d6889a0490e970ebfde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.438325   431 tablet_replica.cc:333] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.438969   431 raft_consensus.cc:2243] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.439805   431 raft_consensus.cc:2272] T 679e72f804654636aca1ad4dc51900c6 P 9b85e1f979194d6889a0490e970ebfde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.442165   431 tablet_replica.cc:333] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.442675   431 raft_consensus.cc:2243] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:57.443142   431 raft_consensus.cc:2272] T 66dc07eb98b744208bc1ea38b3b814ad P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:57.445520   431 tablet_replica.cc:333] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde: stopping tablet replica
I20260709 13:04:57.445998   431 raft_consensus.cc:2243] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:57.446834   431 raft_consensus.cc:2272] T 75504590d7e64c50bbb8f946ea1fc1c3 P 9b85e1f979194d6889a0490e970ebfde [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:57.466985 12430 consensus_peers.cc:597] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.476622 12504 consensus_peers.cc:597] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.521637 12430 consensus_peers.cc:597] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.559536 12504 consensus_peers.cc:597] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.582418 12430 consensus_peers.cc:597] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.623912 12504 consensus_peers.cc:597] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.626914 12430 consensus_peers.cc:597] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.646040 12430 consensus_peers.cc:597] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.653111 12504 consensus_peers.cc:597] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:57.790194 12504 consensus_peers.cc:597] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:58.352311 12881 raft_consensus.cc:493] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.352837 12881 raft_consensus.cc:515] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.354825 12881 leader_election.cc:290] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:58.356540 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff0923e6e974f478093276d83c0a3e5" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
W20260709 13:04:58.358613 12430 leader_election.cc:336] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.359001 12430 leader_election.cc:304] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05
I20260709 13:04:58.359591 12881 raft_consensus.cc:2749] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:58.429322 12881 raft_consensus.cc:493] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.429895 12881 raft_consensus.cc:515] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.431675 12881 leader_election.cc:290] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:58.433368 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75504590d7e64c50bbb8f946ea1fc1c3" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
W20260709 13:04:58.435506 12430 leader_election.cc:336] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.435973 12430 leader_election.cc:304] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05
I20260709 13:04:58.436699 12881 raft_consensus.cc:2749] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:04:58.492144   431 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 12411)
I20260709 13:04:58.503320 12878 raft_consensus.cc:493] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.504060 12878 raft_consensus.cc:515] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.506454 12878 leader_election.cc:290] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:58.507712 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518f8726e0ee4660b8c8f05c648f05a4" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
W20260709 13:04:58.510694 12504 leader_election.cc:336] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.511078 12504 leader_election.cc:304] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0184626ff924a26959a272550a00c05; no voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:58.511704 12878 raft_consensus.cc:2749] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:58.570511 12878 raft_consensus.cc:493] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.570601 12881 raft_consensus.cc:493] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.572587 12878 raft_consensus.cc:515] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.573963 12881 raft_consensus.cc:515] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.576570 12878 leader_election.cc:290] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:58.578012 12881 leader_election.cc:290] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:58.579643 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff0923e6e974f478093276d83c0a3e5" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:58.581009 12465 raft_consensus.cc:2468] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 2.
I20260709 13:04:58.583547 12505 leader_election.cc:304] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:58.583781 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5b5c9972e84588bc82910949c271d5" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:58.585482 12878 raft_consensus.cc:2804] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:04:58.584601 12504 leader_election.cc:336] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.586062 12878 raft_consensus.cc:493] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.588147 12878 raft_consensus.cc:3060] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:58.588667 12430 leader_election.cc:336] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.589511 12430 leader_election.cc:304] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05
I20260709 13:04:58.590590 12881 raft_consensus.cc:2749] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:58.604929 12878 raft_consensus.cc:515] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.607996 12878 leader_election.cc:290] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:58.609400 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aff0923e6e974f478093276d83c0a3e5" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:58.610240 12465 raft_consensus.cc:3060] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:58.614068 12504 leader_election.cc:336] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.623169 12465 raft_consensus.cc:2468] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 3.
I20260709 13:04:58.624992 12505 leader_election.cc:304] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:58.626039 12878 raft_consensus.cc:2804] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:58.626704 12878 raft_consensus.cc:697] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 3 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:58.628149 12878 consensus_queue.cc:237] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.643196 12288 catalog_manager.cc:5697] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 2 to 3, leader changed from 9b85e1f979194d6889a0490e970ebfde (127.0.107.193) to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:58.647269 12878 raft_consensus.cc:493] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.648384 12878 raft_consensus.cc:515] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.653625 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ead0cc836794861a8c8c15929eb6e3a" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:58.655834 12878 leader_election.cc:290] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
W20260709 13:04:58.659662 12504 leader_election.cc:336] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.660053 12504 leader_election.cc:304] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0184626ff924a26959a272550a00c05; no voters: 9b85e1f979194d6889a0490e970ebfde, e82db363f6484aa3be77e60927c02fa9
I20260709 13:04:58.660730 12878 raft_consensus.cc:2749] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:04:58.680683 12881 raft_consensus.cc:493] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.681622 12881 raft_consensus.cc:515] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.683648 12881 leader_election.cc:290] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:58.685411 12539 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518f8726e0ee4660b8c8f05c648f05a4" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:58.686364 12539 raft_consensus.cc:2468] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 1.
I20260709 13:04:58.687543 12430 leader_election.cc:304] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
W20260709 13:04:58.688304 12430 leader_election.cc:336] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.688441 12881 raft_consensus.cc:2804] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:58.688992 12881 raft_consensus.cc:493] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.689445 12881 raft_consensus.cc:3060] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:58.701547 12881 raft_consensus.cc:515] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.704202 12881 leader_election.cc:290] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:58.707768 12535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "518f8726e0ee4660b8c8f05c648f05a4" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:58.708634 12535 raft_consensus.cc:3060] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:58.710803 12430 leader_election.cc:336] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.716650 12535 raft_consensus.cc:2468] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 2.
I20260709 13:04:58.718191 12430 leader_election.cc:304] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:58.718318 12889 raft_consensus.cc:493] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.719029 12873 raft_consensus.cc:2804] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:58.719197 12889 raft_consensus.cc:515] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.719657 12873 raft_consensus.cc:697] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:58.721530 12873 consensus_queue.cc:237] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.722012 12889 leader_election.cc:290] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:58.723518 12535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679e72f804654636aca1ad4dc51900c6" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
W20260709 13:04:58.727591 12430 leader_election.cc:336] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.728081 12430 leader_election.cc:304] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde, a0184626ff924a26959a272550a00c05
I20260709 13:04:58.728929 12889 raft_consensus.cc:2749] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:04:58.740483 12288 catalog_manager.cc:5697] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 1 to 2, leader changed from 9b85e1f979194d6889a0490e970ebfde (127.0.107.193) to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:58.768452 12879 raft_consensus.cc:493] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.769268 12879 raft_consensus.cc:515] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.772176 12879 leader_election.cc:290] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:58.773483 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5b5c9972e84588bc82910949c271d5" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:58.774227 12465 raft_consensus.cc:2468] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 1.
I20260709 13:04:58.775621 12505 leader_election.cc:304] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
W20260709 13:04:58.776309 12504 leader_election.cc:336] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.776592 12879 raft_consensus.cc:2804] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:58.777263 12879 raft_consensus.cc:493] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.777681 12879 raft_consensus.cc:3060] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:58.788198 12879 raft_consensus.cc:515] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.790635 12879 leader_election.cc:290] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:58.791898 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f5b5c9972e84588bc82910949c271d5" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:58.792574 12465 raft_consensus.cc:3060] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:58.794456 12504 leader_election.cc:336] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.800526 12879 raft_consensus.cc:493] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.800972 12465 raft_consensus.cc:2468] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 2.
I20260709 13:04:58.801260 12879 raft_consensus.cc:515] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.802387 12505 leader_election.cc:304] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:58.803396 12879 leader_election.cc:290] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:58.804004 12879 raft_consensus.cc:2804] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:58.804471 12879 raft_consensus.cc:697] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 2 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:58.804971 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679e72f804654636aca1ad4dc51900c6" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
I20260709 13:04:58.805748 12465 raft_consensus.cc:2468] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 1.
I20260709 13:04:58.805644 12879 consensus_queue.cc:237] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.807060 12505 leader_election.cc:304] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
I20260709 13:04:58.808063 12878 raft_consensus.cc:2804] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:04:58.808472 12878 raft_consensus.cc:493] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.808908 12878 raft_consensus.cc:3060] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:04:58.809257 12504 leader_election.cc:336] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.818557 12878 raft_consensus.cc:515] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.820210 12288 catalog_manager.cc:5697] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 1 to 2, leader changed from 9b85e1f979194d6889a0490e970ebfde (127.0.107.193) to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:58.824630 12873 raft_consensus.cc:493] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.825371 12873 raft_consensus.cc:515] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
W20260709 13:04:58.825984 12504 leader_election.cc:336] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.826761 12878 leader_election.cc:290] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:58.827605 12873 leader_election.cc:290] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:58.827687 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "679e72f804654636aca1ad4dc51900c6" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:58.830966 12535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ead0cc836794861a8c8c15929eb6e3a" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05" is_pre_election: true
I20260709 13:04:58.831676 12535 raft_consensus.cc:2468] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 2.
I20260709 13:04:58.828501 12465 raft_consensus.cc:3060] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:04:58.833243 12430 leader_election.cc:304] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 
W20260709 13:04:58.835300 12430 leader_election.cc:336] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.836248 12873 raft_consensus.cc:2804] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:58.836707 12873 raft_consensus.cc:493] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.837090 12873 raft_consensus.cc:3060] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:58.845346 12465 raft_consensus.cc:2468] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 2.
I20260709 13:04:58.846680 12505 leader_election.cc:304] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:58.846722 12873 raft_consensus.cc:515] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.847693 12878 raft_consensus.cc:2804] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:04:58.848219 12878 raft_consensus.cc:697] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 2 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:58.849139 12873 leader_election.cc:290] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 3 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), a0184626ff924a26959a272550a00c05 (127.0.107.195:36193)
I20260709 13:04:58.849795 12878 consensus_queue.cc:237] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.851432 12535 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ead0cc836794861a8c8c15929eb6e3a" candidate_uuid: "e82db363f6484aa3be77e60927c02fa9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a0184626ff924a26959a272550a00c05"
I20260709 13:04:58.852087 12535 raft_consensus.cc:3060] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:58.855014 12430 leader_election.cc:336] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.861500 12535 raft_consensus.cc:2468] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e82db363f6484aa3be77e60927c02fa9 in term 3.
I20260709 13:04:58.863153 12430 leader_election.cc:304] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:58.862628 12288 catalog_manager.cc:5697] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 1 to 2, leader changed from 9b85e1f979194d6889a0490e970ebfde (127.0.107.193) to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:58.864073 12873 raft_consensus.cc:2804] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:58.864550 12873 raft_consensus.cc:697] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 3 LEADER]: Becoming Leader. State: Replica: e82db363f6484aa3be77e60927c02fa9, State: Running, Role: LEADER
I20260709 13:04:58.865643 12873 consensus_queue.cc:237] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.872304 12878 raft_consensus.cc:493] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.873332 12878 raft_consensus.cc:515] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.880601 12878 leader_election.cc:290] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:58.881060 12286 catalog_manager.cc:5697] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 reported cstate change: term changed from 2 to 3, leader changed from 9b85e1f979194d6889a0490e970ebfde (127.0.107.193) to e82db363f6484aa3be77e60927c02fa9 (127.0.107.194). New cstate: current_term: 3 leader_uuid: "e82db363f6484aa3be77e60927c02fa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: UNKNOWN } } }
I20260709 13:04:58.882035 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75504590d7e64c50bbb8f946ea1fc1c3" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9" is_pre_election: true
W20260709 13:04:58.882682 12504 leader_election.cc:336] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.883045 12465 raft_consensus.cc:2468] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 2.
I20260709 13:04:58.884724 12505 leader_election.cc:304] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:58.885634 12878 raft_consensus.cc:2804] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:04:58.886082 12878 raft_consensus.cc:493] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9b85e1f979194d6889a0490e970ebfde)
I20260709 13:04:58.886508 12878 raft_consensus.cc:3060] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:04:58.894253 12878 raft_consensus.cc:515] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.896047 12878 leader_election.cc:290] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 election: Requested vote from peers 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243), e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975)
I20260709 13:04:58.897611 12465 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "75504590d7e64c50bbb8f946ea1fc1c3" candidate_uuid: "a0184626ff924a26959a272550a00c05" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "e82db363f6484aa3be77e60927c02fa9"
I20260709 13:04:58.898394 12465 raft_consensus.cc:3060] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:04:58.900821 12504 leader_election.cc:336] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111)
I20260709 13:04:58.904623 12465 raft_consensus.cc:2468] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0184626ff924a26959a272550a00c05 in term 3.
I20260709 13:04:58.905912 12505 leader_election.cc:304] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0184626ff924a26959a272550a00c05, e82db363f6484aa3be77e60927c02fa9; no voters: 9b85e1f979194d6889a0490e970ebfde
I20260709 13:04:58.906878 12878 raft_consensus.cc:2804] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:04:58.907420 12878 raft_consensus.cc:697] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 3 LEADER]: Becoming Leader. State: Replica: a0184626ff924a26959a272550a00c05, State: Running, Role: LEADER
I20260709 13:04:58.908707 12878 consensus_queue.cc:237] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } }
I20260709 13:04:58.917550 12286 catalog_manager.cc:5697] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 reported cstate change: term changed from 2 to 3, leader changed from 9b85e1f979194d6889a0490e970ebfde (127.0.107.193) to a0184626ff924a26959a272550a00c05 (127.0.107.195). New cstate: current_term: 3 leader_uuid: "a0184626ff924a26959a272550a00c05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9b85e1f979194d6889a0490e970ebfde" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 46243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 } health_report { overall_health: HEALTHY } } }
I20260709 13:04:59.059084 12464 raft_consensus.cc:1275] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:04:59.060559 12878 consensus_queue.cc:1048] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:04:59.085325 12504 consensus_peers.cc:597] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.158993 12535 raft_consensus.cc:1275] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Refusing update from remote peer e82db363f6484aa3be77e60927c02fa9: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:59.160513 12873 consensus_queue.cc:1048] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0184626ff924a26959a272550a00c05" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:59.176560   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
W20260709 13:04:59.181921 12430 consensus_peers.cc:597] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.214792 12504 consensus_peers.cc:597] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.216730 12464 raft_consensus.cc:1275] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:04:59.218854 12848 consensus_queue.cc:1048] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:04:59.235397 12465 raft_consensus.cc:1275] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Refusing update from remote peer a0184626ff924a26959a272550a00c05: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:04:59.237301 12879 consensus_queue.cc:1048] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e82db363f6484aa3be77e60927c02fa9" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 35975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:04:59.255120 12504 consensus_peers.cc:597] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.263242   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:04:59.277065 12505 consensus_peers.cc:597] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 -> Peer e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975): Couldn't send request to peer e82db363f6484aa3be77e60927c02fa9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.278031 12505 consensus_peers.cc:597] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 -> Peer e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975): Couldn't send request to peer e82db363f6484aa3be77e60927c02fa9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.300408 12505 consensus_peers.cc:597] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 -> Peer e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975): Couldn't send request to peer e82db363f6484aa3be77e60927c02fa9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.308558   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:59.309515   431 tablet_replica.cc:333] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.310324   431 raft_consensus.cc:2243] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.310997   431 raft_consensus.cc:2272] T 6bf95d8655624d84abcfb16f84c47bfd P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.313515   431 tablet_replica.cc:333] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.314170   431 raft_consensus.cc:2243] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.315073   431 raft_consensus.cc:2272] T 8843d3f9e08d42a480f5d335516ab427 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.317248   431 tablet_replica.cc:333] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.317953   431 raft_consensus.cc:2243] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.318467   431 raft_consensus.cc:2272] T 2f5b5c9972e84588bc82910949c271d5 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.321575   431 tablet_replica.cc:333] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.322247   431 raft_consensus.cc:2243] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.322760   431 raft_consensus.cc:2272] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.325621   431 tablet_replica.cc:333] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.326190   431 raft_consensus.cc:2243] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.326691   431 raft_consensus.cc:2272] T aff0923e6e974f478093276d83c0a3e5 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.329854   431 tablet_replica.cc:333] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.330506   431 raft_consensus.cc:2243] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.331719   431 raft_consensus.cc:2272] T 887bbd0f3df74e8c890d6025d42553dc P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:04:59.334966 12504 consensus_peers.cc:597] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 -> Peer 9b85e1f979194d6889a0490e970ebfde (127.0.107.193:46243): Couldn't send request to peer 9b85e1f979194d6889a0490e970ebfde. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:46243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.335158   431 tablet_replica.cc:333] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.335948   431 raft_consensus.cc:2243] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.336490   431 raft_consensus.cc:2272] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.339138   431 tablet_replica.cc:333] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.339793   431 raft_consensus.cc:2243] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.340468   431 pending_rounds.cc:70] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9: Trying to abort 1 pending ops.
I20260709 13:04:59.340687   431 pending_rounds.cc:77] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305635016172122112 op_type: NO_OP noop_request { }
I20260709 13:04:59.340977   431 raft_consensus.cc:2889] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:59.341274   431 raft_consensus.cc:2272] T 5ead0cc836794861a8c8c15929eb6e3a P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.343924   431 tablet_replica.cc:333] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.344642   431 raft_consensus.cc:2243] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.345439   431 raft_consensus.cc:2272] T 97cb4f0685fa44f79e19ae9b9eabf3fa P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.348506   431 tablet_replica.cc:333] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.349241   431 raft_consensus.cc:2243] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.349647   431 pending_rounds.cc:70] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9: Trying to abort 1 pending ops.
I20260709 13:04:59.349802   431 pending_rounds.cc:77] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305635016098598912 op_type: NO_OP noop_request { }
I20260709 13:04:59.350021   431 raft_consensus.cc:2889] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:59.350236   431 raft_consensus.cc:2272] T 679e72f804654636aca1ad4dc51900c6 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.352794   431 tablet_replica.cc:333] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.353339   431 raft_consensus.cc:2243] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.354300   431 raft_consensus.cc:2272] T 66dc07eb98b744208bc1ea38b3b814ad P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.356418   431 tablet_replica.cc:333] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.356894   431 raft_consensus.cc:2243] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.357833   431 raft_consensus.cc:2272] T b2723322211a40479ee5ef03f2d6b1ef P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.360026   431 tablet_replica.cc:333] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.360535   431 raft_consensus.cc:2243] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.361358   431 raft_consensus.cc:2272] T 9fa195d27155441086c43051a0eeaa78 P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.363888   431 tablet_replica.cc:333] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.364468   431 raft_consensus.cc:2243] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.365542   431 raft_consensus.cc:2272] T 5a44c20b12ff46099f11c28404c1d89b P e82db363f6484aa3be77e60927c02fa9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.367801   431 tablet_replica.cc:333] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.368340   431 raft_consensus.cc:2243] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.368844   431 raft_consensus.cc:2272] T ee78aec6998e4a29a17f470c6d8f0cd1 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.371142   431 tablet_replica.cc:333] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.371677   431 raft_consensus.cc:2243] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.372198   431 raft_consensus.cc:2272] T 75504590d7e64c50bbb8f946ea1fc1c3 P e82db363f6484aa3be77e60927c02fa9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.374538   431 tablet_replica.cc:333] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.375069   431 raft_consensus.cc:2243] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.376075   431 raft_consensus.cc:2272] T 518f8726e0ee4660b8c8f05c648f05a4 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.379315   431 tablet_replica.cc:333] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9: stopping tablet replica
I20260709 13:04:59.379995   431 raft_consensus.cc:2243] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.380537   431 raft_consensus.cc:2272] T 2c059adc4ffb476596307903819bb1c0 P e82db363f6484aa3be77e60927c02fa9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.428407   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:04:59.448163 12505 consensus_peers.cc:597] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 -> Peer e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975): Couldn't send request to peer e82db363f6484aa3be77e60927c02fa9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:35975: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:04:59.458781 12505 consensus_peers.cc:597] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 -> Peer e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975): Couldn't send request to peer e82db363f6484aa3be77e60927c02fa9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:35975: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.482076   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:04:59.508494 12505 consensus_peers.cc:597] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 -> Peer e82db363f6484aa3be77e60927c02fa9 (127.0.107.194:35975): Couldn't send request to peer e82db363f6484aa3be77e60927c02fa9. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:35975: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:04:59.520294   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:04:59.521315   431 tablet_replica.cc:333] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.522209   431 raft_consensus.cc:2243] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.522871   431 raft_consensus.cc:2272] T 8843d3f9e08d42a480f5d335516ab427 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.525398   431 tablet_replica.cc:333] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.526198   431 raft_consensus.cc:2243] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.527143   431 raft_consensus.cc:2272] T 6bf95d8655624d84abcfb16f84c47bfd P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.529336   431 tablet_replica.cc:333] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.530025   431 raft_consensus.cc:2243] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.530892   431 raft_consensus.cc:2272] T aff0923e6e974f478093276d83c0a3e5 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.533689   431 tablet_replica.cc:333] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.534382   431 raft_consensus.cc:2243] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.535334   431 raft_consensus.cc:2272] T 2f5b5c9972e84588bc82910949c271d5 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.537918   431 tablet_replica.cc:333] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.538543   431 raft_consensus.cc:2243] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.539433   431 raft_consensus.cc:2272] T 2c059adc4ffb476596307903819bb1c0 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.541687   431 tablet_replica.cc:333] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.542246   431 raft_consensus.cc:2243] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.542739   431 raft_consensus.cc:2272] T 518f8726e0ee4660b8c8f05c648f05a4 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.545394   431 tablet_replica.cc:333] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.546087   431 raft_consensus.cc:2243] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.547005   431 raft_consensus.cc:2272] T 81c4673118cf4c2ab6f3a59d06d8ebc3 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.549183   431 tablet_replica.cc:333] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.549786   431 raft_consensus.cc:2243] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.550688   431 raft_consensus.cc:2272] T 7b319df053e54d6dbaaf1e3f0eeb75b9 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.552867   431 tablet_replica.cc:333] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.553586   431 raft_consensus.cc:2243] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.554122   431 raft_consensus.cc:2272] T 887bbd0f3df74e8c890d6025d42553dc P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.556166   431 tablet_replica.cc:333] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.556700   431 raft_consensus.cc:2243] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.557180   431 raft_consensus.cc:2272] T 5ead0cc836794861a8c8c15929eb6e3a P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.559293   431 tablet_replica.cc:333] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.559808   431 raft_consensus.cc:2243] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.560721   431 raft_consensus.cc:2272] T 97cb4f0685fa44f79e19ae9b9eabf3fa P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.563351   431 tablet_replica.cc:333] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.563948   431 raft_consensus.cc:2243] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.564841   431 raft_consensus.cc:2272] T ee78aec6998e4a29a17f470c6d8f0cd1 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.567029   431 tablet_replica.cc:333] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.567569   431 raft_consensus.cc:2243] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.568065   431 raft_consensus.cc:2272] T 66dc07eb98b744208bc1ea38b3b814ad P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.570250   431 tablet_replica.cc:333] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.570749   431 raft_consensus.cc:2243] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.571568   431 raft_consensus.cc:2272] T 679e72f804654636aca1ad4dc51900c6 P a0184626ff924a26959a272550a00c05 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.574036   431 tablet_replica.cc:333] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.574573   431 raft_consensus.cc:2243] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.575055   431 raft_consensus.cc:2272] T 9fa195d27155441086c43051a0eeaa78 P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.577095   431 tablet_replica.cc:333] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.577656   431 raft_consensus.cc:2243] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.578095   431 raft_consensus.cc:2272] T 5a44c20b12ff46099f11c28404c1d89b P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.580174   431 tablet_replica.cc:333] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.580677   431 raft_consensus.cc:2243] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:04:59.581112   431 raft_consensus.cc:2272] T b2723322211a40479ee5ef03f2d6b1ef P a0184626ff924a26959a272550a00c05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.583205   431 tablet_replica.cc:333] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05: stopping tablet replica
I20260709 13:04:59.583798   431 raft_consensus.cc:2243] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.584509   431 pending_rounds.cc:70] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05: Trying to abort 1 pending ops.
I20260709 13:04:59.584707   431 pending_rounds.cc:77] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305635016337866752 op_type: NO_OP noop_request { }
I20260709 13:04:59.584975   431 raft_consensus.cc:2889] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:04:59.585237   431 raft_consensus.cc:2272] T 75504590d7e64c50bbb8f946ea1fc1c3 P a0184626ff924a26959a272550a00c05 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.647894   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:04:59.686345   431 master.cc:562] Master@127.0.107.254:35413 shutting down...
I20260709 13:04:59.710209   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:04:59.710843   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:04:59.711198   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P da2bdb6bde834ca588d7e7a427254aa5: stopping tablet replica
I20260709 13:04:59.734649   431 master.cc:584] Master@127.0.107.254:35413 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (15370 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:04:59.789356   431 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.107.254:37535
I20260709 13:04:59.790575   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:04:59.796639 12906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:59.796702 12905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:04:59.796873 12908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:04:59.799072   431 server_base.cc:1061] running on GCE node
I20260709 13:04:59.800123   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:04:59.800367   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:04:59.800537   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602299800521 us; error 0 us; skew 500 ppm
I20260709 13:04:59.801152   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:04:59.803678   431 webserver.cc:533] Webserver started at http://127.0.107.254:39785/ using document root <none> and password file <none>
I20260709 13:04:59.804190   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:04:59.804394   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:04:59.804667   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:04:59.805907   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/master-0-root/instance:
uuid: "ce544662cc0b4e3fbcdfe294229a8b36"
format_stamp: "Formatted at 2026-07-09 13:04:59 on dist-test-slave-jk05"
I20260709 13:04:59.810648   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:04:59.814086 12913 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:04:59.814904   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:04:59.815207   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/master-0-root
uuid: "ce544662cc0b4e3fbcdfe294229a8b36"
format_stamp: "Formatted at 2026-07-09 13:04:59 on dist-test-slave-jk05"
I20260709 13:04:59.815500   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:04:59.837561   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:04:59.838825   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:04:59.878756   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.254:37535
I20260709 13:04:59.878862 12964 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.254:37535 every 8 connection(s)
I20260709 13:04:59.883003 12965 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:04:59.895913 12965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36: Bootstrap starting.
I20260709 13:04:59.900775 12965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36: Neither blocks nor log segments found. Creating new log.
I20260709 13:04:59.905299 12965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36: No bootstrap required, opened a new log
I20260709 13:04:59.907701 12965 raft_consensus.cc:359] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce544662cc0b4e3fbcdfe294229a8b36" member_type: VOTER }
I20260709 13:04:59.908162 12965 raft_consensus.cc:385] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:04:59.908408 12965 raft_consensus.cc:740] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce544662cc0b4e3fbcdfe294229a8b36, State: Initialized, Role: FOLLOWER
I20260709 13:04:59.909031 12965 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce544662cc0b4e3fbcdfe294229a8b36" member_type: VOTER }
I20260709 13:04:59.909569 12965 raft_consensus.cc:399] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:04:59.909844 12965 raft_consensus.cc:493] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:04:59.910130 12965 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:04:59.915162 12965 raft_consensus.cc:515] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce544662cc0b4e3fbcdfe294229a8b36" member_type: VOTER }
I20260709 13:04:59.915751 12965 leader_election.cc:304] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce544662cc0b4e3fbcdfe294229a8b36; no voters: 
I20260709 13:04:59.916956 12965 leader_election.cc:290] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:04:59.917402 12968 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:04:59.918910 12968 raft_consensus.cc:697] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [term 1 LEADER]: Becoming Leader. State: Replica: ce544662cc0b4e3fbcdfe294229a8b36, State: Running, Role: LEADER
I20260709 13:04:59.919631 12968 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce544662cc0b4e3fbcdfe294229a8b36" member_type: VOTER }
I20260709 13:04:59.920324 12965 sys_catalog.cc:565] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:04:59.925047 12969 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce544662cc0b4e3fbcdfe294229a8b36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce544662cc0b4e3fbcdfe294229a8b36" member_type: VOTER } }
I20260709 13:04:59.925361 12970 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce544662cc0b4e3fbcdfe294229a8b36. Latest consensus state: current_term: 1 leader_uuid: "ce544662cc0b4e3fbcdfe294229a8b36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce544662cc0b4e3fbcdfe294229a8b36" member_type: VOTER } }
I20260709 13:04:59.926252 12970 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:59.926250 12969 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [sys.catalog]: This master's current role is: LEADER
I20260709 13:04:59.929236 12974 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:04:59.937865 12974 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:04:59.939545   431 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:04:59.947413 12974 catalog_manager.cc:1383] Generated new cluster ID: df2e5bc859e44e8380ccaa9c2ca551f7
I20260709 13:04:59.947741 12974 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:04:59.961527 12974 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:04:59.962939 12974 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:04:59.983800 12974 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36: Generated new TSK 0
I20260709 13:04:59.984548 12974 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:05:00.006760   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:00.013190 12986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:00.014593 12987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:00.015671 12989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:00.018482   431 server_base.cc:1061] running on GCE node
I20260709 13:05:00.020955   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:00.021200   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:00.021390   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602300021370 us; error 0 us; skew 500 ppm
I20260709 13:05:00.022027   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:00.024494   431 webserver.cc:533] Webserver started at http://127.0.107.193:42967/ using document root <none> and password file <none>
I20260709 13:05:00.024994   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:00.025203   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:00.025504   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:00.026696   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-0-root/instance:
uuid: "91c9fa98571441c8959020435c520d0b"
format_stamp: "Formatted at 2026-07-09 13:05:00 on dist-test-slave-jk05"
I20260709 13:05:00.031347   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:05:00.034726 12994 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:00.035472   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:05:00.035775   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-0-root
uuid: "91c9fa98571441c8959020435c520d0b"
format_stamp: "Formatted at 2026-07-09 13:05:00 on dist-test-slave-jk05"
I20260709 13:05:00.036075   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:00.053165   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:00.054472   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:00.055943   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:00.058318   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:00.058544   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:00.058800   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:00.059005   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:00.102082   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.193:40339
I20260709 13:05:00.102195 13056 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.193:40339 every 8 connection(s)
I20260709 13:05:00.107612   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:00.115816 13062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:00.118772 13061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:00.122684 13057 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37535
I20260709 13:05:00.123271 13057 heartbeater.cc:461] Registering TS with master...
W20260709 13:05:00.123973 13064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:00.124013   431 server_base.cc:1061] running on GCE node
I20260709 13:05:00.124328 13057 heartbeater.cc:507] Master 127.0.107.254:37535 requested a full tablet report, sending...
I20260709 13:05:00.126026   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:00.126283   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:00.126500   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602300126480 us; error 0 us; skew 500 ppm
I20260709 13:05:00.126974 12930 ts_manager.cc:194] Registered new tserver with Master: 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339)
I20260709 13:05:00.127367   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:00.129714 12930 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42448
I20260709 13:05:00.130487   431 webserver.cc:533] Webserver started at http://127.0.107.194:43387/ using document root <none> and password file <none>
I20260709 13:05:00.131170   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:00.131408   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:00.131688   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:00.133015   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-1-root/instance:
uuid: "1802553b3b0245be83aff21b0b5b23d0"
format_stamp: "Formatted at 2026-07-09 13:05:00 on dist-test-slave-jk05"
I20260709 13:05:00.137764   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:05:00.141189 13069 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:00.142081   431 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:05:00.142412   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-1-root
uuid: "1802553b3b0245be83aff21b0b5b23d0"
format_stamp: "Formatted at 2026-07-09 13:05:00 on dist-test-slave-jk05"
I20260709 13:05:00.142717   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:00.173995   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:00.175237   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:00.176774   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:00.179075   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:00.179309   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:00.179563   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:00.179730   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:00.221553   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.194:36541
I20260709 13:05:00.221665 13131 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.194:36541 every 8 connection(s)
I20260709 13:05:00.226696   431 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:05:00.234512 13135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:05:00.236829 13136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:00.241663 13132 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37535
I20260709 13:05:00.242236 13132 heartbeater.cc:461] Registering TS with master...
W20260709 13:05:00.242693 13138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:05:00.242817   431 server_base.cc:1061] running on GCE node
I20260709 13:05:00.243331 13132 heartbeater.cc:507] Master 127.0.107.254:37535 requested a full tablet report, sending...
I20260709 13:05:00.245738   431 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:05:00.246053   431 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:05:00.246220 12930 ts_manager.cc:194] Registered new tserver with Master: 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:00.246271   431 hybrid_clock.cc:648] HybridClock initialized: now 1783602300246231 us; error 0 us; skew 500 ppm
I20260709 13:05:00.247216   431 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:05:00.247887 12930 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42464
I20260709 13:05:00.250111   431 webserver.cc:533] Webserver started at http://127.0.107.195:41049/ using document root <none> and password file <none>
I20260709 13:05:00.250711   431 fs_manager.cc:362] Metadata directory not provided
I20260709 13:05:00.250921   431 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:05:00.251185   431 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:05:00.252380   431 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-2-root/instance:
uuid: "648954f023084ca68a9dc69c1bdbf372"
format_stamp: "Formatted at 2026-07-09 13:05:00 on dist-test-slave-jk05"
I20260709 13:05:00.257042   431 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:05:00.260655 13143 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:00.261639   431 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:05:00.261916   431 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-2-root
uuid: "648954f023084ca68a9dc69c1bdbf372"
format_stamp: "Formatted at 2026-07-09 13:05:00 on dist-test-slave-jk05"
I20260709 13:05:00.262172   431 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskK4Os1p/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783601935286233-431-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:05:00.274798   431 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:05:00.275949   431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:05:00.277391   431 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:05:00.279893   431 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:05:00.280081   431 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:00.280365   431 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:05:00.280514   431 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:05:00.322307   431 rpc_server.cc:307] RPC server started. Bound to: 127.0.107.195:35167
I20260709 13:05:00.322422 13205 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.107.195:35167 every 8 connection(s)
I20260709 13:05:00.336656 13206 heartbeater.cc:344] Connected to a master server at 127.0.107.254:37535
I20260709 13:05:00.337178 13206 heartbeater.cc:461] Registering TS with master...
I20260709 13:05:00.338052 13206 heartbeater.cc:507] Master 127.0.107.254:37535 requested a full tablet report, sending...
I20260709 13:05:00.339973 12930 ts_manager.cc:194] Registered new tserver with Master: 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:00.340381   431 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014776122s
I20260709 13:05:00.341650 12930 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42472
I20260709 13:05:00.367141 12930 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42486:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:05:00.369992 12930 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:05:00.472435 13097 tablet_service.cc:1511] Processing CreateTablet for tablet aa2c897f99564a1eaf2b475b8094cbaf (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:00.474695 13097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa2c897f99564a1eaf2b475b8094cbaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.476500 13092 tablet_service.cc:1511] Processing CreateTablet for tablet 97da23ee368d48ddbe16515fc5dceff8 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:00.478322 13092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97da23ee368d48ddbe16515fc5dceff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.478638 13021 tablet_service.cc:1511] Processing CreateTablet for tablet da04db13d6144ff084cbd7fa1daee828 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:00.479146 13019 tablet_service.cc:1511] Processing CreateTablet for tablet 54b6273b4d944500bc90ab3c2aa15c91 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:00.479795 13094 tablet_service.cc:1511] Processing CreateTablet for tablet 54b6273b4d944500bc90ab3c2aa15c91 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:00.481670 13020 tablet_service.cc:1511] Processing CreateTablet for tablet ce472f3576f0423bba225ef4d6719f73 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:00.478695 13095 tablet_service.cc:1511] Processing CreateTablet for tablet ce472f3576f0423bba225ef4d6719f73 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:00.483229 13018 tablet_service.cc:1511] Processing CreateTablet for tablet ac72e4f8e9dd4531847ba293bec2f7b0 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:00.484576 13093 tablet_service.cc:1511] Processing CreateTablet for tablet ac72e4f8e9dd4531847ba293bec2f7b0 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:00.484800 13017 tablet_service.cc:1511] Processing CreateTablet for tablet 97da23ee368d48ddbe16515fc5dceff8 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:00.476627 13022 tablet_service.cc:1511] Processing CreateTablet for tablet aa2c897f99564a1eaf2b475b8094cbaf (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:00.486320 13096 tablet_service.cc:1511] Processing CreateTablet for tablet da04db13d6144ff084cbd7fa1daee828 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:00.480232 13021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da04db13d6144ff084cbd7fa1daee828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.487771 13016 tablet_service.cc:1511] Processing CreateTablet for tablet ac66224c363f48a49e1af00db21c47cf (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:00.488971 13094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54b6273b4d944500bc90ab3c2aa15c91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.493379 13016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac66224c363f48a49e1af00db21c47cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.494998 13091 tablet_service.cc:1511] Processing CreateTablet for tablet ac66224c363f48a49e1af00db21c47cf (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:00.496726 13091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac66224c363f48a49e1af00db21c47cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.497331 13019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54b6273b4d944500bc90ab3c2aa15c91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.497973 13093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac72e4f8e9dd4531847ba293bec2f7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.498699 13090 tablet_service.cc:1511] Processing CreateTablet for tablet b344a32e631c427f9275b5d0b7543901 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:00.500201 13090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b344a32e631c427f9275b5d0b7543901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.500763 13017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97da23ee368d48ddbe16515fc5dceff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.502646 13018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac72e4f8e9dd4531847ba293bec2f7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.502779 13089 tablet_service.cc:1511] Processing CreateTablet for tablet 90d7b9d3a73b468ea5499974cdfb073e (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:00.503875 13096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da04db13d6144ff084cbd7fa1daee828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.504369 13020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce472f3576f0423bba225ef4d6719f73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.506215 13022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa2c897f99564a1eaf2b475b8094cbaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.511245 13015 tablet_service.cc:1511] Processing CreateTablet for tablet b344a32e631c427f9275b5d0b7543901 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:00.512854 13015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b344a32e631c427f9275b5d0b7543901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.514709 13014 tablet_service.cc:1511] Processing CreateTablet for tablet 90d7b9d3a73b468ea5499974cdfb073e (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:00.516223 13014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90d7b9d3a73b468ea5499974cdfb073e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.538172 13089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90d7b9d3a73b468ea5499974cdfb073e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.544514 13095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce472f3576f0423bba225ef4d6719f73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.554574 13167 tablet_service.cc:1511] Processing CreateTablet for tablet ac72e4f8e9dd4531847ba293bec2f7b0 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:00.555667 13170 tablet_service.cc:1511] Processing CreateTablet for tablet da04db13d6144ff084cbd7fa1daee828 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:00.556222 13167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac72e4f8e9dd4531847ba293bec2f7b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.557212 13170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da04db13d6144ff084cbd7fa1daee828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.559026 13171 tablet_service.cc:1511] Processing CreateTablet for tablet aa2c897f99564a1eaf2b475b8094cbaf (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:00.559137 13165 tablet_service.cc:1511] Processing CreateTablet for tablet ac66224c363f48a49e1af00db21c47cf (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:00.562604 13164 tablet_service.cc:1511] Processing CreateTablet for tablet b344a32e631c427f9275b5d0b7543901 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:00.563278 13163 tablet_service.cc:1511] Processing CreateTablet for tablet 90d7b9d3a73b468ea5499974cdfb073e (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:00.564570 13171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa2c897f99564a1eaf2b475b8094cbaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.554577 13169 tablet_service.cc:1511] Processing CreateTablet for tablet ce472f3576f0423bba225ef4d6719f73 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:00.561214 13166 tablet_service.cc:1511] Processing CreateTablet for tablet 97da23ee368d48ddbe16515fc5dceff8 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:00.567099 13165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac66224c363f48a49e1af00db21c47cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.564644 13168 tablet_service.cc:1511] Processing CreateTablet for tablet 54b6273b4d944500bc90ab3c2aa15c91 (DEFAULT_TABLE table=test-workload [id=370373267d17415bab082e305d6f4b36]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:00.570850 13163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90d7b9d3a73b468ea5499974cdfb073e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.571404 13221 tablet_bootstrap.cc:492] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:00.575093 13164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b344a32e631c427f9275b5d0b7543901. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.577797 13168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54b6273b4d944500bc90ab3c2aa15c91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.602066 13166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97da23ee368d48ddbe16515fc5dceff8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.610843 13169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce472f3576f0423bba225ef4d6719f73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:00.635021 13221 tablet_bootstrap.cc:654] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.654613 13222 tablet_bootstrap.cc:492] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:00.676486 13223 tablet_bootstrap.cc:492] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:00.689177 13222 tablet_bootstrap.cc:654] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.695458 13223 tablet_bootstrap.cc:654] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.713315 13221 tablet_bootstrap.cc:492] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:00.713897 13221 ts_tablet_manager.cc:1403] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.143s	user 0.016s	sys 0.015s
I20260709 13:05:00.717025 13221 raft_consensus.cc:359] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.717803 13221 raft_consensus.cc:385] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.718109 13221 raft_consensus.cc:740] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.719061 13221 consensus_queue.cc:260] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.723835 13222 tablet_bootstrap.cc:492] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:00.725154 13222 ts_tablet_manager.cc:1403] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.071s	user 0.006s	sys 0.015s
I20260709 13:05:00.725561 13132 heartbeater.cc:499] Master 127.0.107.254:37535 was elected leader, sending a full tablet report...
I20260709 13:05:00.726070 13223 tablet_bootstrap.cc:492] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:00.726397 13221 ts_tablet_manager.cc:1434] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.001s
I20260709 13:05:00.726627 13223 ts_tablet_manager.cc:1403] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.051s	user 0.003s	sys 0.011s
W20260709 13:05:00.727310 13133 tablet.cc:2406] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:05:00.728328 13222 raft_consensus.cc:359] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.728999 13221 tablet_bootstrap.cc:492] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:00.729135 13222 raft_consensus.cc:385] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.729657 13222 raft_consensus.cc:740] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.729915 13223 raft_consensus.cc:359] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } }
I20260709 13:05:00.730736 13223 raft_consensus.cc:385] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.731060 13223 raft_consensus.cc:740] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.731142 13222 consensus_queue.cc:260] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.731992 13223 consensus_queue.cc:260] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } }
I20260709 13:05:00.738756 13221 tablet_bootstrap.cc:654] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.741559 13057 heartbeater.cc:499] Master 127.0.107.254:37535 was elected leader, sending a full tablet report...
I20260709 13:05:00.751269 13223 ts_tablet_manager.cc:1434] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.024s	user 0.013s	sys 0.010s
I20260709 13:05:00.752676 13206 heartbeater.cc:499] Master 127.0.107.254:37535 was elected leader, sending a full tablet report...
I20260709 13:05:00.757213 13223 tablet_bootstrap.cc:492] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:00.741161 13222 ts_tablet_manager.cc:1434] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 13:05:00.761173 13222 tablet_bootstrap.cc:492] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:00.763876 13223 tablet_bootstrap.cc:654] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.770670 13222 tablet_bootstrap.cc:654] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.774230 13221 tablet_bootstrap.cc:492] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:00.774891 13221 ts_tablet_manager.cc:1403] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.046s	user 0.025s	sys 0.016s
I20260709 13:05:00.778285 13221 raft_consensus.cc:359] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.779157 13221 raft_consensus.cc:385] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.779475 13221 raft_consensus.cc:740] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.780361 13221 consensus_queue.cc:260] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.782848 13221 ts_tablet_manager.cc:1434] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:00.783954 13221 tablet_bootstrap.cc:492] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:00.790405 13221 tablet_bootstrap.cc:654] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.802387 13221 tablet_bootstrap.cc:492] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:00.803011 13221 ts_tablet_manager.cc:1403] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 13:05:00.806447 13221 raft_consensus.cc:359] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.807291 13221 raft_consensus.cc:385] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.807740 13221 raft_consensus.cc:740] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.808523 13221 consensus_queue.cc:260] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.808771 13223 tablet_bootstrap.cc:492] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:00.809474 13223 ts_tablet_manager.cc:1403] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.053s	user 0.025s	sys 0.026s
I20260709 13:05:00.809798 13222 tablet_bootstrap.cc:492] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:00.810354 13222 ts_tablet_manager.cc:1403] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.049s	user 0.031s	sys 0.013s
I20260709 13:05:00.810711 13221 ts_tablet_manager.cc:1434] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:05:00.811846 13221 tablet_bootstrap.cc:492] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:00.812470 13223 raft_consensus.cc:359] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.813249 13223 raft_consensus.cc:385] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.813134 13222 raft_consensus.cc:359] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.813692 13223 raft_consensus.cc:740] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.814026 13222 raft_consensus.cc:385] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.814458 13222 raft_consensus.cc:740] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.814509 13223 consensus_queue.cc:260] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.815541 13222 consensus_queue.cc:260] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.819139 13221 tablet_bootstrap.cc:654] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.819377 13223 ts_tablet_manager.cc:1434] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:05:00.820106 13222 ts_tablet_manager.cc:1434] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:05:00.820577 13223 tablet_bootstrap.cc:492] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:00.821154 13222 tablet_bootstrap.cc:492] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:00.828424 13222 tablet_bootstrap.cc:654] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.829286 13223 tablet_bootstrap.cc:654] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.831328 13221 tablet_bootstrap.cc:492] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:00.831888 13221 ts_tablet_manager.cc:1403] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 13:05:00.835215 13221 raft_consensus.cc:359] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.837910 13221 raft_consensus.cc:385] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.838246 13221 raft_consensus.cc:740] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.839053 13221 consensus_queue.cc:260] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.841464 13221 ts_tablet_manager.cc:1434] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:05:00.842506 13221 tablet_bootstrap.cc:492] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:00.849215 13221 tablet_bootstrap.cc:654] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.849335 13223 tablet_bootstrap.cc:492] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:00.850206 13223 ts_tablet_manager.cc:1403] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.011s
I20260709 13:05:00.854844 13223 raft_consensus.cc:359] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.855667 13223 raft_consensus.cc:385] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.856122 13223 raft_consensus.cc:740] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.856995 13223 consensus_queue.cc:260] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.858266 13222 tablet_bootstrap.cc:492] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:00.858904 13222 ts_tablet_manager.cc:1403] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.038s	user 0.015s	sys 0.015s
I20260709 13:05:00.860309 13223 ts_tablet_manager.cc:1434] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:05:00.861619 13223 tablet_bootstrap.cc:492] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:00.861593 13221 tablet_bootstrap.cc:492] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:00.862430 13221 ts_tablet_manager.cc:1403] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.010s
I20260709 13:05:00.862311 13222 raft_consensus.cc:359] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.863173 13222 raft_consensus.cc:385] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.863552 13222 raft_consensus.cc:740] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.864481 13222 consensus_queue.cc:260] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.866304 13221 raft_consensus.cc:359] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.867157 13221 raft_consensus.cc:385] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.867597 13221 raft_consensus.cc:740] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.868433 13221 consensus_queue.cc:260] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.869206 13223 tablet_bootstrap.cc:654] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.870110 13222 ts_tablet_manager.cc:1434] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 13:05:00.870725 13221 ts_tablet_manager.cc:1434] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 13:05:00.871351 13222 tablet_bootstrap.cc:492] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:00.871815 13221 tablet_bootstrap.cc:492] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:00.879474 13221 tablet_bootstrap.cc:654] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.880996 13222 tablet_bootstrap.cc:654] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.894358 13222 tablet_bootstrap.cc:492] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:00.894918 13222 ts_tablet_manager.cc:1403] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 13:05:00.894986 13223 tablet_bootstrap.cc:492] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:00.895574 13223 ts_tablet_manager.cc:1403] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.013s
I20260709 13:05:00.896081 13221 tablet_bootstrap.cc:492] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:00.896773 13221 ts_tablet_manager.cc:1403] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.009s
I20260709 13:05:00.898682 13223 raft_consensus.cc:359] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.898511 13222 raft_consensus.cc:359] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.899533 13223 raft_consensus.cc:385] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.899605 13222 raft_consensus.cc:385] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.899974 13223 raft_consensus.cc:740] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.900029 13222 raft_consensus.cc:740] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.900058 13221 raft_consensus.cc:359] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.901086 13221 raft_consensus.cc:385] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.900923 13223 consensus_queue.cc:260] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.901465 13221 raft_consensus.cc:740] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.901185 13222 consensus_queue.cc:260] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.902375 13221 consensus_queue.cc:260] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.903937 13223 ts_tablet_manager.cc:1434] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:05:00.904788 13221 ts_tablet_manager.cc:1434] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:05:00.906039 13222 ts_tablet_manager.cc:1434] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 13:05:00.907192 13222 tablet_bootstrap.cc:492] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:00.907817 13221 tablet_bootstrap.cc:492] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:00.908840 13223 tablet_bootstrap.cc:492] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:00.914062 13222 tablet_bootstrap.cc:654] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.915271 13221 tablet_bootstrap.cc:654] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.916183 13223 tablet_bootstrap.cc:654] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.927151 13222 tablet_bootstrap.cc:492] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:00.927542 13221 tablet_bootstrap.cc:492] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:00.927949 13222 ts_tablet_manager.cc:1403] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 13:05:00.928136 13221 ts_tablet_manager.cc:1403] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.000s
I20260709 13:05:00.931922 13221 raft_consensus.cc:359] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } }
I20260709 13:05:00.932746 13221 raft_consensus.cc:385] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.932518 13222 raft_consensus.cc:359] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } }
I20260709 13:05:00.933164 13221 raft_consensus.cc:740] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.934243 13222 raft_consensus.cc:385] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:05:00.934443 12978 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:00.934617 13222 raft_consensus.cc:740] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.934357 13221 consensus_queue.cc:260] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } }
I20260709 13:05:00.935374 13222 consensus_queue.cc:260] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } }
I20260709 13:05:00.937631 13221 ts_tablet_manager.cc:1434] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.004s
I20260709 13:05:00.938174 13222 ts_tablet_manager.cc:1434] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.004s
I20260709 13:05:00.938747 13221 tablet_bootstrap.cc:492] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:00.939261 13222 tablet_bootstrap.cc:492] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:00.941008 13223 tablet_bootstrap.cc:492] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:00.941599 13223 ts_tablet_manager.cc:1403] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.011s
I20260709 13:05:00.945003 13223 raft_consensus.cc:359] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.945814 13223 raft_consensus.cc:385] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:00.946122 13223 raft_consensus.cc:740] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:00.946657 13222 tablet_bootstrap.cc:654] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.946698 13221 tablet_bootstrap.cc:654] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.947012 13223 consensus_queue.cc:260] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.952951 13223 ts_tablet_manager.cc:1434] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:05:00.954550 13223 tablet_bootstrap.cc:492] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:00.963537 13229 raft_consensus.cc:493] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.964452 13229 raft_consensus.cc:515] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.969556 13229 leader_election.cc:290] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:00.975891 13223 tablet_bootstrap.cc:654] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:00.991860 13223 tablet_bootstrap.cc:492] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:00.992606 13223 ts_tablet_manager.cc:1403] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.009s
I20260709 13:05:00.993103 13227 raft_consensus.cc:493] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:00.993750 13227 raft_consensus.cc:515] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:00.997879 13227 leader_election.cc:290] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.008404 13228 raft_consensus.cc:493] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.009109 13228 raft_consensus.cc:515] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.009394 13223 raft_consensus.cc:359] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.010545 13223 raft_consensus.cc:385] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:01.010998 13223 raft_consensus.cc:740] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.012058 13223 consensus_queue.cc:260] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.019496 13228 leader_election.cc:290] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.017769 13032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac66224c363f48a49e1af00db21c47cf" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:01.025122 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac66224c363f48a49e1af00db21c47cf" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
W20260709 13:05:01.027724 13145 leader_election.cc:343] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:01.028041 13222 tablet_bootstrap.cc:492] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:01.028934 13222 ts_tablet_manager.cc:1403] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.090s	user 0.025s	sys 0.010s
I20260709 13:05:01.035504 13223 ts_tablet_manager.cc:1434] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.001s
I20260709 13:05:01.038215 13223 tablet_bootstrap.cc:492] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:01.039084 13227 raft_consensus.cc:493] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.039677 13227 raft_consensus.cc:515] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
W20260709 13:05:01.041337 13145 leader_election.cc:343] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:01.041929 13145 leader_election.cc:304] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372; no voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b
I20260709 13:05:01.043248 13221 tablet_bootstrap.cc:492] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:01.043742 13221 ts_tablet_manager.cc:1403] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.105s	user 0.024s	sys 0.035s
I20260709 13:05:01.042001 13227 leader_election.cc:290] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.042551 13229 raft_consensus.cc:493] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.046451 13228 raft_consensus.cc:493] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.048868 13221 raft_consensus.cc:359] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.050810 13228 raft_consensus.cc:515] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.050923 13221 raft_consensus.cc:385] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:01.050370 13229 raft_consensus.cc:515] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } }
I20260709 13:05:01.052373 13221 raft_consensus.cc:740] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.046169 13222 raft_consensus.cc:359] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.054629 13221 consensus_queue.cc:260] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.055338 13227 raft_consensus.cc:493] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.054738 13229 leader_election.cc:290] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339)
I20260709 13:05:01.053660 13222 raft_consensus.cc:385] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:01.056347 13222 raft_consensus.cc:740] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.057102 13222 consensus_queue.cc:260] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.058462 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2c897f99564a1eaf2b475b8094cbaf" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:01.056875 13229 raft_consensus.cc:2749] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:01.059100 13107 raft_consensus.cc:2468] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 0.
I20260709 13:05:01.060261 13032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2c897f99564a1eaf2b475b8094cbaf" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:01.060521 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344a32e631c427f9275b5d0b7543901" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:01.061439 13181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac72e4f8e9dd4531847ba293bec2f7b0" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:01.066906 13255 raft_consensus.cc:493] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.067037 13145 leader_election.cc:304] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
I20260709 13:05:01.059693 13227 raft_consensus.cc:515] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.063748 13228 leader_election.cc:290] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.066360 13180 raft_consensus.cc:2468] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 0.
I20260709 13:05:01.067414 13181 raft_consensus.cc:2468] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 0.
I20260709 13:05:01.074811 13229 raft_consensus.cc:2804] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:01.075435 13229 raft_consensus.cc:493] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.075897 13229 raft_consensus.cc:3060] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.082311 13030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344a32e631c427f9275b5d0b7543901" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:01.082841 13227 leader_election.cc:290] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.084663 13072 leader_election.cc:304] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
I20260709 13:05:01.085750 13072 leader_election.cc:304] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
I20260709 13:05:01.086923 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac72e4f8e9dd4531847ba293bec2f7b0" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:01.067728 13255 raft_consensus.cc:515] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.087710 13031 raft_consensus.cc:2468] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 0.
I20260709 13:05:01.088925 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97da23ee368d48ddbe16515fc5dceff8" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:01.089674 13255 leader_election.cc:290] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
W20260709 13:05:01.091472 13071 leader_election.cc:343] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:01.093672 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97da23ee368d48ddbe16515fc5dceff8" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:01.094650 13031 raft_consensus.cc:2468] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 0.
I20260709 13:05:01.095319 13227 raft_consensus.cc:2804] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:01.105572 13221 ts_tablet_manager.cc:1434] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.062s	user 0.013s	sys 0.019s
I20260709 13:05:01.104665 13227 raft_consensus.cc:493] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.106557 13227 raft_consensus.cc:3060] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.106711 13221 tablet_bootstrap.cc:492] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:01.073199 13032 raft_consensus.cc:2468] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 0.
I20260709 13:05:01.111482 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97da23ee368d48ddbe16515fc5dceff8" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:01.111924 13228 raft_consensus.cc:493] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.112515 13228 raft_consensus.cc:515] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.104207 13071 leader_election.cc:304] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 
W20260709 13:05:01.113262 13145 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:36541, user_credentials={real_user=slave}} blocked reactor thread for 46517.7us
I20260709 13:05:01.103103 13259 raft_consensus.cc:2804] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:05:01.114095 12997 leader_election.cc:343] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:05:01.114853 13072 leader_election.cc:343] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:01.116595 13181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac72e4f8e9dd4531847ba293bec2f7b0" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:01.117084 13032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d7b9d3a73b468ea5499974cdfb073e" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:01.117488 13181 raft_consensus.cc:2468] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 0.
I20260709 13:05:01.123733 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d7b9d3a73b468ea5499974cdfb073e" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:01.123754 13229 raft_consensus.cc:515] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } }
I20260709 13:05:01.124219 13228 leader_election.cc:290] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.124590 13107 raft_consensus.cc:2468] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 0.
I20260709 13:05:01.124895 13228 raft_consensus.cc:493] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.126497 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce472f3576f0423bba225ef4d6719f73" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
W20260709 13:05:01.130292 13145 leader_election.cc:343] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:01.130669 13227 raft_consensus.cc:515] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.131544 12997 leader_election.cc:304] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:01.131811 13145 leader_election.cc:304] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 91c9fa98571441c8959020435c520d0b
I20260709 13:05:01.131982 13107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2c897f99564a1eaf2b475b8094cbaf" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:01.125398 13228 raft_consensus.cc:515] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.132649 13107 raft_consensus.cc:3060] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.115563 13259 raft_consensus.cc:493] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.133289 13227 leader_election.cc:290] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.133733 13259 raft_consensus.cc:3060] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.134083 13227 raft_consensus.cc:493] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.134806 13228 leader_election.cc:290] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.134639 13227 raft_consensus.cc:515] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.126641 13229 leader_election.cc:290] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 election: Requested vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339)
W20260709 13:05:01.132619 12997 leader_election.cc:343] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:01.136006 13229 raft_consensus.cc:2804] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:01.136379 13229 raft_consensus.cc:493] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.137343 13032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2c897f99564a1eaf2b475b8094cbaf" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:01.146266 13107 raft_consensus.cc:2468] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 1.
I20260709 13:05:01.146457 13032 raft_consensus.cc:3060] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.147351 13259 raft_consensus.cc:515] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.138224 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac72e4f8e9dd4531847ba293bec2f7b0" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:01.138418 13227 leader_election.cc:290] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.138509 13256 raft_consensus.cc:2804] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:01.139813 13030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac66224c363f48a49e1af00db21c47cf" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:01.140232 13181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac72e4f8e9dd4531847ba293bec2f7b0" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:01.149667 13181 raft_consensus.cc:3060] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.133260 13257 raft_consensus.cc:2804] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:01.150187 13257 raft_consensus.cc:493] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.150583 13257 raft_consensus.cc:3060] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.137396 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54b6273b4d944500bc90ab3c2aa15c91" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:01.143620 13223 tablet_bootstrap.cc:654] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:01.154505 13071 leader_election.cc:343] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:01.156023 13030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344a32e631c427f9275b5d0b7543901" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:01.140164 13222 ts_tablet_manager.cc:1434] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.111s	user 0.013s	sys 0.033s
I20260709 13:05:01.157681 13032 raft_consensus.cc:2468] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 1.
I20260709 13:05:01.145825 13221 tablet_bootstrap.cc:654] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:01.158687 13222 tablet_bootstrap.cc:492] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:01.159490 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97da23ee368d48ddbe16515fc5dceff8" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:01.140288 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac66224c363f48a49e1af00db21c47cf" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:01.160777 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac72e4f8e9dd4531847ba293bec2f7b0" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:01.147713 13145 leader_election.cc:304] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
W20260709 13:05:01.154528 12997 leader_election.cc:343] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:01.161475 13179 raft_consensus.cc:2468] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 0.
I20260709 13:05:01.162110 13229 raft_consensus.cc:3060] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.163123 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54b6273b4d944500bc90ab3c2aa15c91" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:01.161552 13256 raft_consensus.cc:493] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.163923 13256 raft_consensus.cc:3060] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.164717 13072 leader_election.cc:304] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 91c9fa98571441c8959020435c520d0b
I20260709 13:05:01.169293 13258 raft_consensus.cc:2804] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:01.170217 13258 raft_consensus.cc:697] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Becoming Leader. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Running, Role: LEADER
I20260709 13:05:01.170353 13104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce472f3576f0423bba225ef4d6719f73" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:01.170723 13229 raft_consensus.cc:515] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.171167 13104 raft_consensus.cc:2468] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 0.
I20260709 13:05:01.172534 13222 tablet_bootstrap.cc:654] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:01.171182 13258 consensus_queue.cc:237] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } }
I20260709 13:05:01.175501 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344a32e631c427f9275b5d0b7543901" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:01.175740 13032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d7b9d3a73b468ea5499974cdfb073e" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:01.176175 13179 raft_consensus.cc:3060] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.172799 12996 leader_election.cc:304] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 648954f023084ca68a9dc69c1bdbf372
W20260709 13:05:01.177014 13145 leader_election.cc:343] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:01.173116 13229 leader_election.cc:290] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:01.178807 13181 raft_consensus.cc:2468] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 1.
I20260709 13:05:01.163872 13103 raft_consensus.cc:2468] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 0.
I20260709 13:05:01.162137 13105 raft_consensus.cc:2393] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91c9fa98571441c8959020435c520d0b in current term 1: Already voted for candidate 1802553b3b0245be83aff21b0b5b23d0 in this term.
I20260709 13:05:01.173521 13227 raft_consensus.cc:2804] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:01.174010 13259 leader_election.cc:290] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.172930 13257 raft_consensus.cc:515] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.177134 13228 raft_consensus.cc:2804] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:01.183617 13227 raft_consensus.cc:493] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.184895 13228 raft_consensus.cc:493] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.184993 13227 raft_consensus.cc:3060] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.185464 13228 raft_consensus.cc:3060] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:05:01.173943 13071 leader_election.cc:343] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:01.182507 13072 leader_election.cc:304] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
I20260709 13:05:01.183226 12996 leader_election.cc:304] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:01.192750 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d7b9d3a73b468ea5499974cdfb073e" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:01.196172 13259 raft_consensus.cc:2804] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:01.196609 13105 raft_consensus.cc:3060] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.196780 13259 raft_consensus.cc:697] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Becoming Leader. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:01.197783 13259 consensus_queue.cc:237] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.199743 13031 raft_consensus.cc:2393] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1802553b3b0245be83aff21b0b5b23d0 in current term 1: Already voted for candidate 91c9fa98571441c8959020435c520d0b in this term.
I20260709 13:05:01.206300 13228 raft_consensus.cc:515] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.218107 13228 leader_election.cc:290] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Requested vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.219055 13228 raft_consensus.cc:2804] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:01.219538 13228 raft_consensus.cc:493] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.219992 13228 raft_consensus.cc:3060] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.224087 13221 tablet_bootstrap.cc:492] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:01.224895 13221 ts_tablet_manager.cc:1403] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.118s	user 0.012s	sys 0.020s
I20260709 13:05:01.226622 13181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce472f3576f0423bba225ef4d6719f73" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:01.223786 13257 leader_election.cc:290] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Requested vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.232559 13221 raft_consensus.cc:359] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.237712 13221 raft_consensus.cc:385] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:05:01.238430 12997 leader_election.cc:343] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:01.242937 13221 raft_consensus.cc:740] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.243515 13181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac72e4f8e9dd4531847ba293bec2f7b0" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:01.244787 13181 raft_consensus.cc:2393] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91c9fa98571441c8959020435c520d0b in current term 1: Already voted for candidate 1802553b3b0245be83aff21b0b5b23d0 in this term.
I20260709 13:05:01.247225 13223 tablet_bootstrap.cc:492] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:01.236467 13256 raft_consensus.cc:515] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.250602 13106 raft_consensus.cc:2393] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91c9fa98571441c8959020435c520d0b in current term 1: Already voted for candidate 1802553b3b0245be83aff21b0b5b23d0 in this term.
I20260709 13:05:01.250757 13222 tablet_bootstrap.cc:492] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:01.251348 13222 ts_tablet_manager.cc:1403] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.093s	user 0.007s	sys 0.010s
I20260709 13:05:01.248132 13223 ts_tablet_manager.cc:1403] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.210s	user 0.009s	sys 0.009s
I20260709 13:05:01.211967 13179 raft_consensus.cc:2468] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 1.
I20260709 13:05:01.254709 13222 raft_consensus.cc:359] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.254974 13223 raft_consensus.cc:359] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
W20260709 13:05:01.249604 12996 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:36541, user_credentials={real_user=slave}} blocked reactor thread for 66472.1us
I20260709 13:05:01.255535 13222 raft_consensus.cc:385] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:01.255991 13221 consensus_queue.cc:260] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.256562 13222 raft_consensus.cc:740] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.255401 13072 leader_election.cc:304] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 91c9fa98571441c8959020435c520d0b
I20260709 13:05:01.258112 13260 raft_consensus.cc:2804] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:01.260490 13221 ts_tablet_manager.cc:1434] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.003s
I20260709 13:05:01.260288 13260 raft_consensus.cc:697] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Becoming Leader. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:01.261273 13222 consensus_queue.cc:260] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.262168 13260 consensus_queue.cc:237] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.255769 13223 raft_consensus.cc:385] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:01.248077 12927 catalog_manager.cc:5697] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 reported cstate change: term changed from 0 to 1, leader changed from <none> to 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "648954f023084ca68a9dc69c1bdbf372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.273095 12996 leader_election.cc:304] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91c9fa98571441c8959020435c520d0b; no voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:01.278080 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce472f3576f0423bba225ef4d6719f73" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:01.278748 13106 raft_consensus.cc:3060] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.281457 13227 raft_consensus.cc:515] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.275113 13222 ts_tablet_manager.cc:1434] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.002s
I20260709 13:05:01.281865 12928 catalog_manager.cc:5697] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "1802553b3b0245be83aff21b0b5b23d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.286419 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac72e4f8e9dd4531847ba293bec2f7b0" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:01.286939 13227 leader_election.cc:290] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.287739 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac66224c363f48a49e1af00db21c47cf" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:01.275817 13105 raft_consensus.cc:2468] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 1.
W20260709 13:05:01.289742 13071 leader_election.cc:343] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:01.289899 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97da23ee368d48ddbe16515fc5dceff8" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:01.290683 13106 raft_consensus.cc:2468] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 1.
I20260709 13:05:01.290617 13145 leader_election.cc:304] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 91c9fa98571441c8959020435c520d0b
I20260709 13:05:01.291263 13264 raft_consensus.cc:3060] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.291771 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac66224c363f48a49e1af00db21c47cf" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:01.271202 13223 raft_consensus.cc:740] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.292415 13258 raft_consensus.cc:2804] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:01.293627 13258 raft_consensus.cc:697] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Becoming Leader. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Running, Role: LEADER
I20260709 13:05:01.293962 13181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "97da23ee368d48ddbe16515fc5dceff8" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:01.293816 13228 raft_consensus.cc:515] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.294657 13258 consensus_queue.cc:237] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.296147 13228 leader_election.cc:290] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Requested vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.292594 13257 raft_consensus.cc:493] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.292865 12996 leader_election.cc:304] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91c9fa98571441c8959020435c520d0b; no voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:01.297616 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54b6273b4d944500bc90ab3c2aa15c91" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:01.297930 13257 raft_consensus.cc:515] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.292814 13179 raft_consensus.cc:3060] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.302040 13257 leader_election.cc:290] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.288687 13256 leader_election.cc:290] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.300513 13228 raft_consensus.cc:2749] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:05:01.303923 12997 leader_election.cc:343] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:01.298771 12996 leader_election.cc:304] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:01.307097 13222 tablet_bootstrap.cc:492] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:01.307617 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da04db13d6144ff084cbd7fa1daee828" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:01.308343 13105 raft_consensus.cc:2468] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 0.
I20260709 13:05:01.309698 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da04db13d6144ff084cbd7fa1daee828" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:01.310490 13180 raft_consensus.cc:2468] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 0.
I20260709 13:05:01.293547 13223 consensus_queue.cc:260] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.312124 13228 raft_consensus.cc:2804] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:01.311031 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54b6273b4d944500bc90ab3c2aa15c91" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:01.311867 12997 leader_election.cc:304] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:01.316697 13106 raft_consensus.cc:3060] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.314033 13181 raft_consensus.cc:3060] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.316793 13228 raft_consensus.cc:697] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Becoming Leader. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:01.317966 13228 consensus_queue.cc:237] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.319712 13264 raft_consensus.cc:2749] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:01.320613 13271 raft_consensus.cc:2804] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:01.325001 13106 raft_consensus.cc:2468] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 1.
I20260709 13:05:01.327348 12996 leader_election.cc:304] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:01.328003 13031 raft_consensus.cc:2468] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 1.
I20260709 13:05:01.330740 13181 raft_consensus.cc:2468] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 1.
I20260709 13:05:01.332427 13223 ts_tablet_manager.cc:1434] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.000s
I20260709 13:05:01.333631 13223 tablet_bootstrap.cc:492] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:01.334041 13072 leader_election.cc:304] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
I20260709 13:05:01.335461 13256 raft_consensus.cc:2804] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:01.321061 13271 raft_consensus.cc:493] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:01.336102 13256 raft_consensus.cc:697] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Becoming Leader. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:01.336318 13271 raft_consensus.cc:3060] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.329231 13222 tablet_bootstrap.cc:654] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:01.330484 13179 raft_consensus.cc:2468] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 1.
I20260709 13:05:01.337347 13256 consensus_queue.cc:237] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.341969 13072 leader_election.cc:304] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 91c9fa98571441c8959020435c520d0b
I20260709 13:05:01.343761 13260 raft_consensus.cc:2804] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:01.344401 13260 raft_consensus.cc:697] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Becoming Leader. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:01.345619 13260 consensus_queue.cc:237] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.354390 13271 raft_consensus.cc:515] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.336102 13264 raft_consensus.cc:2804] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:01.357985 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da04db13d6144ff084cbd7fa1daee828" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:01.359014 13106 raft_consensus.cc:3060] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.362265 13271 leader_election.cc:290] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Requested vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.358325 13223 tablet_bootstrap.cc:654] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:01.366199 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "da04db13d6144ff084cbd7fa1daee828" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:01.366916 13179 raft_consensus.cc:3060] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.375615 12930 catalog_manager.cc:5697] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 91c9fa98571441c8959020435c520d0b (127.0.107.193). New cstate: current_term: 1 leader_uuid: "91c9fa98571441c8959020435c520d0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.380684 13179 raft_consensus.cc:2468] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 1.
I20260709 13:05:01.375895 13264 raft_consensus.cc:697] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Becoming Leader. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:01.382104 12997 leader_election.cc:304] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:01.382184 13264 consensus_queue.cc:237] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.383177 13271 raft_consensus.cc:2804] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:01.383700 13271 raft_consensus.cc:697] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Becoming Leader. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:01.374869 13223 tablet_bootstrap.cc:492] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:01.385015 13223 ts_tablet_manager.cc:1403] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.052s	user 0.009s	sys 0.004s
I20260709 13:05:01.384599 13271 consensus_queue.cc:237] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.373850 13106 raft_consensus.cc:2468] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 1.
I20260709 13:05:01.387981 13223 raft_consensus.cc:359] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.388684 13223 raft_consensus.cc:385] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:01.388975 13223 raft_consensus.cc:740] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.390146 13223 consensus_queue.cc:260] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.401389 13223 ts_tablet_manager.cc:1434] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.003s
I20260709 13:05:01.402705 13223 tablet_bootstrap.cc:492] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:01.406383 13222 tablet_bootstrap.cc:492] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:01.406977 13222 ts_tablet_manager.cc:1403] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.100s	user 0.020s	sys 0.027s
I20260709 13:05:01.408739 13223 tablet_bootstrap.cc:654] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:01.409930 13222 raft_consensus.cc:359] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.410681 13222 raft_consensus.cc:385] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:01.410940 13222 raft_consensus.cc:740] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.411885 13222 consensus_queue.cc:260] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.414291 13222 ts_tablet_manager.cc:1434] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:05:01.415304 13222 tablet_bootstrap.cc:492] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:01.421553 13222 tablet_bootstrap.cc:654] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:01.429754 13223 tablet_bootstrap.cc:492] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:01.430462 13223 ts_tablet_manager.cc:1403] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.004s
I20260709 13:05:01.432883 13222 tablet_bootstrap.cc:492] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:01.433522 13222 ts_tablet_manager.cc:1403] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.007s
I20260709 13:05:01.433271 13223 raft_consensus.cc:359] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.434569 13223 raft_consensus.cc:385] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:01.434825 13223 raft_consensus.cc:740] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.435606 13223 consensus_queue.cc:260] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.436388 13222 raft_consensus.cc:359] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.437244 13222 raft_consensus.cc:385] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:01.437541 13222 raft_consensus.cc:740] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:01.438558 13222 consensus_queue.cc:260] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.437839 13223 ts_tablet_manager.cc:1434] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:05:01.440791 13222 ts_tablet_manager.cc:1434] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:05:01.445251 12928 catalog_manager.cc:5697] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "1802553b3b0245be83aff21b0b5b23d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.446864 12928 catalog_manager.cc:5697] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "1802553b3b0245be83aff21b0b5b23d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.448262 12928 catalog_manager.cc:5697] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "1802553b3b0245be83aff21b0b5b23d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.458928 12927 catalog_manager.cc:5697] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 reported cstate change: term changed from 0 to 1, leader changed from <none> to 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "648954f023084ca68a9dc69c1bdbf372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:01.459308 12930 catalog_manager.cc:5697] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 91c9fa98571441c8959020435c520d0b (127.0.107.193). New cstate: current_term: 1 leader_uuid: "91c9fa98571441c8959020435c520d0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.461156 12930 catalog_manager.cc:5697] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 91c9fa98571441c8959020435c520d0b (127.0.107.193). New cstate: current_term: 1 leader_uuid: "91c9fa98571441c8959020435c520d0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:01.539407 13258 raft_consensus.cc:493] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.539959 13258 raft_consensus.cc:515] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.541842 13258 leader_election.cc:290] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:01.542668 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce472f3576f0423bba225ef4d6719f73" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:01.542917 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce472f3576f0423bba225ef4d6719f73" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:01.543756 13106 raft_consensus.cc:2393] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 648954f023084ca68a9dc69c1bdbf372 in current term 1: Already voted for candidate 91c9fa98571441c8959020435c520d0b in this term.
I20260709 13:05:01.544925 13145 leader_election.cc:304] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372; no voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b
I20260709 13:05:01.545833 13258 raft_consensus.cc:3060] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.551424 13258 raft_consensus.cc:2749] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:01.571216 13258 consensus_queue.cc:1048] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:01.578871 13260 consensus_queue.cc:1048] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:01.583840 13255 consensus_queue.cc:1048] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:01.607419 13256 consensus_queue.cc:1048] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:01.614404 13264 raft_consensus.cc:493] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.615182 13264 raft_consensus.cc:515] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.617483 13264 leader_election.cc:290] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.618649 13181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac66224c363f48a49e1af00db21c47cf" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:01.619076 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac66224c363f48a49e1af00db21c47cf" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:01.619589 13181 raft_consensus.cc:2393] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91c9fa98571441c8959020435c520d0b in current term 1: Already voted for candidate 1802553b3b0245be83aff21b0b5b23d0 in this term.
I20260709 13:05:01.620806 12997 leader_election.cc:304] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91c9fa98571441c8959020435c520d0b; no voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:01.621603 13264 raft_consensus.cc:3060] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.629024 13264 raft_consensus.cc:2749] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:01.630985 13179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa2c897f99564a1eaf2b475b8094cbaf"
dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
mode: GRACEFUL
new_leader_uuid: "91c9fa98571441c8959020435c520d0b"
 from {username='slave'} at 127.0.0.1:34552
I20260709 13:05:01.631812 13179 raft_consensus.cc:606] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Received request to transfer leadership to TS 91c9fa98571441c8959020435c520d0b
I20260709 13:05:01.654356 13264 raft_consensus.cc:493] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.654922 13264 raft_consensus.cc:515] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.656725 13264 leader_election.cc:290] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.657516 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d7b9d3a73b468ea5499974cdfb073e" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:01.657720 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90d7b9d3a73b468ea5499974cdfb073e" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:01.658413 13106 raft_consensus.cc:2393] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 91c9fa98571441c8959020435c520d0b in current term 1: Already voted for candidate 648954f023084ca68a9dc69c1bdbf372 in this term.
I20260709 13:05:01.659451 12996 leader_election.cc:304] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91c9fa98571441c8959020435c520d0b; no voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:01.660152 13264 raft_consensus.cc:3060] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.665588 13264 raft_consensus.cc:2749] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:01.669855 13258 raft_consensus.cc:493] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.670404 13258 raft_consensus.cc:515] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.672200 13258 leader_election.cc:290] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:01.672948 13032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54b6273b4d944500bc90ab3c2aa15c91" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:01.673241 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54b6273b4d944500bc90ab3c2aa15c91" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:01.674031 13106 raft_consensus.cc:2393] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 648954f023084ca68a9dc69c1bdbf372 in current term 1: Already voted for candidate 91c9fa98571441c8959020435c520d0b in this term.
I20260709 13:05:01.674894 13145 leader_election.cc:304] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372; no voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b
I20260709 13:05:01.675557 13258 raft_consensus.cc:3060] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.680737 13258 raft_consensus.cc:2749] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:01.757637 13256 consensus_queue.cc:1048] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:01.765753 13264 consensus_queue.cc:1048] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:01.771517 13267 consensus_queue.cc:1048] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:01.772969 13227 consensus_queue.cc:1048] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:01.773406 13259 consensus_queue.cc:1048] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:01.778832 13031 raft_consensus.cc:3060] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:01.788527 13271 raft_consensus.cc:493] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:01.789215 13271 raft_consensus.cc:515] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:01.791828 13228 consensus_queue.cc:1048] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:01.793862 13260 consensus_queue.cc:1048] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:01.797578 13271 leader_election.cc:290] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:01.797608 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344a32e631c427f9275b5d0b7543901" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:01.809597 13181 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344a32e631c427f9275b5d0b7543901" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:01.813009 12997 leader_election.cc:304] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91c9fa98571441c8959020435c520d0b; no voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:01.819427 13257 consensus_queue.cc:1048] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:01.821447 13257 consensus_queue.cc:1048] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:01.822992 13271 raft_consensus.cc:2749] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:01.832105 13227 consensus_queue.cc:1048] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:01.841645 13257 consensus_queue.cc:1048] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:01.855890 13258 consensus_queue.cc:1048] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:01.858830 13264 consensus_queue.cc:1048] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:01.897028 13255 consensus_queue.cc:1048] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:02.047415 13229 raft_consensus.cc:993] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372: : Instructing follower 91c9fa98571441c8959020435c520d0b to start an election
I20260709 13:05:02.047986 13255 raft_consensus.cc:1081] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Signalling peer 91c9fa98571441c8959020435c520d0b to start an election
I20260709 13:05:02.049650 13032 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aa2c897f99564a1eaf2b475b8094cbaf"
dest_uuid: "91c9fa98571441c8959020435c520d0b"
 from {username='slave'} at 127.0.0.1:35518
I20260709 13:05:02.050230 13032 raft_consensus.cc:493] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:02.050572 13032 raft_consensus.cc:3060] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.055754 13032 raft_consensus.cc:515] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } }
I20260709 13:05:02.057567 13032 leader_election.cc:290] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 2 election: Requested vote from peers 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:02.058713 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2c897f99564a1eaf2b475b8094cbaf" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:02.059213 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa2c897f99564a1eaf2b475b8094cbaf" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:02.059394 13103 raft_consensus.cc:3060] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.059825 13179 raft_consensus.cc:3055] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:02.060122 13179 raft_consensus.cc:740] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Running, Role: LEADER
I20260709 13:05:02.060832 13179 consensus_queue.cc:260] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } }
I20260709 13:05:02.061753 13179 raft_consensus.cc:3060] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:02.065133 13103 raft_consensus.cc:2468] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 2.
I20260709 13:05:02.065938 13105 raft_consensus.cc:1240] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Rejecting Update request from peer 648954f023084ca68a9dc69c1bdbf372 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:02.066532 12996 leader_election.cc:304] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:02.067377 13264 raft_consensus.cc:2804] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:02.067850 13264 raft_consensus.cc:697] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [term 2 LEADER]: Becoming Leader. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:02.067873 13179 raft_consensus.cc:2468] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 2.
I20260709 13:05:02.068718 13264 consensus_queue.cc:237] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } }
I20260709 13:05:02.076202 12930 catalog_manager.cc:5697] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b reported cstate change: term changed from 1 to 2, leader changed from 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195) to 91c9fa98571441c8959020435c520d0b (127.0.107.193). New cstate: current_term: 2 leader_uuid: "91c9fa98571441c8959020435c520d0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:02.471514 13180 raft_consensus.cc:1275] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Refusing update from remote peer 91c9fa98571441c8959020435c520d0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:02.472522 13105 raft_consensus.cc:1275] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Refusing update from remote peer 91c9fa98571441c8959020435c520d0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:02.472961 13271 consensus_queue.cc:1048] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:02.473954 13264 consensus_queue.cc:1048] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:04.684686 12930 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39272:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:05:04.686838 12930 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:05:04.743065 13015 tablet_service.cc:1511] Processing CreateTablet for tablet adb982abf32d42c194842360d6a1fb46 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:04.744923 13096 tablet_service.cc:1511] Processing CreateTablet for tablet adb982abf32d42c194842360d6a1fb46 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:04.745383 13015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adb982abf32d42c194842360d6a1fb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.746433 13096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adb982abf32d42c194842360d6a1fb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.748219 13168 tablet_service.cc:1511] Processing CreateTablet for tablet adb982abf32d42c194842360d6a1fb46 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:05:04.748871 13016 tablet_service.cc:1511] Processing CreateTablet for tablet 7de68b718e464371bc15d4b3b8fd082e (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:04.750264 13168 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adb982abf32d42c194842360d6a1fb46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.756567 13091 tablet_service.cc:1511] Processing CreateTablet for tablet 7de68b718e464371bc15d4b3b8fd082e (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:04.758071 13091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7de68b718e464371bc15d4b3b8fd082e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.760180 13014 tablet_service.cc:1511] Processing CreateTablet for tablet 1862ebf0bab3451c9059f05dd1420b46 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:04.763209 13097 tablet_service.cc:1511] Processing CreateTablet for tablet 1862ebf0bab3451c9059f05dd1420b46 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:04.763685 13169 tablet_service.cc:1511] Processing CreateTablet for tablet 7de68b718e464371bc15d4b3b8fd082e (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:05:04.765050 13169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7de68b718e464371bc15d4b3b8fd082e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.750360 13016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7de68b718e464371bc15d4b3b8fd082e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.769685 13166 tablet_service.cc:1511] Processing CreateTablet for tablet 1862ebf0bab3451c9059f05dd1420b46 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:05:04.770365 13021 tablet_service.cc:1511] Processing CreateTablet for tablet 126207f102f0423a9b67d6b388cc4595 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:04.771703 13021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 126207f102f0423a9b67d6b388cc4595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.773813 13014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1862ebf0bab3451c9059f05dd1420b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.775775 13094 tablet_service.cc:1511] Processing CreateTablet for tablet 126207f102f0423a9b67d6b388cc4595 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:04.777875 13166 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1862ebf0bab3451c9059f05dd1420b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.780184 13164 tablet_service.cc:1511] Processing CreateTablet for tablet 126207f102f0423a9b67d6b388cc4595 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:05:04.781665 13164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 126207f102f0423a9b67d6b388cc4595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.789690 13097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1862ebf0bab3451c9059f05dd1420b46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.790819 13094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 126207f102f0423a9b67d6b388cc4595. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.800872 13089 tablet_service.cc:1511] Processing CreateTablet for tablet f047c82cb27e4832861399c9c33043f7 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:04.802275 13018 tablet_service.cc:1511] Processing CreateTablet for tablet f047c82cb27e4832861399c9c33043f7 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:04.803649 13018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f047c82cb27e4832861399c9c33043f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.803918 13019 tablet_service.cc:1511] Processing CreateTablet for tablet 7a4e244cb05a427dac7505e343db9c46 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:04.805222 13019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a4e244cb05a427dac7505e343db9c46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.802331 13089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f047c82cb27e4832861399c9c33043f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.807055 13163 tablet_service.cc:1511] Processing CreateTablet for tablet f047c82cb27e4832861399c9c33043f7 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:05:04.808420 13163 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f047c82cb27e4832861399c9c33043f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.816397 13093 tablet_service.cc:1511] Processing CreateTablet for tablet 7a4e244cb05a427dac7505e343db9c46 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:04.818085 13093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a4e244cb05a427dac7505e343db9c46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.818352 13167 tablet_service.cc:1511] Processing CreateTablet for tablet 7a4e244cb05a427dac7505e343db9c46 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:05:04.819747 13167 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a4e244cb05a427dac7505e343db9c46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.829799 13022 tablet_service.cc:1511] Processing CreateTablet for tablet a13fa28d81b3452597c141ed18a32b32 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:04.831348 13022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a13fa28d81b3452597c141ed18a32b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.833377 13095 tablet_service.cc:1511] Processing CreateTablet for tablet a13fa28d81b3452597c141ed18a32b32 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:04.834792 13095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a13fa28d81b3452597c141ed18a32b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.837092 13020 tablet_service.cc:1511] Processing CreateTablet for tablet ce559953fc0d4764b0db5595e6fbc8bc (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:04.838553 13020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce559953fc0d4764b0db5595e6fbc8bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.839241 13090 tablet_service.cc:1511] Processing CreateTablet for tablet ce559953fc0d4764b0db5595e6fbc8bc (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:04.840588 13090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce559953fc0d4764b0db5595e6fbc8bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.844107 13165 tablet_service.cc:1511] Processing CreateTablet for tablet ce559953fc0d4764b0db5595e6fbc8bc (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:05:04.844412 13017 tablet_service.cc:1511] Processing CreateTablet for tablet d653af1be1a84530a8f8e3ed2365ce72 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:04.845593 13165 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce559953fc0d4764b0db5595e6fbc8bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.845834 13017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d653af1be1a84530a8f8e3ed2365ce72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.854036 13164 tablet_service.cc:1511] Processing CreateTablet for tablet a13fa28d81b3452597c141ed18a32b32 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:05:04.855646 13164 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a13fa28d81b3452597c141ed18a32b32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.870563 13169 tablet_service.cc:1511] Processing CreateTablet for tablet d653af1be1a84530a8f8e3ed2365ce72 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:04.881718 13169 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d653af1be1a84530a8f8e3ed2365ce72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.829871 13327 tablet_bootstrap.cc:492] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:04.890684 13328 tablet_bootstrap.cc:492] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:04.869560 13096 tablet_service.cc:1511] Processing CreateTablet for tablet d653af1be1a84530a8f8e3ed2365ce72 (DEFAULT_TABLE table=soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:05:04.906647 13096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d653af1be1a84530a8f8e3ed2365ce72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:05:04.917234 13327 tablet_bootstrap.cc:654] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:04.951769 13328 tablet_bootstrap.cc:654] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:04.957602 13329 tablet_bootstrap.cc:492] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:04.965021 13329 tablet_bootstrap.cc:654] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:04.973917 12978 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:05:04.977319 13327 tablet_bootstrap.cc:492] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:04.977993 13327 ts_tablet_manager.cc:1403] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.148s	user 0.027s	sys 0.011s
I20260709 13:05:04.981194 13327 raft_consensus.cc:359] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:04.981993 13327 raft_consensus.cc:385] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:04.982247 13328 tablet_bootstrap.cc:492] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:04.982297 13327 raft_consensus.cc:740] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:04.982965 13328 ts_tablet_manager.cc:1403] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.093s	user 0.021s	sys 0.002s
I20260709 13:05:04.983456 13327 consensus_queue.cc:260] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:04.985996 13327 ts_tablet_manager.cc:1434] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 13:05:04.986435 13328 raft_consensus.cc:359] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:04.987287 13328 raft_consensus.cc:385] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:04.987334 13327 tablet_bootstrap.cc:492] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:04.987708 13328 raft_consensus.cc:740] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:04.989008 13328 consensus_queue.cc:260] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:04.991611 13328 ts_tablet_manager.cc:1434] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:04.992893 13328 tablet_bootstrap.cc:492] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:04.994472 13327 tablet_bootstrap.cc:654] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.001173 13328 tablet_bootstrap.cc:654] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.007156 13314 raft_consensus.cc:493] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.007999 13314 raft_consensus.cc:515] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.009320 13327 tablet_bootstrap.cc:492] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:05.010011 13327 ts_tablet_manager.cc:1403] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.009s
I20260709 13:05:05.011690 13328 tablet_bootstrap.cc:492] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:05.012274 13328 ts_tablet_manager.cc:1403] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.003s
I20260709 13:05:05.012136 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1862ebf0bab3451c9059f05dd1420b46" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:05.013810 13327 raft_consensus.cc:359] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.014763 13327 raft_consensus.cc:385] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.015383 13327 raft_consensus.cc:740] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.016211 13327 consensus_queue.cc:260] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
W20260709 13:05:05.016553 13145 leader_election.cc:343] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:05.016765 13328 raft_consensus.cc:359] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.017869 13328 raft_consensus.cc:385] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.018342 13328 raft_consensus.cc:740] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.019454 13328 consensus_queue.cc:260] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.020195 13314 leader_election.cc:290] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:05.022679 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1862ebf0bab3451c9059f05dd1420b46" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
W20260709 13:05:05.025519 13145 leader_election.cc:343] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:05.026264 13145 leader_election.cc:304] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372; no voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b
I20260709 13:05:05.026988 13327 ts_tablet_manager.cc:1434] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.007s
I20260709 13:05:05.028641 13327 tablet_bootstrap.cc:492] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:05.045491 13314 raft_consensus.cc:2749] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:05.050698 13327 tablet_bootstrap.cc:654] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:05.095570 13145 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.107.194:36541, user_credentials={real_user=slave}} blocked reactor thread for 70132.5us
I20260709 13:05:05.100001 13329 tablet_bootstrap.cc:492] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:05.100140 13328 ts_tablet_manager.cc:1434] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.087s	user 0.041s	sys 0.043s
I20260709 13:05:05.100714 13329 ts_tablet_manager.cc:1403] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.143s	user 0.036s	sys 0.079s
I20260709 13:05:05.101081 13335 raft_consensus.cc:493] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.101460 13328 tablet_bootstrap.cc:492] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:05.101660 13335 raft_consensus.cc:515] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.104113 13329 raft_consensus.cc:359] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.104951 13329 raft_consensus.cc:385] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.105317 13329 raft_consensus.cc:740] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.105903 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb982abf32d42c194842360d6a1fb46" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:05.106274 13329 consensus_queue.cc:260] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.107764 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb982abf32d42c194842360d6a1fb46" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:05.108206 13328 tablet_bootstrap.cc:654] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
W20260709 13:05:05.109555 12997 leader_election.cc:343] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:05.109724 13335 leader_election.cc:290] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
W20260709 13:05:05.110411 12996 leader_election.cc:343] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:05.110913 13327 tablet_bootstrap.cc:492] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:05.110864 12996 leader_election.cc:304] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91c9fa98571441c8959020435c520d0b; no voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:05.112428 13335 raft_consensus.cc:2749] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:05:05.113620 13329 ts_tablet_manager.cc:1434] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:05:05.114337 13327 ts_tablet_manager.cc:1403] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.086s	user 0.022s	sys 0.037s
I20260709 13:05:05.114938 13329 tablet_bootstrap.cc:492] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:05.119124 13327 raft_consensus.cc:359] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.120193 13327 raft_consensus.cc:385] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.120559 13327 raft_consensus.cc:740] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.120690 13328 tablet_bootstrap.cc:492] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:05.121305 13328 ts_tablet_manager.cc:1403] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.009s
I20260709 13:05:05.121456 13327 consensus_queue.cc:260] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.122527 13329 tablet_bootstrap.cc:654] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.124929 13327 ts_tablet_manager.cc:1434] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:05:05.125521 13328 raft_consensus.cc:359] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.126590 13327 tablet_bootstrap.cc:492] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:05.126907 13328 raft_consensus.cc:385] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.127311 13328 raft_consensus.cc:740] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.128221 13328 consensus_queue.cc:260] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.130865 13328 ts_tablet_manager.cc:1434] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:05:05.134240 13327 tablet_bootstrap.cc:654] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.134744 13329 tablet_bootstrap.cc:492] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:05.135352 13329 ts_tablet_manager.cc:1403] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.002s
I20260709 13:05:05.132040 13328 tablet_bootstrap.cc:492] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:05.138820 13329 raft_consensus.cc:359] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.139658 13329 raft_consensus.cc:385] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.140023 13329 raft_consensus.cc:740] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.140959 13329 consensus_queue.cc:260] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.146564 13327 tablet_bootstrap.cc:492] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:05.146579 13329 ts_tablet_manager.cc:1434] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:05:05.147042 13328 tablet_bootstrap.cc:654] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.147394 13327 ts_tablet_manager.cc:1403] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.008s
I20260709 13:05:05.148079 13329 tablet_bootstrap.cc:492] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:05.151551 13315 raft_consensus.cc:493] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.151157 13327 raft_consensus.cc:359] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.152232 13327 raft_consensus.cc:385] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.152110 13315 raft_consensus.cc:515] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.152725 13327 raft_consensus.cc:740] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.153868 13327 consensus_queue.cc:260] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.156019 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de68b718e464371bc15d4b3b8fd082e" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:05.156955 13179 raft_consensus.cc:2468] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 0.
I20260709 13:05:05.157413 13327 ts_tablet_manager.cc:1434] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:05:05.159137 13328 tablet_bootstrap.cc:492] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:05.158987 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de68b718e464371bc15d4b3b8fd082e" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:05.158921 13072 leader_election.cc:304] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
I20260709 13:05:05.160095 13328 ts_tablet_manager.cc:1403] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.028s	user 0.014s	sys 0.007s
I20260709 13:05:05.160650 13227 raft_consensus.cc:2804] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:05.161046 13227 raft_consensus.cc:493] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:05:05.161065 13071 leader_election.cc:343] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:05.161530 13227 raft_consensus.cc:3060] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.162678 13327 tablet_bootstrap.cc:492] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:05.163456 13328 raft_consensus.cc:359] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.164278 13328 raft_consensus.cc:385] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.164580 13328 raft_consensus.cc:740] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.159283 13315 leader_election.cc:290] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.165349 13328 consensus_queue.cc:260] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.168071 13328 ts_tablet_manager.cc:1434] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:05:05.169385 13328 tablet_bootstrap.cc:492] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:05.170553 13327 tablet_bootstrap.cc:654] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.171077 13227 raft_consensus.cc:515] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.178663 13328 tablet_bootstrap.cc:654] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.185217 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de68b718e464371bc15d4b3b8fd082e" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b"
W20260709 13:05:05.187500 13071 leader_election.cc:343] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:05:05.188822 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de68b718e464371bc15d4b3b8fd082e" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:05.190106 13179 raft_consensus.cc:3060] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.191710 13328 tablet_bootstrap.cc:492] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:05.192314 13328 ts_tablet_manager.cc:1403] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.009s
I20260709 13:05:05.193048 13329 tablet_bootstrap.cc:654] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.189045 13327 tablet_bootstrap.cc:492] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:05.194062 13327 ts_tablet_manager.cc:1403] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.032s	user 0.016s	sys 0.005s
I20260709 13:05:05.195576 13328 raft_consensus.cc:359] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.196375 13328 raft_consensus.cc:385] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.196893 13328 raft_consensus.cc:740] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.197299 13327 raft_consensus.cc:359] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.197757 13328 consensus_queue.cc:260] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.198354 13327 raft_consensus.cc:385] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.198689 13327 raft_consensus.cc:740] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.199466 13179 raft_consensus.cc:2468] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 1.
I20260709 13:05:05.200711 13227 leader_election.cc:290] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.201033 13072 leader_election.cc:304] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 91c9fa98571441c8959020435c520d0b
I20260709 13:05:05.200874 13328 ts_tablet_manager.cc:1434] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 13:05:05.199484 13327 consensus_queue.cc:260] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.202826 13328 tablet_bootstrap.cc:492] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:05.203760 13227 raft_consensus.cc:2804] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:05.205116 13227 raft_consensus.cc:697] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Becoming Leader. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:05.205469 13327 ts_tablet_manager.cc:1434] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 13:05:05.206768 13327 tablet_bootstrap.cc:492] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:05.211908 13328 tablet_bootstrap.cc:654] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.215184 13329 tablet_bootstrap.cc:492] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:05.206382 13227 consensus_queue.cc:237] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.215808 13329 ts_tablet_manager.cc:1403] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.068s	user 0.011s	sys 0.002s
I20260709 13:05:05.216298 13327 tablet_bootstrap.cc:654] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.219260 13329 raft_consensus.cc:359] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.220104 13329 raft_consensus.cc:385] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.220400 13329 raft_consensus.cc:740] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.221266 13329 consensus_queue.cc:260] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.229807 13329 ts_tablet_manager.cc:1434] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.014s	user 0.011s	sys 0.001s
I20260709 13:05:05.231001 13329 tablet_bootstrap.cc:492] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:05.231827 13327 tablet_bootstrap.cc:492] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:05.232383 13327 ts_tablet_manager.cc:1403] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.012s
I20260709 13:05:05.235666 13327 raft_consensus.cc:359] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.236517 13327 raft_consensus.cc:385] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.236881 13327 raft_consensus.cc:740] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.237751 13327 consensus_queue.cc:260] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.237607 12929 catalog_manager.cc:5697] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "1802553b3b0245be83aff21b0b5b23d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.239197 13329 tablet_bootstrap.cc:654] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.239331 13328 tablet_bootstrap.cc:492] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:05.240177 13328 ts_tablet_manager.cc:1403] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.007s
I20260709 13:05:05.240751 13327 ts_tablet_manager.cc:1434] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:05:05.242422 13327 tablet_bootstrap.cc:492] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:05.243853 13328 raft_consensus.cc:359] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.245034 13328 raft_consensus.cc:385] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.245461 13328 raft_consensus.cc:740] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.246541 13328 consensus_queue.cc:260] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.253296 13328 ts_tablet_manager.cc:1434] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:05:05.254424 13327 tablet_bootstrap.cc:654] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.254572 13328 tablet_bootstrap.cc:492] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:05.259837 13329 tablet_bootstrap.cc:492] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:05.260427 13329 ts_tablet_manager.cc:1403] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.030s	user 0.008s	sys 0.003s
I20260709 13:05:05.262729 13328 tablet_bootstrap.cc:654] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.263078 13327 tablet_bootstrap.cc:492] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:05.263861 13327 ts_tablet_manager.cc:1403] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.001s
I20260709 13:05:05.263784 13329 raft_consensus.cc:359] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.264835 13329 raft_consensus.cc:385] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.265169 13329 raft_consensus.cc:740] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.266328 13329 consensus_queue.cc:260] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.267928 13327 raft_consensus.cc:359] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.268838 13327 raft_consensus.cc:385] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.269322 13327 raft_consensus.cc:740] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.270171 13327 consensus_queue.cc:260] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.270462 13329 ts_tablet_manager.cc:1434] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.005s
I20260709 13:05:05.274670 13329 tablet_bootstrap.cc:492] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:05.277599 13327 ts_tablet_manager.cc:1434] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.003s
I20260709 13:05:05.278903 13327 tablet_bootstrap.cc:492] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:05.279546 13349 raft_consensus.cc:493] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.280113 13349 raft_consensus.cc:515] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.282409 13349 leader_election.cc:290] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.283753 13329 tablet_bootstrap.cc:654] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.283623 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "126207f102f0423a9b67d6b388cc4595" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:05.284027 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "126207f102f0423a9b67d6b388cc4595" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:05.284716 13179 raft_consensus.cc:2468] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 0.
I20260709 13:05:05.284637 13031 raft_consensus.cc:2468] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 0.
I20260709 13:05:05.286281 13072 leader_election.cc:304] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
I20260709 13:05:05.287230 13349 raft_consensus.cc:2804] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:05.287603 13349 raft_consensus.cc:493] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.287940 13349 raft_consensus.cc:3060] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.303532 13329 tablet_bootstrap.cc:492] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:05.304164 13349 raft_consensus.cc:515] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.304478 13329 ts_tablet_manager.cc:1403] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.004s
I20260709 13:05:05.307616 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "126207f102f0423a9b67d6b388cc4595" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:05.307766 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "126207f102f0423a9b67d6b388cc4595" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:05.308318 13031 raft_consensus.cc:3060] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.308388 13179 raft_consensus.cc:3060] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.309362 13329 raft_consensus.cc:359] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.310068 13349 leader_election.cc:290] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.310176 13329 raft_consensus.cc:385] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.301252 13327 tablet_bootstrap.cc:654] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.310654 13329 raft_consensus.cc:740] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.311614 13328 tablet_bootstrap.cc:492] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:05.312168 13328 ts_tablet_manager.cc:1403] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.058s	user 0.020s	sys 0.033s
I20260709 13:05:05.311769 13329 consensus_queue.cc:260] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.314821 13329 ts_tablet_manager.cc:1434] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.009s
I20260709 13:05:05.315948 13329 tablet_bootstrap.cc:492] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:05.316490 13328 raft_consensus.cc:359] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.317358 13328 raft_consensus.cc:385] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.317731 13328 raft_consensus.cc:740] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.318627 13328 consensus_queue.cc:260] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.321636 13328 ts_tablet_manager.cc:1434] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.003s
I20260709 13:05:05.322904 13328 tablet_bootstrap.cc:492] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:05.323786 13329 tablet_bootstrap.cc:654] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.326747 13179 raft_consensus.cc:2468] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 1.
I20260709 13:05:05.328926 13327 tablet_bootstrap.cc:492] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:05.329133 13072 leader_election.cc:304] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
I20260709 13:05:05.329490 13327 ts_tablet_manager.cc:1403] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.051s	user 0.009s	sys 0.010s
I20260709 13:05:05.327729 13031 raft_consensus.cc:2468] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 1.
I20260709 13:05:05.330013 13349 raft_consensus.cc:2804] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:05.330595 13349 raft_consensus.cc:697] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Becoming Leader. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:05.330746 13328 tablet_bootstrap.cc:654] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.331539 13349 consensus_queue.cc:237] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.332587 13327 raft_consensus.cc:359] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.333482 13327 raft_consensus.cc:385] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.333765 13327 raft_consensus.cc:740] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.334448 13327 consensus_queue.cc:260] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.336474 13327 ts_tablet_manager.cc:1434] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:05:05.337793 13327 tablet_bootstrap.cc:492] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b: Bootstrap starting.
I20260709 13:05:05.338783 13329 tablet_bootstrap.cc:492] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:05.340914 13328 tablet_bootstrap.cc:492] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:05.341466 13329 ts_tablet_manager.cc:1403] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.008s
I20260709 13:05:05.344053 13327 tablet_bootstrap.cc:654] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.341673 13328 ts_tablet_manager.cc:1403] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.019s	user 0.005s	sys 0.012s
I20260709 13:05:05.345952 13329 raft_consensus.cc:359] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.346798 13329 raft_consensus.cc:385] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.347165 13329 raft_consensus.cc:740] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.347478 13328 raft_consensus.cc:359] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.348603 13328 raft_consensus.cc:385] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.348235 13329 consensus_queue.cc:260] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.349026 13328 raft_consensus.cc:740] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.350060 13328 consensus_queue.cc:260] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.351898 13329 ts_tablet_manager.cc:1434] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:05:05.352805 13328 ts_tablet_manager.cc:1434] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:05:05.353227 13329 tablet_bootstrap.cc:492] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:05.353956 13328 tablet_bootstrap.cc:492] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372: Bootstrap starting.
I20260709 13:05:05.360962 13329 tablet_bootstrap.cc:654] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.362170 13327 tablet_bootstrap.cc:492] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b: No bootstrap required, opened a new log
I20260709 13:05:05.362826 13327 ts_tablet_manager.cc:1403] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b: Time spent bootstrapping tablet: real 0.025s	user 0.004s	sys 0.014s
I20260709 13:05:05.360965 13328 tablet_bootstrap.cc:654] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.365959 13327 raft_consensus.cc:359] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.366850 13327 raft_consensus.cc:385] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.367189 13327 raft_consensus.cc:740] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.368045 13327 consensus_queue.cc:260] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.378091 13327 ts_tablet_manager.cc:1434] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:05:05.381747 13328 tablet_bootstrap.cc:492] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372: No bootstrap required, opened a new log
I20260709 13:05:05.381747 13329 tablet_bootstrap.cc:492] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:05.382632 13328 ts_tablet_manager.cc:1403] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372: Time spent bootstrapping tablet: real 0.029s	user 0.021s	sys 0.003s
I20260709 13:05:05.382665 13329 ts_tablet_manager.cc:1403] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.008s
I20260709 13:05:05.385958 13329 raft_consensus.cc:359] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.388513 13329 raft_consensus.cc:385] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.388943 13329 raft_consensus.cc:740] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.389880 13329 consensus_queue.cc:260] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.387737 13328 raft_consensus.cc:359] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.391060 13328 raft_consensus.cc:385] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.390491 12928 catalog_manager.cc:5697] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "1802553b3b0245be83aff21b0b5b23d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.391439 13328 raft_consensus.cc:740] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.392242 13328 consensus_queue.cc:260] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.395232 13329 ts_tablet_manager.cc:1434] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:05:05.396430 13329 tablet_bootstrap.cc:492] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:05.396948 13328 ts_tablet_manager.cc:1434] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.002s
I20260709 13:05:05.402558 13329 tablet_bootstrap.cc:654] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.407775 13329 tablet_bootstrap.cc:492] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:05.408401 13329 ts_tablet_manager.cc:1403] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.012s	user 0.011s	sys 0.000s
I20260709 13:05:05.412177 13329 raft_consensus.cc:359] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.412993 13329 raft_consensus.cc:385] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.413311 13329 raft_consensus.cc:740] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.414163 13329 consensus_queue.cc:260] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.416657 13329 ts_tablet_manager.cc:1434] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.002s
I20260709 13:05:05.418098 13329 tablet_bootstrap.cc:492] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0: Bootstrap starting.
I20260709 13:05:05.423897 13329 tablet_bootstrap.cc:654] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0: Neither blocks nor log segments found. Creating new log.
I20260709 13:05:05.431051 13329 tablet_bootstrap.cc:492] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0: No bootstrap required, opened a new log
I20260709 13:05:05.431512 13329 ts_tablet_manager.cc:1403] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 13:05:05.433822 13329 raft_consensus.cc:359] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.434479 13329 raft_consensus.cc:385] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:05:05.434708 13329 raft_consensus.cc:740] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Initialized, Role: FOLLOWER
I20260709 13:05:05.435312 13329 consensus_queue.cc:260] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.437175 13329 ts_tablet_manager.cc:1434] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:05:05.448238 13336 raft_consensus.cc:493] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.448865 13336 raft_consensus.cc:515] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.451467 13336 leader_election.cc:290] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:05.452108 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f047c82cb27e4832861399c9c33043f7" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:05.452469 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f047c82cb27e4832861399c9c33043f7" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:05.452894 13031 raft_consensus.cc:2468] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 0.
I20260709 13:05:05.453106 13105 raft_consensus.cc:2468] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 0.
I20260709 13:05:05.454100 13145 leader_election.cc:304] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:05.454804 13336 raft_consensus.cc:2804] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:05.455206 13336 raft_consensus.cc:493] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.455497 13336 raft_consensus.cc:3060] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.461391 13336 raft_consensus.cc:515] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.462988 13336 leader_election.cc:290] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:05.464182 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f047c82cb27e4832861399c9c33043f7" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:05.464254 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f047c82cb27e4832861399c9c33043f7" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:05.465087 13031 raft_consensus.cc:3060] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.465359 13105 raft_consensus.cc:3060] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.465301 13336 raft_consensus.cc:493] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.466180 13336 raft_consensus.cc:515] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.467104 13335 raft_consensus.cc:493] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.467641 13335 raft_consensus.cc:515] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.468293 13336 leader_election.cc:290] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:05.470203 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce559953fc0d4764b0db5595e6fbc8bc" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:05.470430 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13fa28d81b3452597c141ed18a32b32" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:05.470901 13103 raft_consensus.cc:2468] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 0.
I20260709 13:05:05.470935 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13fa28d81b3452597c141ed18a32b32" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:05.471030 13106 raft_consensus.cc:2468] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 0.
I20260709 13:05:05.471885 13179 raft_consensus.cc:2468] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 0.
I20260709 13:05:05.469728 13335 leader_election.cc:290] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.472584 12996 leader_election.cc:304] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:05.473539 13335 raft_consensus.cc:2804] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:05.473763 13105 raft_consensus.cc:2468] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 1.
I20260709 13:05:05.473954 13335 raft_consensus.cc:493] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.474428 13335 raft_consensus.cc:3060] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.474745 13032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce559953fc0d4764b0db5595e6fbc8bc" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:05.475431 13032 raft_consensus.cc:2468] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 0.
I20260709 13:05:05.473881 13145 leader_election.cc:304] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
I20260709 13:05:05.477101 13145 leader_election.cc:304] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
I20260709 13:05:05.478351 13337 raft_consensus.cc:2804] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:05.478914 13337 raft_consensus.cc:493] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.479238 13337 raft_consensus.cc:3060] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.480105 13336 raft_consensus.cc:2804] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:05.480686 13336 raft_consensus.cc:697] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Becoming Leader. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Running, Role: LEADER
I20260709 13:05:05.481788 13031 raft_consensus.cc:2468] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 1.
I20260709 13:05:05.481595 13336 consensus_queue.cc:237] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.491372 13335 raft_consensus.cc:515] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.491523 13337 raft_consensus.cc:515] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.492801 13264 raft_consensus.cc:493] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.493393 13264 raft_consensus.cc:515] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.494112 13335 leader_election.cc:290] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Requested vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.494922 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13fa28d81b3452597c141ed18a32b32" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:05.495555 13105 raft_consensus.cc:3060] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.495786 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13fa28d81b3452597c141ed18a32b32" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:05.496380 13179 raft_consensus.cc:3060] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.495641 13264 leader_election.cc:290] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.497859 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1862ebf0bab3451c9059f05dd1420b46" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:05.498631 13106 raft_consensus.cc:2468] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 0.
I20260709 13:05:05.499684 12996 leader_election.cc:304] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:05.500474 13264 raft_consensus.cc:2804] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:05.500919 13264 raft_consensus.cc:493] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.501317 13264 raft_consensus.cc:3060] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.501631 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1862ebf0bab3451c9059f05dd1420b46" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:05.501305 12928 catalog_manager.cc:5697] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 reported cstate change: term changed from 0 to 1, leader changed from <none> to 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "648954f023084ca68a9dc69c1bdbf372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:05.502411 13180 raft_consensus.cc:2468] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 0.
I20260709 13:05:05.507359 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce559953fc0d4764b0db5595e6fbc8bc" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:05.508105 13031 raft_consensus.cc:3060] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.508030 13103 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce559953fc0d4764b0db5595e6fbc8bc" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:05.508946 13103 raft_consensus.cc:3060] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.511814 13264 raft_consensus.cc:515] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.515085 13264 leader_election.cc:290] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Requested vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.504029 13179 raft_consensus.cc:2468] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 1.
I20260709 13:05:05.515384 13104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1862ebf0bab3451c9059f05dd1420b46" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:05.516090 13104 raft_consensus.cc:3060] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.517675 13031 raft_consensus.cc:2468] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 1.
I20260709 13:05:05.518155 13103 raft_consensus.cc:2468] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 1.
I20260709 13:05:05.518253 12997 leader_election.cc:304] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:05.519585 13264 raft_consensus.cc:2804] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:05.519626 13145 leader_election.cc:304] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:05.520471 13105 raft_consensus.cc:2468] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 1.
I20260709 13:05:05.521273 13336 raft_consensus.cc:2804] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:05.521983 13336 raft_consensus.cc:697] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Becoming Leader. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Running, Role: LEADER
I20260709 13:05:05.522825 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1862ebf0bab3451c9059f05dd1420b46" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:05.522998 13336 consensus_queue.cc:237] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.507447 13337 leader_election.cc:290] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 1 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:05.523842 13179 raft_consensus.cc:3060] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.529147 13335 raft_consensus.cc:493] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.529763 13335 raft_consensus.cc:515] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.530519 13104 raft_consensus.cc:2468] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 1.
I20260709 13:05:05.532013 13335 leader_election.cc:290] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.532183 12996 leader_election.cc:304] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:05.534582 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e244cb05a427dac7505e343db9c46" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:05.534657 13335 raft_consensus.cc:2804] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:05.535465 13105 raft_consensus.cc:2468] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 0.
I20260709 13:05:05.535959 13335 raft_consensus.cc:697] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Becoming Leader. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:05.537344 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e244cb05a427dac7505e343db9c46" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:05.538156 13180 raft_consensus.cc:2468] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 0.
I20260709 13:05:05.537796 13335 consensus_queue.cc:237] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.539834 12996 leader_election.cc:304] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:05.543831 13264 raft_consensus.cc:697] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Becoming Leader. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:05.544968 13264 consensus_queue.cc:237] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.546185 13369 raft_consensus.cc:2804] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:05.546205 13179 raft_consensus.cc:2468] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 1.
I20260709 13:05:05.546885 13369 raft_consensus.cc:493] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.548003 13369 raft_consensus.cc:3060] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.552824 12928 catalog_manager.cc:5697] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 reported cstate change: term changed from 0 to 1, leader changed from <none> to 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195). New cstate: current_term: 1 leader_uuid: "648954f023084ca68a9dc69c1bdbf372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:05.555341 13369 raft_consensus.cc:515] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.558640 13104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e244cb05a427dac7505e343db9c46" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:05.559332 13104 raft_consensus.cc:3060] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.560195 13369 leader_election.cc:290] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Requested vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.562184 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e244cb05a427dac7505e343db9c46" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:05.562940 13179 raft_consensus.cc:3060] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.566596 12929 catalog_manager.cc:5697] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 91c9fa98571441c8959020435c520d0b (127.0.107.193). New cstate: current_term: 1 leader_uuid: "91c9fa98571441c8959020435c520d0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.571130 13367 raft_consensus.cc:493] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.571841 13179 raft_consensus.cc:2468] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 1.
I20260709 13:05:05.571730 13367 raft_consensus.cc:515] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.573372 13104 raft_consensus.cc:2468] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 1.
I20260709 13:05:05.574198 13367 leader_election.cc:290] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.575325 12996 leader_election.cc:304] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:05.575636 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d653af1be1a84530a8f8e3ed2365ce72" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:05.576193 13367 raft_consensus.cc:2804] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:05.576481 13180 raft_consensus.cc:2468] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 0.
I20260709 13:05:05.576699 13367 raft_consensus.cc:697] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Becoming Leader. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:05.578249 13367 consensus_queue.cc:237] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.578567 12997 leader_election.cc:304] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:05.579702 13335 raft_consensus.cc:2804] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:05.580093 13335 raft_consensus.cc:493] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.580192 13104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d653af1be1a84530a8f8e3ed2365ce72" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:05.580500 13335 raft_consensus.cc:3060] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.581605 13104 raft_consensus.cc:2468] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 0.
I20260709 13:05:05.590189 13335 raft_consensus.cc:515] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.592144 13335 leader_election.cc:290] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Requested vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.593385 13104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d653af1be1a84530a8f8e3ed2365ce72" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:05.593393 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d653af1be1a84530a8f8e3ed2365ce72" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:05.594295 13104 raft_consensus.cc:3060] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.594404 13179 raft_consensus.cc:3060] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.593672 12929 catalog_manager.cc:5697] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 91c9fa98571441c8959020435c520d0b (127.0.107.193). New cstate: current_term: 1 leader_uuid: "91c9fa98571441c8959020435c520d0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.595732 12929 catalog_manager.cc:5697] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 91c9fa98571441c8959020435c520d0b (127.0.107.193). New cstate: current_term: 1 leader_uuid: "91c9fa98571441c8959020435c520d0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.601374 13104 raft_consensus.cc:2468] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 1.
I20260709 13:05:05.601593 13179 raft_consensus.cc:2468] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 1.
I20260709 13:05:05.602967 12997 leader_election.cc:304] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:05.603900 13335 raft_consensus.cc:2804] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:05.604387 13335 raft_consensus.cc:697] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Becoming Leader. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:05.605165 13335 consensus_queue.cc:237] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.609181 13349 raft_consensus.cc:493] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.609866 13349 raft_consensus.cc:515] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.612578 13349 leader_election.cc:290] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.613314 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb982abf32d42c194842360d6a1fb46" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b" is_pre_election: true
I20260709 13:05:05.614027 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb982abf32d42c194842360d6a1fb46" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:05.614138 13031 raft_consensus.cc:2468] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 0.
I20260709 13:05:05.614737 13179 raft_consensus.cc:2468] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 0.
I20260709 13:05:05.615471 13071 leader_election.cc:304] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:05.615334 12929 catalog_manager.cc:5697] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 91c9fa98571441c8959020435c520d0b (127.0.107.193). New cstate: current_term: 1 leader_uuid: "91c9fa98571441c8959020435c520d0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.616417 13349 raft_consensus.cc:2804] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:05:05.617506 13349 raft_consensus.cc:493] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:05:05.617877 13349 raft_consensus.cc:3060] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.623725 13349 raft_consensus.cc:515] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.625860 13349 leader_election.cc:290] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.626948 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb982abf32d42c194842360d6a1fb46" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:05.627249 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb982abf32d42c194842360d6a1fb46" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:05.627707 13031 raft_consensus.cc:3060] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.627914 13179 raft_consensus.cc:3060] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.629186 13349 consensus_queue.cc:1048] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:05.635044 13031 raft_consensus.cc:2468] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 1.
I20260709 13:05:05.636768 13071 leader_election.cc:304] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:05.638628 13315 raft_consensus.cc:2804] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:05:05.639117 13315 raft_consensus.cc:697] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Becoming Leader. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:05.639482 13179 raft_consensus.cc:2468] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 1.
I20260709 13:05:05.640015 13315 consensus_queue.cc:237] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.647011 13031 raft_consensus.cc:3060] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:05:05.656940 12929 catalog_manager.cc:5697] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194). New cstate: current_term: 1 leader_uuid: "1802553b3b0245be83aff21b0b5b23d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:05.661916 13349 consensus_queue.cc:1048] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:05.662461 13335 raft_consensus.cc:493] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:05:05.663017 13335 raft_consensus.cc:515] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:05.666724 13104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de68b718e464371bc15d4b3b8fd082e" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1802553b3b0245be83aff21b0b5b23d0" is_pre_election: true
I20260709 13:05:05.666774 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de68b718e464371bc15d4b3b8fd082e" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "648954f023084ca68a9dc69c1bdbf372" is_pre_election: true
I20260709 13:05:05.665206 13335 leader_election.cc:290] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:05.672734 12997 leader_election.cc:304] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91c9fa98571441c8959020435c520d0b; no voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:05.673944 13335 raft_consensus.cc:2749] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:05:05.834369 13349 consensus_queue.cc:1048] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:05.847128 13315 consensus_queue.cc:1048] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:05.895679 13104 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "adb982abf32d42c194842360d6a1fb46"
dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
mode: GRACEFUL
new_leader_uuid: "91c9fa98571441c8959020435c520d0b"
 from {username='slave'} at 127.0.0.1:34098
I20260709 13:05:05.896519 13104 raft_consensus.cc:606] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Received request to transfer leadership to TS 91c9fa98571441c8959020435c520d0b
I20260709 13:05:05.903769 13104 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7de68b718e464371bc15d4b3b8fd082e"
dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
mode: GRACEFUL
new_leader_uuid: "91c9fa98571441c8959020435c520d0b"
 from {username='slave'} at 127.0.0.1:34098
I20260709 13:05:05.904434 13104 raft_consensus.cc:606] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Received request to transfer leadership to TS 91c9fa98571441c8959020435c520d0b
I20260709 13:05:05.912202 13336 consensus_queue.cc:1048] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:05.913986 13104 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "126207f102f0423a9b67d6b388cc4595"
dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
mode: GRACEFUL
new_leader_uuid: "91c9fa98571441c8959020435c520d0b"
 from {username='slave'} at 127.0.0.1:34098
I20260709 13:05:05.914623 13104 raft_consensus.cc:606] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Received request to transfer leadership to TS 91c9fa98571441c8959020435c520d0b
I20260709 13:05:05.921231 13179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f047c82cb27e4832861399c9c33043f7"
dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
mode: GRACEFUL
new_leader_uuid: "1802553b3b0245be83aff21b0b5b23d0"
 from {username='slave'} at 127.0.0.1:34552
I20260709 13:05:05.922020 13179 raft_consensus.cc:606] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Received request to transfer leadership to TS 1802553b3b0245be83aff21b0b5b23d0
I20260709 13:05:05.934229 13337 consensus_queue.cc:1048] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:05.948289 13031 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7a4e244cb05a427dac7505e343db9c46"
dest_uuid: "91c9fa98571441c8959020435c520d0b"
mode: GRACEFUL
new_leader_uuid: "1802553b3b0245be83aff21b0b5b23d0"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:05:05.948913 13031 raft_consensus.cc:606] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Received request to transfer leadership to TS 1802553b3b0245be83aff21b0b5b23d0
I20260709 13:05:05.955196 13031 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a13fa28d81b3452597c141ed18a32b32"
dest_uuid: "91c9fa98571441c8959020435c520d0b"
mode: GRACEFUL
new_leader_uuid: "1802553b3b0245be83aff21b0b5b23d0"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:05:05.955843 13031 raft_consensus.cc:606] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Received request to transfer leadership to TS 1802553b3b0245be83aff21b0b5b23d0
I20260709 13:05:05.956473 13335 consensus_queue.cc:1048] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:05.961558 13179 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce559953fc0d4764b0db5595e6fbc8bc"
dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
mode: GRACEFUL
new_leader_uuid: "1802553b3b0245be83aff21b0b5b23d0"
 from {username='slave'} at 127.0.0.1:34552
I20260709 13:05:05.962179 13179 raft_consensus.cc:606] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Received request to transfer leadership to TS 1802553b3b0245be83aff21b0b5b23d0
I20260709 13:05:05.965242 13335 consensus_queue.cc:1048] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:05.968364 13031 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d653af1be1a84530a8f8e3ed2365ce72"
dest_uuid: "91c9fa98571441c8959020435c520d0b"
mode: GRACEFUL
new_leader_uuid: "648954f023084ca68a9dc69c1bdbf372"
 from {username='slave'} at 127.0.0.1:38478
I20260709 13:05:05.968948 13031 raft_consensus.cc:606] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Received request to transfer leadership to TS 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:05.981029 13335 consensus_queue.cc:1048] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:05.991276 13369 raft_consensus.cc:993] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b: : Instructing follower 1802553b3b0245be83aff21b0b5b23d0 to start an election
I20260709 13:05:05.991915 13369 raft_consensus.cc:1081] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Signalling peer 1802553b3b0245be83aff21b0b5b23d0 to start an election
I20260709 13:05:05.996863 13103 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a13fa28d81b3452597c141ed18a32b32"
dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
 from {username='slave'} at 127.0.0.1:43196
I20260709 13:05:05.997587 13103 raft_consensus.cc:493] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:05.998009 13103 raft_consensus.cc:3060] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.005098 13335 consensus_queue.cc:1048] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:06.006340 13103 raft_consensus.cc:515] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.013196 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13fa28d81b3452597c141ed18a32b32" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:06.013581 13103 leader_election.cc:290] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 2 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:06.013979 13031 raft_consensus.cc:3055] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:06.014325 13031 raft_consensus.cc:740] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:06.014829 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13fa28d81b3452597c141ed18a32b32" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:06.015213 13031 consensus_queue.cc:260] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.016474 13031 raft_consensus.cc:3060] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.026859 13031 raft_consensus.cc:2468] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 2.
I20260709 13:05:06.028220 13071 leader_election.cc:304] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:06.029846 13315 raft_consensus.cc:2804] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:06.030485 13315 raft_consensus.cc:697] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 LEADER]: Becoming Leader. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:06.032024 13315 consensus_queue.cc:237] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.036469 13337 consensus_queue.cc:1048] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:06.039469 13264 consensus_queue.cc:1048] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:06.041339 12929 catalog_manager.cc:5697] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 reported cstate change: term changed from 1 to 2, leader changed from 91c9fa98571441c8959020435c520d0b (127.0.107.193) to 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "1802553b3b0245be83aff21b0b5b23d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:06.050926 13385 consensus_queue.cc:1048] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:06.059619 13367 consensus_queue.cc:1048] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:06.076176 13264 raft_consensus.cc:993] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b: : Instructing follower 1802553b3b0245be83aff21b0b5b23d0 to start an election
I20260709 13:05:06.076876 13264 raft_consensus.cc:1081] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Signalling peer 1802553b3b0245be83aff21b0b5b23d0 to start an election
I20260709 13:05:06.076243 13337 raft_consensus.cc:993] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372: : Instructing follower 1802553b3b0245be83aff21b0b5b23d0 to start an election
I20260709 13:05:06.078035 13337 raft_consensus.cc:1081] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Signalling peer 1802553b3b0245be83aff21b0b5b23d0 to start an election
I20260709 13:05:06.079030 13105 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7a4e244cb05a427dac7505e343db9c46"
dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
 from {username='slave'} at 127.0.0.1:43196
I20260709 13:05:06.079695 13105 raft_consensus.cc:493] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:06.080072 13105 raft_consensus.cc:3060] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.080258 13106 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f047c82cb27e4832861399c9c33043f7"
dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
 from {username='slave'} at 127.0.0.1:43180
I20260709 13:05:06.080873 13106 raft_consensus.cc:493] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:06.081234 13106 raft_consensus.cc:3060] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.088476 13105 raft_consensus.cc:515] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.089784 13106 raft_consensus.cc:515] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.090927 13104 raft_consensus.cc:1240] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Rejecting Update request from peer 648954f023084ca68a9dc69c1bdbf372 for earlier term 1. Current term is 2. Ops: []
I20260709 13:05:06.092805 13031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f047c82cb27e4832861399c9c33043f7" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:06.094271 13314 consensus_queue.cc:1059] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:05:06.094612 13032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e244cb05a427dac7505e343db9c46" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:06.095185 13336 raft_consensus.cc:3055] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:06.095973 13336 raft_consensus.cc:740] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Running, Role: LEADER
I20260709 13:05:06.096232 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7a4e244cb05a427dac7505e343db9c46" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:06.095382 13032 raft_consensus.cc:3055] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:06.096832 13180 raft_consensus.cc:3060] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.096778 13336 consensus_queue.cc:260] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.097585 13106 leader_election.cc:290] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 2 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:06.098068 13105 leader_election.cc:290] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 2 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:06.098269 13336 raft_consensus.cc:3060] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.096894 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f047c82cb27e4832861399c9c33043f7" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:06.094643 13031 raft_consensus.cc:3060] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.097026 13032 raft_consensus.cc:740] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:06.101164 13032 consensus_queue.cc:260] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.102536 13032 raft_consensus.cc:3060] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.106995 13031 raft_consensus.cc:2468] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 2.
I20260709 13:05:06.108408 13071 leader_election.cc:304] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:06.109274 13315 raft_consensus.cc:2804] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:06.109795 13315 raft_consensus.cc:697] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 LEADER]: Becoming Leader. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:06.110481 13315 consensus_queue.cc:237] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.113981 13179 raft_consensus.cc:2468] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 2.
I20260709 13:05:06.117249 13180 raft_consensus.cc:2468] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 2.
I20260709 13:05:06.118577 13032 raft_consensus.cc:2468] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 2.
I20260709 13:05:06.118654 13072 leader_election.cc:304] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
I20260709 13:05:06.119720 13315 raft_consensus.cc:2804] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:06.120277 13315 raft_consensus.cc:697] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 LEADER]: Becoming Leader. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:06.119810 12929 catalog_manager.cc:5697] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 reported cstate change: term changed from 1 to 2, leader changed from 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195) to 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "1802553b3b0245be83aff21b0b5b23d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:06.121078 13315 consensus_queue.cc:237] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.132828 12929 catalog_manager.cc:5697] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 reported cstate change: term changed from 1 to 2, leader changed from 91c9fa98571441c8959020435c520d0b (127.0.107.193) to 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "1802553b3b0245be83aff21b0b5b23d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:06.180487 13349 raft_consensus.cc:993] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0: : Instructing follower 91c9fa98571441c8959020435c520d0b to start an election
I20260709 13:05:06.181066 13315 raft_consensus.cc:1081] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Signalling peer 91c9fa98571441c8959020435c520d0b to start an election
I20260709 13:05:06.182780 13032 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7de68b718e464371bc15d4b3b8fd082e"
dest_uuid: "91c9fa98571441c8959020435c520d0b"
 from {username='slave'} at 127.0.0.1:35530
I20260709 13:05:06.183387 13032 raft_consensus.cc:493] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:06.183764 13032 raft_consensus.cc:3060] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.190235 13032 raft_consensus.cc:515] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.192035 13032 leader_election.cc:290] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 2 election: Requested vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:06.192854 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de68b718e464371bc15d4b3b8fd082e" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:06.193121 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7de68b718e464371bc15d4b3b8fd082e" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:06.193589 13106 raft_consensus.cc:3055] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:06.193845 13180 raft_consensus.cc:3060] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.193984 13106 raft_consensus.cc:740] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:06.194845 13106 consensus_queue.cc:260] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.195839 13106 raft_consensus.cc:3060] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.200397 13180 raft_consensus.cc:2468] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 2.
I20260709 13:05:06.200897 13264 consensus_queue.cc:1048] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:06.201807 12997 leader_election.cc:304] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:06.210047 13335 raft_consensus.cc:993] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b: : Instructing follower 648954f023084ca68a9dc69c1bdbf372 to start an election
I20260709 13:05:06.202584 13264 raft_consensus.cc:2804] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:06.212481 13335 raft_consensus.cc:1081] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Signalling peer 648954f023084ca68a9dc69c1bdbf372 to start an election
I20260709 13:05:06.212914 13264 raft_consensus.cc:697] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 2 LEADER]: Becoming Leader. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:06.215117 13264 consensus_queue.cc:237] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.202551 13106 raft_consensus.cc:2468] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 2.
I20260709 13:05:06.219805 13180 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d653af1be1a84530a8f8e3ed2365ce72"
dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
 from {username='slave'} at 127.0.0.1:34540
I20260709 13:05:06.220553 13180 raft_consensus.cc:493] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:06.221168 13180 raft_consensus.cc:3060] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.221521 13264 consensus_queue.cc:1048] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:06.226122 13315 consensus_queue.cc:1048] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:06.229061 12930 catalog_manager.cc:5697] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b reported cstate change: term changed from 1 to 2, leader changed from 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194) to 91c9fa98571441c8959020435c520d0b (127.0.107.193). New cstate: current_term: 2 leader_uuid: "91c9fa98571441c8959020435c520d0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:06.238271 13180 raft_consensus.cc:515] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.239601 13349 raft_consensus.cc:993] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0: : Instructing follower 91c9fa98571441c8959020435c520d0b to start an election
I20260709 13:05:06.240149 13315 raft_consensus.cc:1081] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Signalling peer 91c9fa98571441c8959020435c520d0b to start an election
I20260709 13:05:06.241466 13032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d653af1be1a84530a8f8e3ed2365ce72" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:06.242198 13032 raft_consensus.cc:3055] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:06.242555 13032 raft_consensus.cc:740] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:06.242851 13029 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "adb982abf32d42c194842360d6a1fb46"
dest_uuid: "91c9fa98571441c8959020435c520d0b"
 from {username='slave'} at 127.0.0.1:35530
I20260709 13:05:06.243511 13029 raft_consensus.cc:493] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:06.243405 13032 consensus_queue.cc:260] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.244045 13029 raft_consensus.cc:3060] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.245909 13032 raft_consensus.cc:3060] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.247407 13227 consensus_queue.cc:1048] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:05:06.247658 13180 leader_election.cc:290] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 2 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:06.249531 13105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d653af1be1a84530a8f8e3ed2365ce72" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:06.250190 13105 raft_consensus.cc:3060] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.254288 13032 raft_consensus.cc:2468] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 2.
I20260709 13:05:06.255045 13029 raft_consensus.cc:515] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.255788 13145 leader_election.cc:304] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:06.256754 13336 raft_consensus.cc:2804] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:06.257258 13336 raft_consensus.cc:697] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [term 2 LEADER]: Becoming Leader. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Running, Role: LEADER
I20260709 13:05:06.258157 13336 consensus_queue.cc:237] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.261924 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb982abf32d42c194842360d6a1fb46" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:06.262652 13106 raft_consensus.cc:3055] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:06.263037 13106 raft_consensus.cc:740] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:06.263087 13105 raft_consensus.cc:2468] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 2.
I20260709 13:05:06.264516 13029 leader_election.cc:290] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 2 election: Requested vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:06.266288 13106 consensus_queue.cc:260] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.265504 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "adb982abf32d42c194842360d6a1fb46" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:06.267568 13106 raft_consensus.cc:3060] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.270089 13227 raft_consensus.cc:993] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0: : Instructing follower 91c9fa98571441c8959020435c520d0b to start an election
I20260709 13:05:06.270663 13227 raft_consensus.cc:1081] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Signalling peer 91c9fa98571441c8959020435c520d0b to start an election
I20260709 13:05:06.272507 13032 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "126207f102f0423a9b67d6b388cc4595"
dest_uuid: "91c9fa98571441c8959020435c520d0b"
 from {username='slave'} at 127.0.0.1:35530
I20260709 13:05:06.273077 13032 raft_consensus.cc:493] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:06.273499 13032 raft_consensus.cc:3060] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.277477 13106 raft_consensus.cc:2468] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 2.
I20260709 13:05:06.279445 12996 leader_election.cc:304] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:06.280745 13264 raft_consensus.cc:2804] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:06.281263 13264 raft_consensus.cc:697] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 2 LEADER]: Becoming Leader. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:06.281718 13032 raft_consensus.cc:515] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.282119 13264 consensus_queue.cc:237] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.284710 13032 leader_election.cc:290] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 2 election: Requested vote from peers 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:06.285171 13179 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "126207f102f0423a9b67d6b388cc4595" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:06.285866 13179 raft_consensus.cc:3060] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.286348 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "126207f102f0423a9b67d6b388cc4595" candidate_uuid: "91c9fa98571441c8959020435c520d0b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:06.287060 13106 raft_consensus.cc:3055] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:06.287519 13106 raft_consensus.cc:740] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:06.287122 12930 catalog_manager.cc:5697] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 reported cstate change: term changed from 1 to 2, leader changed from 91c9fa98571441c8959020435c520d0b (127.0.107.193) to 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "648954f023084ca68a9dc69c1bdbf372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:06.288525 13106 consensus_queue.cc:260] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.289961 13106 raft_consensus.cc:3060] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.293383 13179 raft_consensus.cc:2468] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 2.
I20260709 13:05:06.294780 12997 leader_election.cc:304] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 648954f023084ca68a9dc69c1bdbf372, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:06.296476 13264 raft_consensus.cc:2804] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:06.296274 12928 catalog_manager.cc:5697] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b reported cstate change: term changed from 1 to 2, leader changed from 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194) to 91c9fa98571441c8959020435c520d0b (127.0.107.193). New cstate: current_term: 2 leader_uuid: "91c9fa98571441c8959020435c520d0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:06.297039 13264 raft_consensus.cc:697] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [term 2 LEADER]: Becoming Leader. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:06.297760 13106 raft_consensus.cc:2468] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91c9fa98571441c8959020435c520d0b in term 2.
I20260709 13:05:06.298255 13264 consensus_queue.cc:237] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.310495 12929 catalog_manager.cc:5697] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b reported cstate change: term changed from 1 to 2, leader changed from 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194) to 91c9fa98571441c8959020435c520d0b (127.0.107.193). New cstate: current_term: 2 leader_uuid: "91c9fa98571441c8959020435c520d0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:06.442795 13179 raft_consensus.cc:3060] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.445680 13314 raft_consensus.cc:993] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372: : Instructing follower 1802553b3b0245be83aff21b0b5b23d0 to start an election
I20260709 13:05:06.446480 13337 raft_consensus.cc:1081] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Signalling peer 1802553b3b0245be83aff21b0b5b23d0 to start an election
I20260709 13:05:06.448984 13106 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce559953fc0d4764b0db5595e6fbc8bc"
dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
 from {username='slave'} at 127.0.0.1:43180
I20260709 13:05:06.449628 13106 raft_consensus.cc:493] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:06.449980 13106 raft_consensus.cc:3060] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.451164 13179 raft_consensus.cc:1275] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Refusing update from remote peer 1802553b3b0245be83aff21b0b5b23d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.452786 13315 consensus_queue.cc:1048] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.457206 13106 raft_consensus.cc:515] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.460611 13029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce559953fc0d4764b0db5595e6fbc8bc" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:06.461248 13029 raft_consensus.cc:3060] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.466684 13032 raft_consensus.cc:1275] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Refusing update from remote peer 1802553b3b0245be83aff21b0b5b23d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.468317 13315 consensus_queue.cc:1048] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.472407 13029 raft_consensus.cc:2468] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 2.
I20260709 13:05:06.474397 13071 leader_election.cc:304] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 91c9fa98571441c8959020435c520d0b; no voters: 
I20260709 13:05:06.474948 13106 leader_election.cc:290] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [CANDIDATE]: Term 2 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195:35167)
I20260709 13:05:06.476225 13315 raft_consensus.cc:2804] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:06.477126 13180 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce559953fc0d4764b0db5595e6fbc8bc" candidate_uuid: "1802553b3b0245be83aff21b0b5b23d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
I20260709 13:05:06.477898 13180 raft_consensus.cc:3055] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:06.478216 13180 raft_consensus.cc:740] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Running, Role: LEADER
I20260709 13:05:06.479408 13315 raft_consensus.cc:697] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [term 2 LEADER]: Becoming Leader. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:06.479166 13180 consensus_queue.cc:260] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.480377 13315 consensus_queue.cc:237] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:06.481666 13180 raft_consensus.cc:3060] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.490687 13180 raft_consensus.cc:2468] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1802553b3b0245be83aff21b0b5b23d0 in term 2.
I20260709 13:05:06.492841 12928 catalog_manager.cc:5697] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 reported cstate change: term changed from 1 to 2, leader changed from 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195) to 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194). New cstate: current_term: 2 leader_uuid: "1802553b3b0245be83aff21b0b5b23d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: UNKNOWN } } }
I20260709 13:05:06.499420 13180 raft_consensus.cc:1275] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Refusing update from remote peer 1802553b3b0245be83aff21b0b5b23d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.501040 13227 consensus_queue.cc:1048] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.510392 13032 raft_consensus.cc:1275] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Refusing update from remote peer 1802553b3b0245be83aff21b0b5b23d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.512370 13401 consensus_queue.cc:1048] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.575798 13032 raft_consensus.cc:1275] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Refusing update from remote peer 1802553b3b0245be83aff21b0b5b23d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.577327 13401 consensus_queue.cc:1048] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.587787 13180 raft_consensus.cc:1275] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Refusing update from remote peer 1802553b3b0245be83aff21b0b5b23d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.589371 13401 consensus_queue.cc:1048] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.724967 13180 raft_consensus.cc:3060] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:06.731385 13180 raft_consensus.cc:1275] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Refusing update from remote peer 91c9fa98571441c8959020435c520d0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.732968 13264 consensus_queue.cc:1048] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.743763 13105 raft_consensus.cc:1275] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Refusing update from remote peer 91c9fa98571441c8959020435c520d0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.745705 13264 consensus_queue.cc:1048] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.747126 13105 raft_consensus.cc:1275] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Refusing update from remote peer 91c9fa98571441c8959020435c520d0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.748692 13264 consensus_queue.cc:1048] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.750881 13106 raft_consensus.cc:1275] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Refusing update from remote peer 648954f023084ca68a9dc69c1bdbf372: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.753144 13337 consensus_queue.cc:1048] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:06.763412 13032 raft_consensus.cc:1275] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Refusing update from remote peer 648954f023084ca68a9dc69c1bdbf372: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.770001 13180 raft_consensus.cc:1275] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Refusing update from remote peer 91c9fa98571441c8959020435c520d0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.770256 13336 consensus_queue.cc:1048] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.773554 13369 consensus_queue.cc:1048] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.820727 13180 raft_consensus.cc:1275] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Refusing update from remote peer 91c9fa98571441c8959020435c520d0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.822095 13369 consensus_queue.cc:1048] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.830538 13106 raft_consensus.cc:1275] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Refusing update from remote peer 91c9fa98571441c8959020435c520d0b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.832607 13369 consensus_queue.cc:1048] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:06.912317 13032 raft_consensus.cc:1275] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Refusing update from remote peer 1802553b3b0245be83aff21b0b5b23d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.913890 13227 consensus_queue.cc:1048] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:06.924834 13180 raft_consensus.cc:1275] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Refusing update from remote peer 1802553b3b0245be83aff21b0b5b23d0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:06.926468 13349 consensus_queue.cc:1048] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:08.972882 12930 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39272:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:05:09.002951 12930 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:39272:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:05:09.073660   431 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:05:09.223068 13424 log.cc:927] Time spent T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b: Append to log took a long time: real 0.086s	user 0.000s	sys 0.003s
W20260709 13:05:09.308502 13442 log.cc:927] Time spent T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372: Append to log took a long time: real 0.126s	user 0.000s	sys 0.002s
W20260709 13:05:09.336174 13427 log.cc:927] Time spent T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b: Append to log took a long time: real 0.183s	user 0.000s	sys 0.006s
I20260709 13:05:09.350528 13455 tablet.cc:1724] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.378295 13462 tablet.cc:1724] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.383255 13463 tablet.cc:1724] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.385159 13459 tablet.cc:1724] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.389461 13460 tablet.cc:1724] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:05:09.391485 13448 log.cc:927] Time spent T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:05:09.403045 13447 log.cc:927] Time spent T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 13:05:09.420954 13461 tablet.cc:1724] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:05:09.422914 13450 log.cc:927] Time spent T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 13:05:09.434939 13441 log.cc:927] Time spent T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372: Append to log took a long time: real 0.147s	user 0.000s	sys 0.003s
I20260709 13:05:09.468067 13469 tablet.cc:1724] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:05:09.517133 12997 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.195:35167, user_credentials={real_user=slave}} blocked reactor thread for 55252.7us
I20260709 13:05:09.468847 13465 tablet.cc:1724] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.550748 13472 tablet.cc:1724] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:05:09.564026 12996 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.0.107.194:36541, user_credentials={real_user=slave}} blocked reactor thread for 68970.7us
I20260709 13:05:09.573798 13460 tablet.cc:1724] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.580642 13471 tablet.cc:1724] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.589947 13475 tablet.cc:1724] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.597369 13478 tablet.cc:1724] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.618062 13106 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b344a32e631c427f9275b5d0b7543901"
dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
mode: GRACEFUL
new_leader_uuid: "648954f023084ca68a9dc69c1bdbf372"
 from {username='slave'} at 127.0.0.1:34110
I20260709 13:05:09.619051 13106 raft_consensus.cc:606] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Received request to transfer leadership to TS 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:09.628683 13440 raft_consensus.cc:993] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0: : Instructing follower 648954f023084ca68a9dc69c1bdbf372 to start an election
I20260709 13:05:09.635241 13460 tablet.cc:1724] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.629374 13440 raft_consensus.cc:1081] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Signalling peer 648954f023084ca68a9dc69c1bdbf372 to start an election
I20260709 13:05:09.649933 13181 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b344a32e631c427f9275b5d0b7543901"
dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
 from {username='slave'} at 127.0.0.1:34526
I20260709 13:05:09.650684 13181 raft_consensus.cc:493] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:09.651587 13181 raft_consensus.cc:3060] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:09.651151 13461 tablet.cc:1724] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.652047 13462 tablet.cc:1724] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.669665 13181 raft_consensus.cc:515] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:09.686537 13459 tablet.cc:1724] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.687376 13463 tablet.cc:1724] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.696328 13455 tablet.cc:1724] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.702008 13465 tablet.cc:1724] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.705299 13463 tablet.cc:1724] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.721582 13029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344a32e631c427f9275b5d0b7543901" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:09.722426 13029 raft_consensus.cc:3060] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:09.729923 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b344a32e631c427f9275b5d0b7543901" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:09.730836 13106 raft_consensus.cc:3055] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:09.731256 13106 raft_consensus.cc:740] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1802553b3b0245be83aff21b0b5b23d0, State: Running, Role: LEADER
I20260709 13:05:09.732709 13106 consensus_queue.cc:260] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:09.727929 13181 leader_election.cc:290] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 2 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:09.734251 13106 raft_consensus.cc:3060] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:09.741971 13472 tablet.cc:1724] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.745708 13030 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ce472f3576f0423bba225ef4d6719f73"
dest_uuid: "91c9fa98571441c8959020435c520d0b"
mode: GRACEFUL
new_leader_uuid: "648954f023084ca68a9dc69c1bdbf372"
 from {username='slave'} at 127.0.0.1:38486
I20260709 13:05:09.749625 13030 raft_consensus.cc:606] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Received request to transfer leadership to TS 648954f023084ca68a9dc69c1bdbf372
I20260709 13:05:09.752837   431 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:05:09.753594   431 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:05:09.766122 13462 tablet.cc:1724] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.768961 13029 raft_consensus.cc:2468] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 2.
I20260709 13:05:09.773027 13106 raft_consensus.cc:2468] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 2.
I20260709 13:05:09.774767 13145 leader_election.cc:304] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
I20260709 13:05:09.774812 13478 tablet.cc:1724] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.775787 13464 raft_consensus.cc:2804] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:09.775252 13459 tablet.cc:1724] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.776427 13464 raft_consensus.cc:697] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [term 2 LEADER]: Becoming Leader. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Running, Role: LEADER
I20260709 13:05:09.777539 13464 consensus_queue.cc:237] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:09.769737 13469 tablet.cc:1724] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.780869 13475 tablet.cc:1724] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:05:09.822767 12929 catalog_manager.cc:6242] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36: soft_deleted_table [id=be4b0fe9babb4d41b94de1d1e20c5c1d] alter complete (version 1)
I20260709 13:05:09.822861 12928 catalog_manager.cc:5697] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 reported cstate change: term changed from 1 to 2, leader changed from 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194) to 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "648954f023084ca68a9dc69c1bdbf372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:10.037961 13428 raft_consensus.cc:993] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b: : Instructing follower 648954f023084ca68a9dc69c1bdbf372 to start an election
I20260709 13:05:10.038628 13435 raft_consensus.cc:1081] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Signalling peer 648954f023084ca68a9dc69c1bdbf372 to start an election
I20260709 13:05:10.040171 13181 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ce472f3576f0423bba225ef4d6719f73"
dest_uuid: "648954f023084ca68a9dc69c1bdbf372"
 from {username='slave'} at 127.0.0.1:34540
I20260709 13:05:10.040726 13181 raft_consensus.cc:493] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:05:10.041118 13181 raft_consensus.cc:3060] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:10.047015 13181 raft_consensus.cc:515] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:10.048712 13181 leader_election.cc:290] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 2 election: Requested vote from peers 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339), 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541)
I20260709 13:05:10.049917 13029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce472f3576f0423bba225ef4d6719f73" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91c9fa98571441c8959020435c520d0b"
I20260709 13:05:10.050105 13106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce472f3576f0423bba225ef4d6719f73" candidate_uuid: "648954f023084ca68a9dc69c1bdbf372" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1802553b3b0245be83aff21b0b5b23d0"
I20260709 13:05:10.050789 13029 raft_consensus.cc:3055] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:05:10.050794 13106 raft_consensus.cc:3060] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:10.051383 13029 raft_consensus.cc:740] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 91c9fa98571441c8959020435c520d0b, State: Running, Role: LEADER
I20260709 13:05:10.052376 13029 consensus_queue.cc:260] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:10.053637 13029 raft_consensus.cc:3060] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:05:10.058200 13106 raft_consensus.cc:2468] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 2.
I20260709 13:05:10.059566 13145 leader_election.cc:304] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1802553b3b0245be83aff21b0b5b23d0, 648954f023084ca68a9dc69c1bdbf372; no voters: 
I20260709 13:05:10.059971 13029 raft_consensus.cc:2468] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 648954f023084ca68a9dc69c1bdbf372 in term 2.
I20260709 13:05:10.060421 13464 raft_consensus.cc:2804] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:05:10.061030 13464 raft_consensus.cc:697] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 2 LEADER]: Becoming Leader. State: Replica: 648954f023084ca68a9dc69c1bdbf372, State: Running, Role: LEADER
I20260709 13:05:10.062199 13464 consensus_queue.cc:237] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } }
I20260709 13:05:10.070749 12929 catalog_manager.cc:5697] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 reported cstate change: term changed from 1 to 2, leader changed from 91c9fa98571441c8959020435c520d0b (127.0.107.193) to 648954f023084ca68a9dc69c1bdbf372 (127.0.107.195). New cstate: current_term: 2 leader_uuid: "648954f023084ca68a9dc69c1bdbf372" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "648954f023084ca68a9dc69c1bdbf372" member_type: VOTER last_known_addr { host: "127.0.107.195" port: 35167 } health_report { overall_health: HEALTHY } } }
I20260709 13:05:10.186012 13106 raft_consensus.cc:1275] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Refusing update from remote peer 648954f023084ca68a9dc69c1bdbf372: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:10.187443 13336 consensus_queue.cc:1048] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:10.194972 13029 raft_consensus.cc:1275] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Refusing update from remote peer 648954f023084ca68a9dc69c1bdbf372: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:10.196532 13336 consensus_queue.cc:1048] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:10.494264 13106 raft_consensus.cc:1275] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Refusing update from remote peer 648954f023084ca68a9dc69c1bdbf372: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:10.495533 13336 consensus_queue.cc:1048] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1802553b3b0245be83aff21b0b5b23d0" member_type: VOTER last_known_addr { host: "127.0.107.194" port: 36541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:05:10.506716 13029 raft_consensus.cc:1275] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Refusing update from remote peer 648954f023084ca68a9dc69c1bdbf372: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:05:10.509063 13488 consensus_queue.cc:1048] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91c9fa98571441c8959020435c520d0b" member_type: VOTER last_known_addr { host: "127.0.107.193" port: 40339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:05:10.822113   431 tablet_server.cc:179] TabletServer@127.0.107.193:0 shutting down...
I20260709 13:05:10.857062   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:10.858536   431 tablet_replica.cc:333] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
W20260709 13:05:10.859066 13071 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 82 similar messages]
I20260709 13:05:10.859649   431 raft_consensus.cc:2243] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.860376   431 raft_consensus.cc:2272] T d653af1be1a84530a8f8e3ed2365ce72 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:10.862435 13071 consensus_peers.cc:597] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 -> Peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Couldn't send request to peer 91c9fa98571441c8959020435c520d0b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:10.863534   431 tablet_replica.cc:333] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.864506   431 raft_consensus.cc:2243] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.865058   431 raft_consensus.cc:2272] T a13fa28d81b3452597c141ed18a32b32 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.867437   431 tablet_replica.cc:333] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.868130   431 raft_consensus.cc:2243] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.868688   431 raft_consensus.cc:2272] T 7a4e244cb05a427dac7505e343db9c46 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.870961   431 tablet_replica.cc:333] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.871555   431 raft_consensus.cc:2243] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.871989   431 raft_consensus.cc:2272] T ac66224c363f48a49e1af00db21c47cf P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.874265   431 tablet_replica.cc:333] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.874814   431 raft_consensus.cc:2243] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:10.875674   431 raft_consensus.cc:2272] T 7de68b718e464371bc15d4b3b8fd082e P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.878031   431 tablet_replica.cc:333] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.878664   431 raft_consensus.cc:2243] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.879169   431 raft_consensus.cc:2272] T 97da23ee368d48ddbe16515fc5dceff8 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.881837   431 tablet_replica.cc:333] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.882513   431 raft_consensus.cc:2243] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:10.883405   431 raft_consensus.cc:2272] T 54b6273b4d944500bc90ab3c2aa15c91 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.886284   431 tablet_replica.cc:333] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.887029   431 raft_consensus.cc:2243] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.887650   431 raft_consensus.cc:2272] T f047c82cb27e4832861399c9c33043f7 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.890112   431 tablet_replica.cc:333] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.890875   431 raft_consensus.cc:2243] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:10.892211   431 raft_consensus.cc:2272] T 1862ebf0bab3451c9059f05dd1420b46 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.895035   431 tablet_replica.cc:333] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.895900   431 raft_consensus.cc:2243] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:10.897073   431 raft_consensus.cc:2272] T adb982abf32d42c194842360d6a1fb46 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.899536   431 tablet_replica.cc:333] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.900199   431 raft_consensus.cc:2243] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:10.901070   431 raft_consensus.cc:2272] T 126207f102f0423a9b67d6b388cc4595 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.904083   431 tablet_replica.cc:333] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.904742   431 raft_consensus.cc:2243] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.905234   431 raft_consensus.cc:2272] T ce472f3576f0423bba225ef4d6719f73 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.908581   431 tablet_replica.cc:333] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.909333   431 raft_consensus.cc:2243] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.910061   431 raft_consensus.cc:2272] T b344a32e631c427f9275b5d0b7543901 P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.914019   431 tablet_replica.cc:333] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.915023   431 raft_consensus.cc:2243] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.915565   431 raft_consensus.cc:2272] T ce559953fc0d4764b0db5595e6fbc8bc P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.918313   431 tablet_replica.cc:333] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.918946   431 raft_consensus.cc:2243] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.919453   431 raft_consensus.cc:2272] T 90d7b9d3a73b468ea5499974cdfb073e P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.921861   431 tablet_replica.cc:333] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.922489   431 raft_consensus.cc:2243] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:10.923424   431 raft_consensus.cc:2272] T aa2c897f99564a1eaf2b475b8094cbaf P 91c9fa98571441c8959020435c520d0b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.926092   431 tablet_replica.cc:333] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.926693   431 raft_consensus.cc:2243] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:10.927557   431 raft_consensus.cc:2272] T da04db13d6144ff084cbd7fa1daee828 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:10.930097   431 tablet_replica.cc:333] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b: stopping tablet replica
I20260709 13:05:10.930655   431 raft_consensus.cc:2243] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:10.931121   431 raft_consensus.cc:2272] T ac72e4f8e9dd4531847ba293bec2f7b0 P 91c9fa98571441c8959020435c520d0b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:10.965788 12995 connection.cc:466] server connection from 127.0.0.1:35530 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35530 (request call id 156) could send its response
W20260709 13:05:10.968498 13071 consensus_peers.cc:597] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 -> Peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Couldn't send request to peer 91c9fa98571441c8959020435c520d0b. Status: Network error: recv got EOF from 127.0.107.193:40339 (error 108). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:10.976385 13145 consensus_peers.cc:597] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 -> Peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Couldn't send request to peer 91c9fa98571441c8959020435c520d0b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:10.990957   431 tablet_server.cc:196] TabletServer@127.0.107.193:0 shutdown complete.
I20260709 13:05:11.036485   431 tablet_server.cc:179] TabletServer@127.0.107.194:0 shutting down...
W20260709 13:05:11.055513 13071 consensus_peers.cc:597] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 -> Peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Couldn't send request to peer 91c9fa98571441c8959020435c520d0b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:11.066251 13145 consensus_peers.cc:597] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 -> Peer 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541): Couldn't send request to peer 1802553b3b0245be83aff21b0b5b23d0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:11.072919   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:05:11.073400 13071 consensus_peers.cc:597] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 -> Peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Couldn't send request to peer 91c9fa98571441c8959020435c520d0b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:11.073977   431 tablet_replica.cc:333] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.074764   431 raft_consensus.cc:2243] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.075292   431 raft_consensus.cc:2272] T d653af1be1a84530a8f8e3ed2365ce72 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.077500   431 tablet_replica.cc:333] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.078120   431 raft_consensus.cc:2243] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:11.079064   431 raft_consensus.cc:2272] T f047c82cb27e4832861399c9c33043f7 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.082293   431 tablet_replica.cc:333] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.083388   431 raft_consensus.cc:2243] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.084024   431 raft_consensus.cc:2272] T 1862ebf0bab3451c9059f05dd1420b46 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:11.087220 13071 consensus_peers.cc:597] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 -> Peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Couldn't send request to peer 91c9fa98571441c8959020435c520d0b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:11.087448   431 tablet_replica.cc:333] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.088302   431 raft_consensus.cc:2243] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:05:11.088557 13145 consensus_peers.cc:597] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 -> Peer 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541): Couldn't send request to peer 1802553b3b0245be83aff21b0b5b23d0. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:11.088960   431 raft_consensus.cc:2272] T da04db13d6144ff084cbd7fa1daee828 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.091413   431 tablet_replica.cc:333] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.092058   431 raft_consensus.cc:2243] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:11.093021   431 raft_consensus.cc:2272] T 97da23ee368d48ddbe16515fc5dceff8 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.095640   431 tablet_replica.cc:333] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.096568   431 raft_consensus.cc:2243] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.097282   431 raft_consensus.cc:2272] T 54b6273b4d944500bc90ab3c2aa15c91 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.099650   431 tablet_replica.cc:333] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.100279   431 raft_consensus.cc:2243] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.100835   431 raft_consensus.cc:2272] T 7de68b718e464371bc15d4b3b8fd082e P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.103205   431 tablet_replica.cc:333] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.103834   431 raft_consensus.cc:2243] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.104315   431 raft_consensus.cc:2272] T aa2c897f99564a1eaf2b475b8094cbaf P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.107156   431 tablet_replica.cc:333] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.107966   431 raft_consensus.cc:2243] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:11.109184   431 raft_consensus.cc:2272] T a13fa28d81b3452597c141ed18a32b32 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.112366   431 tablet_replica.cc:333] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.113529   431 raft_consensus.cc:2243] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:11.114512   431 raft_consensus.cc:2272] T 7a4e244cb05a427dac7505e343db9c46 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.116715   431 tablet_replica.cc:333] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.117467   431 raft_consensus.cc:2243] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:11.118451   431 raft_consensus.cc:2272] T ac66224c363f48a49e1af00db21c47cf P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.120808   431 tablet_replica.cc:333] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.121511   431 raft_consensus.cc:2243] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.122012   431 raft_consensus.cc:2272] T adb982abf32d42c194842360d6a1fb46 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.124197   431 tablet_replica.cc:333] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.124766   431 raft_consensus.cc:2243] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.125255   431 raft_consensus.cc:2272] T 126207f102f0423a9b67d6b388cc4595 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.127343   431 tablet_replica.cc:333] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.127930   431 raft_consensus.cc:2243] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.128381   431 raft_consensus.cc:2272] T ce472f3576f0423bba225ef4d6719f73 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.131729   431 tablet_replica.cc:333] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.132522   431 raft_consensus.cc:2243] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.133067   431 raft_consensus.cc:2272] T b344a32e631c427f9275b5d0b7543901 P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.136309   431 tablet_replica.cc:333] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.136946   431 raft_consensus.cc:2243] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:11.138085   431 raft_consensus.cc:2272] T ce559953fc0d4764b0db5595e6fbc8bc P 1802553b3b0245be83aff21b0b5b23d0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.140761   431 tablet_replica.cc:333] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.141346   431 raft_consensus.cc:2243] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.141952   431 raft_consensus.cc:2272] T 90d7b9d3a73b468ea5499974cdfb073e P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.144295   431 tablet_replica.cc:333] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0: stopping tablet replica
I20260709 13:05:11.145175   431 raft_consensus.cc:2243] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:11.146104   431 raft_consensus.cc:2272] T ac72e4f8e9dd4531847ba293bec2f7b0 P 1802553b3b0245be83aff21b0b5b23d0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:05:11.163363 13145 consensus_peers.cc:597] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 -> Peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Couldn't send request to peer 91c9fa98571441c8959020435c520d0b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:11.194231   431 tablet_server.cc:196] TabletServer@127.0.107.194:0 shutdown complete.
W20260709 13:05:11.195667 13145 consensus_peers.cc:597] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 -> Peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Couldn't send request to peer 91c9fa98571441c8959020435c520d0b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:11.233224   431 tablet_server.cc:179] TabletServer@127.0.107.195:0 shutting down...
W20260709 13:05:11.259045 13145 consensus_peers.cc:597] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 -> Peer 1802553b3b0245be83aff21b0b5b23d0 (127.0.107.194:36541): Couldn't send request to peer 1802553b3b0245be83aff21b0b5b23d0. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.194:36541: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:05:11.269934 13145 consensus_peers.cc:597] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 -> Peer 91c9fa98571441c8959020435c520d0b (127.0.107.193:40339): Couldn't send request to peer 91c9fa98571441c8959020435c520d0b. Status: Network error: Client connection negotiation failed: client connection to 127.0.107.193:40339: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:05:11.272773   431 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:05:11.273759   431 tablet_replica.cc:333] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.274514   431 raft_consensus.cc:2243] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.275128   431 raft_consensus.cc:2272] T adb982abf32d42c194842360d6a1fb46 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.278469   431 tablet_replica.cc:333] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.279261   431 raft_consensus.cc:2243] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:11.280287   431 raft_consensus.cc:2272] T d653af1be1a84530a8f8e3ed2365ce72 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.283406   431 tablet_replica.cc:333] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.284121   431 raft_consensus.cc:2243] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.284636   431 raft_consensus.cc:2272] T 126207f102f0423a9b67d6b388cc4595 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.286787   431 tablet_replica.cc:333] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.287371   431 raft_consensus.cc:2243] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:11.288200   431 raft_consensus.cc:2272] T b344a32e631c427f9275b5d0b7543901 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.290683   431 tablet_replica.cc:333] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.291260   431 raft_consensus.cc:2243] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:05:11.292076   431 raft_consensus.cc:2272] T ce472f3576f0423bba225ef4d6719f73 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.295177   431 tablet_replica.cc:333] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.295815   431 raft_consensus.cc:2243] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.296667   431 raft_consensus.cc:2272] T ac72e4f8e9dd4531847ba293bec2f7b0 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.299038   431 tablet_replica.cc:333] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.299655   431 raft_consensus.cc:2243] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.300153   431 raft_consensus.cc:2272] T 7de68b718e464371bc15d4b3b8fd082e P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.302343   431 tablet_replica.cc:333] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.302902   431 raft_consensus.cc:2243] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.303376   431 raft_consensus.cc:2272] T 97da23ee368d48ddbe16515fc5dceff8 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.305475   431 tablet_replica.cc:333] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.306083   431 raft_consensus.cc:2243] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.306574   431 raft_consensus.cc:2272] T 54b6273b4d944500bc90ab3c2aa15c91 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.308701   431 tablet_replica.cc:333] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.309317   431 raft_consensus.cc:2243] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.309870   431 raft_consensus.cc:2272] T 7a4e244cb05a427dac7505e343db9c46 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.311981   431 tablet_replica.cc:333] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.312580   431 raft_consensus.cc:2243] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.313057   431 raft_consensus.cc:2272] T a13fa28d81b3452597c141ed18a32b32 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.315382   431 tablet_replica.cc:333] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.316001   431 raft_consensus.cc:2243] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.316488   431 raft_consensus.cc:2272] T ac66224c363f48a49e1af00db21c47cf P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.318997   431 tablet_replica.cc:333] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.319607   431 raft_consensus.cc:2243] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.320159   431 raft_consensus.cc:2272] T f047c82cb27e4832861399c9c33043f7 P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.322402   431 tablet_replica.cc:333] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.322940   431 raft_consensus.cc:2243] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.323408   431 raft_consensus.cc:2272] T 1862ebf0bab3451c9059f05dd1420b46 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.325665   431 tablet_replica.cc:333] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.326215   431 raft_consensus.cc:2243] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.326709   431 raft_consensus.cc:2272] T ce559953fc0d4764b0db5595e6fbc8bc P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.328693   431 tablet_replica.cc:333] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.329248   431 raft_consensus.cc:2243] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:11.330114   431 raft_consensus.cc:2272] T 90d7b9d3a73b468ea5499974cdfb073e P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.332341   431 tablet_replica.cc:333] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.332852   431 raft_consensus.cc:2243] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.333312   431 raft_consensus.cc:2272] T da04db13d6144ff084cbd7fa1daee828 P 648954f023084ca68a9dc69c1bdbf372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.335472   431 tablet_replica.cc:333] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372: stopping tablet replica
I20260709 13:05:11.336038   431 raft_consensus.cc:2243] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:05:11.336489   431 raft_consensus.cc:2272] T aa2c897f99564a1eaf2b475b8094cbaf P 648954f023084ca68a9dc69c1bdbf372 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.379565   431 tablet_server.cc:196] TabletServer@127.0.107.195:0 shutdown complete.
I20260709 13:05:11.419734   431 master.cc:562] Master@127.0.107.254:37535 shutting down...
I20260709 13:05:11.453573   431 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:05:11.454156   431 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:05:11.454486   431 tablet_replica.cc:333] T 00000000000000000000000000000000 P ce544662cc0b4e3fbcdfe294229a8b36: stopping tablet replica
I20260709 13:05:11.477350   431 master.cc:584] Master@127.0.107.254:37535 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (11744 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (27115 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (376079 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 13:05:11.536928   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 535 messages since previous log ~569 seconds ago
I20260709 13:05:11.537190   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 535 messages since previous log ~569 seconds ago
I20260709 13:05:11.537376   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 527 messages since previous log ~569 seconds ago
I20260709 13:05:11.537643   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~36 seconds ago
I20260709 13:05:11.537837   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~36 seconds ago
I20260709 13:05:11.538074   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 3 messages since previous log ~138 seconds ago
I20260709 13:05:11.538254   431 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 1 messages since previous log ~285 seconds ago
I20260709 13:05:11.538530   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 11 messages since previous log ~0 seconds ago
I20260709 13:05:11.538834   431 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~10 seconds ago
